The previous burst is doing a program module, its basic function is to read a file specified format, then organize the contents of the file into memory into memory. In this development task, we only need to generate memory data. Yes, how data is handled by other groups, so the data structure is also defined by the other party. After a series of design, development, testing, we successfully complete the development of the module on Windows and is also integrated into their procedures by other groups. However, the Bug ticket is not yet long ago, saying that our module runs in Solaris to crash, which may affect the entire software. . . . As the main design and development person of this module, I quickly debug this module, but it is imposing that the entire debugging process can only be performed under Windows because of our unit unable to build the program debugging environment under Solaris. Compile, the final execution program observes its effects. After careful investigation, I am sure that the procedure does not have any problems, but it is really a problem that it will collapse on Solaris. After that, I will ask a C master, which is roughly determined that this belongs to the program's memory is alignment. The wrong mistake, then, after a week of repeated modification, we finally solved this bug, let's introduce the beginning of this bug below.
First, the data we have to build is a tree structure that is unfolded with Structs, which is assumed to have the following types (actual system to be more complicated. For convenience, it is simplified): typedef struct db {module * * Modulelist; / * Dynamic list of modules * /} DB;
Typedef struct module {structure; / * dynamic list of instances * /} module;
Typedef struct instant {char * name; / * instance name * /} institution
Their actual structure is as follows:
As you can see, the final result data will have a total DB node, and its moduleList will point to a series of Module nodes, and it is not directly pointing to the Module node, but through a module * array, then find Really Module node. There is instlist in the same module, which can be used to specify institutional INSTs under Module, which are also interconnected by an array of instates. Obviously, the two arrays used above must be in the continuous space, but because the entire memory DB is established by reading the content of the file content, how do you know how many of your moduleList and instlist? Data, the answer is the int LENGTH data above the module * array (and the instant), which is placed on the MODULELIST pointer to the above, with an INT type value to represent the length of the array. When you need to traverse the modulelist under DB, you only need to get an array length according to the ModuleList pointer, and you can access the array length as you can access the general array. Speaking here, I have to admire the designer's conceivement. Everyone knows that the speed of arrays is the fastest in the list access, but the maximum disadvantage of the array is that the data cannot be dynamically added (unless you use realloc () to reassign the memory. ), And now use the current structure, the dynamic data is organized into an array form, which will be very efficient for data access. But this structure has made a big problem for our generation data, we can't know how much data has been put in the actual file. If you do a similar system experience, you read a data append one, then repeated Memory distribution is definitely not avoided, so after research, we decided to adopt this method: pre-traverse the entire number of documents, count the total number of various types of data (ok, generally each type of text, statistics Very convenient), then assign all the required memory, then start reading the file from the header, reading the file, and assign space from the pre-allocated memory to the object to be generated. The entire processing process is still relatively simple, suppose we already have a DB node, in the initial state, its moduleList pointer is null, we pre-allocated to Module's memory block called moduleblock (Byte * type), there is one ModuleBlockcur is used to indicate what the current memory is allocated, and the same location has instablock and instablockcur. When you need to add an INST to its moduleList: 1) Determine whether the ModuleList pointer is empty. If it is empty, you know that this is the first data in the array, then the current location of ModuleBlock first A spatial space is divided, and it is assigned to the value zero, then change the ModuleList pointer, but it points to the location of the int type space.
Code is as follows: if (* pModuleList == NULL) {// create an new module list int * length = (int *) (ModuleBlock ModuleBlockCur); * length = 0; mCurDBMem-> ModuleBlockCur = sizeof (int); M_ASSERT (mCurDBMem-> ModuleBlockCur <= mCurDBMem-> ModuleBlockSize); * pModuleList = (Module **) (mCurDBMem-> ModuleBlock mCurDBMem-> ModuleBlockCur);} in this way, when the initialization is completed moduleList first distribution function.
2) Use the moduleList pointer (after step 1, the ModuleList pointer must be a valid address) move upward, you can take the length of the current array, add 1 value, starting from the current location of ModuleBlock It is divided into a SizeOf (Module *) length, as module *, which is equivalent to increasing the length of the array.
3) Assign a MODULE type of memory, pointing the newly allocated Module pointer to the module. This completes the function of allocating Module and adds the list of lists.
The above two steps is: int * length = (int *) ((t_int8 *) * PModuleList - sizeof (int)); // Get the modulelist's length module ** ppmodule = (Module **) (ModuleBlock ModuleBlockcur) ; // get new memory address ModuleBlockCur = sizeof (Module *); M_ASSERT (ModuleBlockCur <= ModuleBlockSize); * length = 1; * ppmodule = (Module *) AllocMem (sizeof (Module)); // allocate module
The function of allocmem can be regarded as like Malloc (), but in order to manage the system's performance, we use the method of managing the memory block list to handle these scattered memory, here is not described in detail.
Through the above steps, all module * can be assigned in a continuous memory. The same Module is also built in consecutive memory with a similar method, and there is no big loss on the continuous memory. After the development is complete, we deliver the module to the other party, and adopted various tests (of course, only the Windows platform), prove that such a practice is quite robust and efficient.
However, when Solaris crashed under the problem, after this last week, I realized that the fact that it was a quite serious mistake: I have all the above deals, it is built. On the Windows platform, since the Windows platform is a 32-bit operating system, its data pointer has a length of 4Byte, and the length of the INT type value is also 4Byte, then the MODULELIST I have generated will be like this. (Suppose MODULELIST points to 0x0004): It can be seen that the space occupied by the INT-shaped Length field is the same as the space occupied by the pointer Module * field, which is just a coincidence. If not use INT type as Length Type, but use byte type, assuming the address or 0x0004 of ModuleList, then the start address of Length turns 0x0003, under the Windows system, this will not be wrong, this is due to the powerful compatibility of the Windows platform.
Let's take a look at the memory distribution under Solaris. We use a 64-bit Solaris system. Under the system, the length of the INT type is still 4Byte, but the length of the pointer is 16Byte (note !!), then after I have the above process After that, the memory will change (or assuming the address of ModuleList is 0x0004):
The memory distribution is indeed different (red marks different memory addresses), but this should be normal, at least I used to develop, I have been developed, but I think so. . . . But I am sorry, this memory distribution will trigger alignment errors on the Solaris system. Simply, the Solaris system must ensure the efficiency of the chip operation, all pointers must be within the integer time of the 16th multiplier address (unless special Directive), otherwise, when you want to access the pointer points to the data, it will cause access to an error. In general, the space allocated with malloc under the Solaris system will be on the integer multiple of the 16, and the ModuleBlock we all overweight is also like this. However, when the moduleblock is digging, it is used to assign to the INT type Length, and the address assigned to module * is obviously not a total of 16, and at this time, we just use a mandatory type conversion. In order to point the pointer to the illegal address for the Solaris system, so when the subsequent code is to access the memory, the program crashed. . . .
I know the problem, and the solution is of course not difficult. We just guarantee that Module * is allocated to the integer multiple memory addresses, well, after modification, our procedures have made the following modifications: 1) Let ModuleBlock Module ** type, ModuleBlockcur is no longer used to represent memory offset, but it is used to indicate how many module * spaces currently allocated (these modifications are unrelated to function, but the code is clearer, this is also Reconstruction. Whenever you need to assign new MODULIST, we don't assign an int type space, but allocate a Module * type space (Module * is a pointer type, under the Solaris platform, it is larger than the INT type 4Byte, in the Windows platform Next, the same is the same as the INT type). 2) The original use of the allocmem () function (although there is no detailed introduction, it is actually assigned by the various scattered structs and char * strings that the entire program needs to be prepared in advance, char * characters. The string is unregulated, and the result of allocating memory is to assign memory to the Struct, which may not be assigned on the 16 integer multiple memory addresses, which will also cause an Alignment error, so for all kinds of Struct Memory allocation, by using another function allocstructMem () is responsible for allocating Struct's memory, because there is no compact Struct compilation instruction, so each Struct is also 16 integer times, it will not affect the next one Struct assignment.) Modify to use the allocstructMem () function. 3) Change the function to Template so that multiple block calls, code becomes like this: Template
M_assert (PBLockCur <= PBLOCKSIZE); PLIST = PBLOCK PBLOCKCUR;
Char * Pplength = (CHAR *) PLIST) - SIZEOF (Int); Int Length; Memcpy (& Length); Length ; Memcpy (Pplength, & Length, SizeOf (int)); type ** PPData = PBLOCK PBLOCKCUR; PBLOCKCUR ; M_ASSERT (PBLockCur <= PBLOCKSIZE); * PPDATA = (Type *) AllocstructMem (SizeOf (Type)); Return * Ppdata;
When you assign memory to Module: Module * PModule = AllocGeneralmem (ModuleBlock, ModuleBlockcur, ModuleBlocksize, PModuleList); M_ASSERT (PModule);
Ok, go to Solaris to compile, run, the program is no longer collapsed (at that time, I didn't collapse after I saw the program after nearly 4 days, I was excited to get out of the chair !!) . However, it seems still not very correct, because the same test file, there should be graphics in Windows, but nothing is displayed under Solaris. The only thing we can generate DB memory data is read when you are read by the external program, how can you, how can you go see how the external program is dry, in the code, a mad, suddenly discovered There is such two macros in the data definition file to our data: / * ================================== ============================================================================================== * Dynamic List * * Struct MEMBERS NAMED . "* List" point to a list of objects plus the * list length (before the first array element) All Lists must be built up * like AnyList * Eventually we should check the pointer offset with:. * assert ((((char *) NULL) - ((Char *) & ((Struct Anylist *) NULL) -> Entry)))% * SizeOf (int) == 0) * ============== ============================================================================================================================================================================================================= ============ * / struct Anylist {Int length; void * entry [1];}; # define intoff ((int *) NULL) - ((int *) & (((INT *) & (((INT *) & Struct Anylist *) NULL) -> Entry)) # Define ZlistLength ((INT *) ((INT *)) [INTOFF])
God, it's too big! The other party actually has a consideration of data alignment, so the two macros have been written, and their use will not have any platform differences, but I have been discovered these two baby macros now. . . I will simply analyze: struct anylist, it has two fields: length and entry, what is actually our moduleList (instlist) pointing to Length data above the memory, Entry is a Void pointer, because there is no compilation instruction here, therefore When compiling each of the builds, each field in Struct can be quickly accessed to the address in the memory, on Windows, the default is 8byte, that is, Length occupies 4Byte, then there will be 4Byte free. The space is followed by the Entry. And under Solaris, Length also 4byte, and the alignment is 16byte, so there will be 12Byte's idle space, then it is Entry. Such struct definitions will have different sizes on different systems, while the distance between Length and Entry will also be different. Then, INTOFF, by migration of the same data (NULL) to the Struct Anylist's address space, you can know that under the current system, their spacing. The last macro: ZListlength, the specified list, the offset calculated by intoff, the address of the INT LENGTH is found, and the data is taken out, and the length of the List can be calculated using this macro. And my code is written by calculating the length of the List: INT * Length = (int *) ((byte *) * ModuleList - sizeof (int)); see what is the difference? . . . Yes, under Windows, the results of these two ways to take Length are the same, but under Solaris is turned this:
My program stores and acquires Length from the orange address in the figure, and the other party gets length from the green address in the figure, and the result is of course the other person's program, it is considered to be 0 pull (I have cleared the whole memory, " Otherwise, it will be an uncertain value). Hey, I have to admire the other party's designer.
Well, continue to modify the code, this becomes: template
INT * PLENGTH = & ((int *)) [INTOFF]); M_ASSERT (PLIST) == * PLENGTH); * PLENGTH = * PLENGTH 1; type ** ppdata = PBLOCK PBLOCKCUR; PBLOCKCUR ; M_ASSERT (pBlockCur <= pBlockSize); * ppdata = (TYPE *) AllocStructMem (sizeof (TYPE)); return * ppdata;} Module * pmodule = AllocGeneralMem (ModuleBlock, ModuleBlockCur, ModuleBlockSize, pModuleList); M_ASSERT (pmodule) ;
It also uses the above INTOFF macro to get the address of Length. Return to Solaris again and run. . . . Oh, the result is finally complete!
After this whole week of bug modification, I learned a lot of compiler and operating system knowledge. On the other hand, I also hone my own intact in the shell, huh, huh, really, really harvest less. I also realized that I have to learn too much in technology. . . .
Welcome to Taxus software: http://www.tonixsoft.com
