Deep exploration Win32 executable format
Matt pietrek translation: Jiang Qingdong
Summary: The in-depth understanding of executables will take you deep into the system depth. If you know what stuff is your EXE / DLL, you are a more knowledgeable programmer. As a series of articles, it will pay attention to changes in PE formats in the past few years, and also briefly introduce PE format. After this update, the author has joined the PE format and how to collaborate with .NET collaboration and PE File Sections, RVA, The DataDirectory, function input and so on.
One
A long time ago, I wrote a article called "Peering Inside the PE: A Tour of the Win32 Portable Executable File File File," later than I expected, and now I am. I also heard that someone is using it (it is still in MSDN). Unfortunately, the problem of that article still exists, Win32 world is quietly changed, and that article has expired. From this month, I will use these two articles to make up.
You may ask why I should understand the PE format, the answer is still: the executable file format of the operating system and the data structure exposes the underlying details of the system. By understanding this, your program will be better.
Of course, you can read Microsoft's documentation to understand that I will tell you. However, like a lot of documents, 'Ning Ke, but it is a full'.
I put the focus in providing some content that is not suitable in the formal document. In addition, some knowledge in this article is not necessarily found in the official document.
1. Tearing of cracks.
Let me give you some examples from I wrote that article in 1994 to PE format. Win16 has become history and there is no need to make a comparison and explanation. Another abominable thing is Win32S in Windows 3.1, which is so unstable on it.
At that time, Windows 95 (also called Chicago) has not been released. NT or version 3.5. Microsoft's connector has not started a large-scale optimization, despite this, There WERE MIPS AND DEC Alpha Implementations of Windows Nt That Added To The Story.
So, what is the new things come out? 64-bit Windows has its own PE variants, Windows CE supports various CPUs, various optimizations such as DLL delay load, and the merger of the section table, the dynamic bundle, etc. have also been introduced.
There are a lot of things that have happened.
Let us have forgotten. Net. How is it cut with the system? For the operating system, the .NET's executable file format is compatible with the old PE format. Although this said, in the runtime, .NET is organized by metadata and intermediate languages, this is its core. In this article, I will open the door of .NET metadata, but don't discuss in depth.
If these changes in Win32 are not enough to let me rewrite this article, the original mistakes also make me sweat. For example, my description of TLS is just over, I have only you live in the western United States for time stamps. Also, some things have been made today, I have said. RDATA has almost no row, today is also, I also said that. IDATA section is readable, but some people who are engaged in API discovery It seems to be wrong.
In the process of updating this article, I also checked the PEDUMP for the program used to tilting the PE file. This program can compile and run under the 0x86 and IA-64 platforms.
2. PE format overview
Microsoft's executable file format, that is, everyone familiar with PE format is part of the official document. However, it is derived from the Coff on the VAX / VMS, most of the Windows NT team is coming from DEC, which is understandable. Very nature, these people will use their past code on the development of NT.
The term 'Portable Executable' is used because Microsoft wants to have a generic file format on all Windows platforms and all CPUs. From a big aspect, this goal has been implemented. It applies to NT and procedural, 95 and procedural, and CE.
Microsoft's OBJ files are formatted in Coff. When you see a lot of domains, you can use octal encoding, you will find what she is old. Coff obj files have been used with a lot of data structures and enumerations like PE, and I will mention some.
The 64-bit Windows only made a little change in PE format. This new format is called PE32 . There is no set of fields and only deletes a field. Other changes are to extend the previous 32-bit fields into 64-bit. For C code, these differences have been blocked by macro defined Windows header files.
The difference between EXE and DLL is completely semantically. They use the same file format -pe. The only difference is that one of the fields identified is EXE or DLL. There are many DLL expansions such as OCX, CPL, etc., are DLL. They have the same entity.
The knowledge you want to know about PE is the same as the data structure layout in the disk, and the data structure layout in memory. The primary task loaded into the executable (such as loadLibary) is to map files in the disk to the address space of the process. Therefore, like image_nt_header (hereinafter explained) is the same in the disk and memory. The key is that you have to know how you get some information in the disk, when it loads memory, you can get the same, basically no difference (ie, memory map files). But knowing that it is important to map the normal memory map file. The Windows loader looks to the PE file to determine where to map, then from the beginning of the file to a higher address map, but some things are different in the file and the offset in the memory will be different. Despite this, you have enough information to convert file offsets into memory offset. see picture 1
Figure 1
Offsets
When the Windows loader loads the PE into memory, it is called module in memory, and the file is mapping from HMODULE. Remember this: Give you hmodule, from then you can know a data structure (image_dos_header), then you can also know all the data structure. This powerful feature is particularly meaningful for API interception. (Accurately said: For Windows CE, this is not established, but this is later).
Memory's modules represent all code, data, resources required from this executable. Other portions can be read, but may not map (eg, repositioning section). There are also some parts that are not mapped at all, such as when debugging information is placed in the end of the file. There is a field telling the system how much memory needs to map files to memory. Unwanted data is placed on the end of the file, and in the past, all parts are mapped.
The PE format is described in Winnt.h. In this document, there is almost all about the data structure, enumeration, and # define. Of course, there are related documents elsewhere, but it is still Winnt.h.
There are a lot of tools for detecting the PE file. There are Visual Studio's Dumpbin, Depends in the SDK, I prefer Depends because it detects the introduction of the file in a simple way. A free PE viewer, Pebrowse, from smidgenosoft. My PEDUMP is also very useful, it has the same functionality as Dumpbin. From the API stand, Imagehlp.dll provides a mechanism for reading and writing PE files.
Review some of the basic concepts of the PE file is meaningful before the discussion of the PE file. In the following sections, I will discuss: PE, relative virtual address (RVA), data directory, and functions.
3. PE section
The PE section represents code or data in a case, the code is code, but there are multiple types of data, readable program data (such as global variables), other festival containing an API introduction table, resource, and heavy Location. Each section has its own properties, including whether it is a code section, is it read or read or readable, and the data is global shared.
Typically, the data logic in the section is associated. The PE file is generally at least two sections, one is code, the other is data, generally has another type of data for data. Later I will describe various types of fees.
Each section has a unique name. This name is used to convey this section. For example, .rdata represents a read-only festival, the name of the section is meaningless for the operating system, just for people to understand. Named a section as foobar and .text is the same. Microsoft named their section a distinctive name, but this is not required. Borland's connector is used for Code and Data.
The general compiler will produce a series of standards, but this is not incredible. You can build and name your own section, and the connector will automatically contain them in the program file. In Visual C , you can use the #pragma instruction to let the compiler into the data into one section. Like this:
#pragma data_seg ("my_data")
. . . Need to initialize
#pragma data_seg ()
You can also do the same thing to .DATA. Most of the procedures are only generated by the compiler, but sometimes you need this. For example, establish a global sharing festival.
The section is not all determined by the connector, and they can be placed from the compiler from the OBJ file during the compiler. The work of the connector is to consolidate all OBJ and the festival required to be required in the library. For example, all OBJ in your project may have an .TEXT section containing code, the connector merges these sections into one .Text section. Also for .DATA, etc .; these themes beyond the scope of this article. There are more rules about connector. In the OBJ file, it is specifically used to Linker, not in the PE file, this section is used to deliver information to the connector.
There are two of the aligned fields, a corresponding disk file, another file in the corresponding memory. The PE file header points out these two values, they can be different. The offset of each section begins with the multiple of the alignment value, for example, the typical alignment value is 0x200, then the offset of each section must be a multiple of 0x200. Once the memory is loaded, the start address of the section always is aligned in the page. X86CPU's page size is 4K, AL-64 is 8K
Here is the information of the .Text .data section of the PEDUMP's Poured Out of Pedump.
Section Table
01 .Text Virtsize: 0007465 Virtaddr: 00001000
Raw data offs: 00000400 RAW DATA SIZE: 00074800 •••••
02 .data virtsize: 000028CA virtaddr: 00076000
Raw Data Offs: 00074C00 RAW DATA Size: 00002400
Building a section of the offset in the file and the same PE file as the offset of the load address is possible. This will accelerate in 98 / me
Loading of files. The default option / opt: win98j of Visual Studio 6.0 is the file. Whether to use / opt: NOWIN98 in Visual Studio.net depends on whether the file is small enough.
An interesting connector feature is the ability of the consolidated section. If there is a similar compatible property, you can merge it as a section when connecting. This depends on whether the / merger switch is used. Combine .rdata and .text into one section .Text
/Merge:.rdata=.text
The advantage of the combined festival is to save space for disk and memory. Each section takes at least one point memory, if you can reduce the number of executables from 4 to 3, it is likely to use less memory. Of course, this depends on whether the hollow space in the two knots reaches a page.
When you consolidate the festival, it will become interesting because this has no hard and easy rules. For example, you can merge .rdata to .text,
But you can't merge .rsrc.reloc.pdata to other sections. Previous Visual Studio .NET allowed to merge. IDATA, and later not allowed. But when issuing, the connector can also merge .idata into other sections.
Because some of the introduction section will be written in the loader, you may be surprised how it is put in a read-only festival. Such, the system will temporarily change the page containing the introduction section when loading, and then restores the original attribute after the page is completed.
4. Relative virtual address
In the executable, there are many places to specify memory addresses, for example, when references global variables, you need to specify its address. PE files Although there is a preferred load address, they can load anywhere in the process space, so you can't rely on the load point of PE. Due to this, there must be a method to specify an address without relying on the address of the PE load point. In order to avoid the hardcoding of the memory address into the PE file, RVA is proposed. RVA is a simple memory offset relative to the PE load point. For example, the PE load point is 0x400000, then the address 0x401000 RVA in the code section is
0x401000 - (Load Address) 0x400000 = (RVA) 0x1000.
Put the RVA plus the actual address of the load point of the PE, you can convert the RVA to the actual address. By the way, press PE, the actual address in the memory is called VA (Virtual Address). Don't forget that I will say that the load point I said is hmodule.
Want to find any DLL in the memory? Get the load point with g etcodulehanle (lpctstr), use your PE knowledge to work.
5. Data directory
There are many data structures in the PE file that require rapid positioning, obvious examples, introduction functions, lead functions, resources, and relocation. These things are positioned in a consistent way, this is the data directory.
The data directory is an array of structures that contain 16 structures. Each element has a defined logo as follows:
// Export Directory
#DEFINE Image_DIRECTORY_ENTRY_EXPORT 0
// Import Directory
#DEFINE Image_DIRECTORY_ENTRY_IMPORT 1
// Resource Directory # define image_directory_entry_resource 2
// Exception Directory
#DEFINE Image_DIRECTORY_ENTRY_EXCEPTION 3
// Security Directory
#DEFINE Image_DIRECTORY_ENTRY_SECURITY 4
// Base Relocation Table
#DEFINE Image_DIRECTORY_ENTRY_BASERELOC 5
// debug directory
#DEFINE Image_DIRECTORY_ENTRY_DEBUG 6
// Description String
#DEFINE Image_DIRECTORY_ENTRY_COPYRIGHT 7
// Machine Value (MIPS GP)
#define image_directory_entry_globalptr 8
// TLS DIRECTORY
#DEFINE Image_DIRECTORY_ENTRY_TLS 9
// load configure Directory
#DEFINE Image_DIRECTORY_ENTRY_LOAD_CONFIG 10
Typedef struct _image_data_directory {
Ulong VirtualAddress;
Ulong size;
} Image_data_directory, * pimage_data_directory;
6. Import functions
When you use the code or data in another DLL, it is called the introduction. When PE is loaded, one of the loaders is to locate all introduction functions and data so that those addresses are visible for loaded PE. The details are discussed later, just about it.
When you use code or data in a DLL, you secretly connect to this DLL. But you don't have to make these addresses to make anything to your code, the loader do this for you. One of the methods is explicitly connected, so that you will determine that the DLL has been loaded, and the address of the function. You can call LoadLibary and getProcAddress.
When you secretly connect DLL, LoadLibary and getProcadDress are also executed. Only the loader did this for you. The loader also guarantees that any additional DLLs required by the PE file have been loaded. For example, when you connect kernel32.dll, it introduces Ntdll.dll's function, such as when you connect GDI32.DLL, and it rely on user32, advapi32, ntdll, and kernel32 dlls function, load The unit guarantees the resolutions of these DLLs that are loaded and functions.
Dark connection, the resolution process happens in the PE file. If there is any problem at this time (such as this DLL file is not found), the process is terminated.
Visual C 6.0 joins the feature of the delay load of the DLL. It is a mix of dark connections and explicit connections. When you delay in the DLL, the connector makes some and introduces something similar to the standard rule DLL, but the operating system does no matter what these things, but in the first time you call the function in this DLL (if it is not yet Into the entry, then call the address of the GetProcAddress to get the function.
There is a corresponding structural array for the DLL to be introduced for the PE file. Each structure indicates the name of this DLL and points to a function pointer array. This function pointer array is the so-called IAT (IMT ADDRESS TABLE). Each input function There is a reserved slot in IAT, and the loader will write a true function address there. Finally, it is important to be: Once the module is loaded, IAT contains the address of the introduction function to be called. It is very meaningful to put all the input functions in the IAT place, so that no matter how many times in the code is called, it is a function pointer in IAT.
Let us see how it is called an introduction function. There are two situations that require consideration: effective and efficiency. Best condition
Like this:
Call dword ptr [0x00405030]
Call directly in [0x405030], 0x405030 is located at the IAT section. The way the efficiency is as follows:
Call 0x0040100C
•••••
0x0040100C:
JMP DWORD PTR [0x00405030]
This situation, Call forwards control to a subroutine, JMP instruction in the subroutine jumps to 0x00405030 in IAT, simply, it has used 5 bytes and JMP more time.
You may be surprised that the introduction function uses this way, there is a good explanation, the compiler cannot distinguish between the call and the normal function call for the import function. For each function call, the compiler generates only the following instructions:
Call xxxxxxxx
XXXXXXXX is RVA filled by the connector, note that this instruction is not from the function pointer, but the actual address in the code.
For the balance of causality, the connector must generate a piece of code instead of going to the XXXXXXXXX, a simple method is to call a JMP Stub above.
So what is JMP Stub coming from there? Amazing is that it takes from the introduction library of the input function. If you look at an introduction library, you will find an instruction similar to the JMP Stub above.
Then, another question is how to optimize this form, the answer is your modifier of your compiler, __ decspec (import) modifier tells the compiler, which comes from another DLL, so the compiler will generate the first instruction.
In addition, the compiler will add the __imp_ prefix and then give the connector resolution, so you can send __imp_xxx to IAT, you don't need JMP Stub.
What is the meaning of us? If you write something that leads the function and provides a header file, don't forget to add a modifier before the function __declspec (import)
__Declspec (dllimport) Void foo (void);
This is doing this in system header files such as Winnt.h.
7. PE file structure
Now let's start studying the PE file format, I will start from the head of the file, describe the various data structures in each PE file, then, I will discuss more specialized data structures such as introducing tables and resources. These structures are defined in Winnt.h unless otherwise stated.
In general, these structures have 32 and 64 bits, such as image_nt_headers32, image_nt_header64, etc. They are basically the same, except for 64-bit extensions. Both #define winnt.h shield these differences, select that data structure depends on how you want to compile (eg, whether it is defined _win64)
The MS-DOS HEADER
Every PE file begins with a DOS program, which reminds Windows in the early age without such a considerable user. At least one message can be displayed when the executable is running on the non-Windows platform, indicating that it requires Windows. The beginning of the PE file is an image_dos_header structure. There is only two important fields e_magic and e_lfanew; E_LFANEW indicates the offset of PE File Header, and E_MAGIC needs to be positioned 0x5A4D, and is #define into image_dos_signature it ASCII is 'mz', Mark Zbikowski's first letters, one of the original builders of DOS.
The Image_NT_Headers HEADER
This structure is the main positioning information of the PE file. Its offset is given by the E_LFANEW of Image_DOS_HEADER
There is indeed 64 and 32 points, but I will not consider in the discussion, and they have little difference.
Typedef struct _image_nt_headers {
DWORD SIGNATURE;
Image_file_header fileheader;
Image_optional_header32 optionalheader;
} Image_nt_headers32, * pimage_nt_headers32;
In an effective PE file, SIGNTURE is set to 0x00004500, ASCII is 'pe00', # define image_nt_signture 0x00004500; the second field is an image_file_header structure, which contains basic information of the file, especially important, it pointed out image_optional_header Size (important?); In the PE file, Image_Optional_Header is very important, but still referred to as image_optional_header.
Image_optional_header The end of the structure is the address book used to locate important information in the PE file - the data directory, which is defined as follows:
Typedef struct _image_data_directory {
DWORD VirtualAddress; // RVA of the Data
DWORD size; // size of the data
}
The section TABLE
Immediately after image_nt_headers is a knot, the section table is an array of image_section_header. Image_section_header contains information on the section it associated, such as position, length, and feature; the number of the array is pointed out by image_nt_headers.fileHeader.NumberofSections. Specifically see the following figure
The sum of the size of the section in PE is to be aligned. The default values in Visual Studio 6.0 are 4K unless you use / opt: NOWIN98 or / ALIGN switch; in .NET, the default / opt: Win98 However, if the file is less than a specific size, 0x200 is used as alignment value.
Another interesting thing about alignment in the .NET document. The .NET file has a memory alignment value of 8K rather than 4K on the ordinary x86 platform, which guarantees that programs compiled in the X86 platform can run on the IA-64 platform. If the memory alignment is 4K, then the IA-64 loader cannot load this program because its page is 8K.
two
This part will discuss: Terminal, lead, binding, delay loading, debugging information, thread local storage, resource section. Considering that this part of the discussion of some things is not deep enough, the translator will make some additions to other information.
The Exports Section
to be continued
Sorry everyone, work is very busy, this is an article when I have learned PE, I feel uncomfortable now, it is difficult for everyone. The next article may wait for a long time, it is recommended that everyone will look directly. In order to thank everyone, I posted a Vir's C code about PE, most of the online is ASM.
http://www.9cbs.net/develop/read_article.asp?id=17437
