Chapter II The Windows 2000 Native API
Translation: kendiv (fcczj@263.net)
Update:
Saturday, January 29, 2005
Disclaimer: Please indicate the source and guarantee the integrity of the article, and all rights to the translation.
This chapter is the discussion of Windows 2000 Native APIs, mainly focused on the relationship between these APIs and system modules, will focus on the interrupt mechanism used by Windows 2000. Windows 2000 uses this mechanism to pass the request from the kernel service from the user mode to the kernel mode. In addition, Win32K interfaces and some main runtime libraries associated with Native APIs will also be mentioned, and some of the frequently used data types will also be described.
There are many detailed discussions about the Windows 2000 architecture. Many discussions about Windows NT are equally applicable to Windows 2000. The first and second editions of "Inside Windows NT" (Custer 1993, Solomon 1998) are all good books in this regard, and there are also "INSIDE Windows 2000".
NT * () and zw * () function set
A interesting fact about the Windows 2000 architecture is: it simulates multiple operating systems. Windows 2000 built-in three subsystems to support Win32, POSIX, and OS / 2 applications. The Win32 subsystem is the most popular, so it is more than the developer and operating system. In Windows 9x, the Win32 interface is actually implemented as the infrastructure of the entire system. However, the design of Windows 2000 is very different. Although the Win32 subsystem contains a system module named kernel32.dll, this is not an actual operating system kernel. It is just a basic component of the Win32 subsystem. In many programming books, the software development of Windows NT / 2000 is simplified to deal with Win32 API, and a hidden fact that NT platform exposed is another more basic call interface: Native API. I believe that developers who write kernel-Mode Driver or File System Driver have been very familiar with the Native API, because the kernel-mode module is located at a lower system layer, where the subsystem is invisible. However, you don't need to go to the driver to access this interface ---- even if a normal Win32 application can also call Native API at any time. There is no technical limit ---- just Microsoft does not support this application development model. Therefore, information about this topic is not a lot, Neither SDK Nor The Ddk Make The Native API Available To Win32 Application.
Uncapiled level
Most things in this book come from information called unsociated. This usually means that Microsoft does not disclose this information. However, there are several levels that are not documentation, because the information on huge operating systems (such as Windows 2000) may be published is very much. My personal system is classified as follows:
l Official documentation: This information comes from Microsoft published books, documents, or development tools. Most important information comes from SDK, DDK and MSDN.
l SemiDocument: Although it is not a formal document, this information can be excavated from the Microsoft officially released file. For example, many functions and structures of Windows 2000 are not mentioned in SDK or DDK documents, but appear in some header files or queues. Take Windows 2000 as an example, many important half-text file information is from head file NTDDK.H and NTDEF.H, both of which are part of DDK. l Not documentation, but there is no hide: This information cannot be found in any official document and development document, but part of the debug tool is available. All symbolized information in the executable or symbol file belong to this section. The best example is the kernel debugger! ProcessFields, and! Threadfields commands, these two commands give two unknown structures: EPRocess and Ethread member names and their offset.
l Completely not documentation: Microsoft hides certain information, it is necessary to obtain them only by reverse engineering and reasoning. This type of information contains many information about the details, and no one thinks that Windows 2000 developers need to pay attention to them, but this information is very precious for people of system developers and developing debugging software. The information inside the mining system is very difficult, but it is also very interesting.
The internal details of the Windows 2000 discussed in this book cover the last three of the above system classification.
System Service Distributor (System Service Dispatcher)
The relationship between Win32 subsystems and Native APIs can be greatly explained by the dependence between Win32 core modules and Windows 2000 kernel modules. Figure 2-1 shows the dependency between the modules, the box represents the system module, the arrow represents the dependencies between the modules. If an arrow points from the module A to the module B, this means that a depends on B, ie, the module A calls B. The module is connected by a bidirectional arrow, which means that the two are interdependent. In Figure 2-1, module: user32.dll, advapi32.dll, gdi32.dll, rpcrt4.dll, and kernel32.dll implement basic Win32 API. Of course, there are other DLLs (such as Version.dll, Shell32.dll and ComctL32.dll) also support Win32 API, in order to be clearer, I omitted them. Figure 2-1 shows a feature that is very interesting, all Win32 API calls finally transferred to NTDLL.DLL, while NTDLL.DLL transferred to Ntoskrnl.exe.
NTDLL.DLL is an operating system component that accurately provides services for NTIVE API, NTDLL.DLL is the front end of Native API in user mode. The Native API real interface is implemented in ntoskrnl.exe. From its file name, you can guess it is the NT operating system kernel. In fact, the kernel mode driver will enter the module most of the request for system services. The main task of NTDLL.DLL is to provide a subset of a determined kernel function for programs running on user mode, including Win32 subsystem DLLs. In Figure 2-1, INT 2EH, which is marked next to NTDLL.DLL, pointing to NTOSKRNL.EXE, indicates that Windows 2000 uses this interrupt to switch CPU privileges from user mode to kernel mode. The Kernel-Mode mode program believes that the code of user mode is an aggressive, erroneous and dangerous. Therefore, these code must be kept away from the kernel function. By switching the privilege level from user mode to kernel mode in the calling API, it is a way to control these issues. The calling program never touches the kernel, which can only look at them.
For example, the Win32 API function devices exported by kernel32.dll devices will eventually call NtDeviceIocontrolFile () exported by NTDLL.DLL. By compiling this function, this function will be found in amazing implementation - it is so simple! Troubleshoot 2-1 shows these. First, the CPU register EAX is loaded into a "magic" number 0x38, which is an assigned ID. Next, the register EDX is set to somewhere in the stack, and its address is added to the stack pointer ESP plus 4, and therefore, the EDX will point to the back address in the stack, which will be saved immediately when entering NTDeviceIocontrolFile (). . Obviously, the location of the EDX pointing is to store the passage of parameters when it is temporarily stored. The next instruction is a simple INT 2EH that jumps to the interrupt process routine (Interrupt Handler) stored in the 0x2e position of the interrupt descriptor table (IDT). Is this very familiar? In fact, this has some int 21h calls under DOS. However, Windows 2000's int 2EH interface is far more useful than a simple API call, and the Dispatcher uses it from user mode into kernel mode. Note that this mode switching method is unique to the X86 processor. On the Alpha platform, there are different ways to achieve this function. NtDeviceIocontrolfile:
Mov Eax, 38h
Lea Edx, [ESP 4]
Int 2eh
Ret 28h
Down 2-1. NTDLL.NTDEVICEIOCONTROLFILE () implementation
The Windows 2000 Native API consists of 248 functions, which use the above-described way to enter the kernel. 37 more than Windows NT 4.0. You easily recognize them through the NTDLL.DLL export list. NTDLL.DLL exports 249 such symbols. Many of the functions are ntcurrentteb (), which is a pure user mode function that does not need to enter the kernel. Table B-1 in Appendix B lists all available Native APIs. The table also points out that function is exported by ntoskrnl.exe. Surprisingly, in the module in the kernel mode, only one subset of the Native API can be called. On the other hand, Ntoskrnl.exe exports NT * symbols provided by two NTDLL.DLL (refer to NTBUILDNUMBER and NTGLOBALFLAG. Both symbols do not point to the entry address of the function, but point to variables in ntoskrnl.exe. Driver module can use the C compiler's extern keyword to import these variables. WINDOW 2000 exports a lot of variables in this way, and later I will give a sample code to use several of them.
You may be strange why NTDLL.DLL and NTOSKRNL.EXE are NTDLL.DLL and NTOSKRNL.EXE, respectively, named: NTDLL.NT *, NTDLL.ZW * and NTOSKRNL.NT *, NTOSKRNL .Zw *. The reason is that the two modules export two sets of interrelated Native API symbols. The left column in Table B-1 (in Appendix B) gives a symbol of all names that contain NT prefixes. Another collection contains a similar name, but the ZW prefix has replaced NT. The anti-compilation ndll.dll can see that each pair of symbols points to the same code. This seems to be a waste of memory. However, if you reflect NTOSKRNL.exe, you will find that the NT * symbol points to the actual code and zw * points to INT 2Est Stubs (such as the list 2-1). This means that the zw * function set will transfer from the user mode into kernel mode, while the code directly pointed directly from the NT * symbol will be executed after mode switching. There are two things in Table B-1 (in Appendix B) require special attention. First, the ntcurrentteb () function does not have a corresponding ZW * function. This is not a big problem, because NTDLL.DLL exports NT * and ZW * functions in a similar manner. Second, NTOSKRNL.EXE no longer has a relatively paired export NT / ZW function. Some of them appear in the form of NT * or ZW *. I don't know why, I guess NToskrnl.exe only exports those functions that have document records in the Windows 2000 DDK and those functions must be made by other system modules. Note that the reserved Native API function is still implemented inside NTOSKRNL.EXE. These functions do not have an open entry point, but can reach them via Int 2EH.
Service Descriptor Table (The Service Descriptor Tables)
It can be seen from the anti-compile code given from Example 2-1, and INT 2Eh is called together with two parameters of the CPU register EAX and EDX. I have mentioned that "Magic" numbers in EAX are an assigned ID. All Native APIs other than NTCurrentTeb () use this way, handle INT 2EH code must determine that each call will be assigned to that function. This is the reason for the dispatch ID. Interrupt Processing routines in Ntoskrnl.exe will query a specific table as an index as an index. This table is called the system service table (SST) This table corresponds to the C structure --- System_Service_Table definitions of SMSTEM_TABLE in List 2-1. In this list, it also includes the definition of the service_descriptor_table structure, which has four arrays of four SST types, where the first two are for specific purposes.
Although the two tables described above are basic data types, they do not have the corresponding documentation in the Windows 2000 DDK, and many of the code snipps that appear in this book contain undentified data types and functions. Therefore, this information cannot be guaranteed is completely trusted. All symbolized information, such as structural names, structural members, and parameters. When you create these symbols, I tried to use the appropriate name, which is based on a naming scheme that is obtained from a small subset of known symbols (including those obtained from the symbol file). However, this heuristic method in many occasions is not successful. Only all information is included in the original code, but I can't get them. In fact, I don't intend to read these source code, because this needs to sign a NDA (Non-Disclosure Agreement,, non-leak agreement), which will be difficult to write about non-document information due to the NDA restrictions. book of.
Typedef NTSTATUS (NTAPI * NTPROC) ();
Typedef ntproc * pntproc; #define ntproc_ sizeof (ntproc)
Typedef struct _system_service_table
{
PNTProc ServiceTable; // Array Of Entry Points
PDOWRD CounterTable; // Array of Usage Counters
DWORD serviceLIMIT; // Number of Table Entries
PBYTE argumenttable; // array of byte counters
}
System_Service_Table,
* Psystem_service_table,
** ppsystem_service_table;
/ / -------------------------------------------------------------------------------------------- -------------------------------------------------- ---------
Typedef struct _service_descriptor_table
{
System_service_table ntoskrnl; // ntoskrnl.exe (Native API)
System_service_table win32k; // Win32k.sys (GDI / User Support)
SYSTEM_SERVICE_TABLE TABLE3; / / NOT USED
SYSTEM_SERVICE_TABLE TABLE4; // NOT USED
}
System_descriptor_table,
* Psystem_descriptor_table,
** ppsystem_descriptor_table;
Listing 2-1 Structure Definition of System Services Descriptive Table
Now, return to the secret of the SDT (Service Descriptor Table). The definition of this structure given from the list 2-1 can be seen that the first two arrays of the structure remains to the NTOSKRNL.EXE and the Win32 subsystem (located in the kernel-mode) section in Win32k.sys. Call from GDI32.DLL and User32.dll is allocated via Win32K system service table (SST). Ntolkrnl.exe exports a pointer (symbolic keserviceDescriptable "pointing to its primary service descriptor table (Main SDT). The kernel also maintains an alternative SDT with the name: KeserviceDescriptAbleshadow, but this SDT is not exported. Accessing the primary service descriptor table (SDT) from the module in kernel mode is very easy, you only need two C instructions, as shown in the list 2-2. The first is a variable description by an Extern key modified variable, which tells the linker that the variable is not included in this module, and does not need to parse the corresponding symbol name when the link is required. When the module is loaded into the address space of the process, reference to the symbol is dynamically connected to the corresponding module. The second C instruction in the list 2-2 is such a reference. When you assign a variable of the type PServer_Descriptor_Table to KeserviceDescriptAble, you will create a dynamic connection with Ntoskrnl.exe. This is very like calling an API function in a DLL.
// Import SDT Pointer
EXTERN PSERVICE_DESCRIPTOR_TABLE KESERVICEDEScriptable;
// CREATE SDT REFERENCE
PSERVICE_DESCRIPTOR_TABLE PSDT = keserviceDescriptAble; List 2-2 Access System Services Descriptor Table
The serviceable member of each SST in the SDT is a pointer, pointing to an array consisting of a function pointer. This function pointer is: NtProc, which provides placeholders for the Native API, this way and in Win32 programming The proc type used is similar. NTPROC is defined in the previous list 2-1. The Native API function typically returns a NTSTATUS type code and uses NTAPI call mode, NTAPI is actually _stdcall. ServiceLimit members save the number of entry addresses found in the Servietable array. In Windows 2000, its default is 248. The ArgumentTable member is an array of btye types, which corresponds to the arguments referred to in ServiceTable, and gives the parameters required for each function pointer in the caller of the caller. This information is used with the pointers provided by the EDX register. This information is required when the kernel copys parameters from the caller's stack to their own stack. CounterTable members are not used in the Free Build version of Windows 2000. In the Debug Build version, the member points to an array of DWORD types as the usage counters of each function (Usage Counters). This Information can be used for profile purposes.
Use the kernel modulator of Windows 2000 to easily display the contents of the SDT. If you haven't set this useful program yet, please refer to the first chapter. In the quail 2-2, I used the DD KeserviceDescriptAble command for the first time. The debugger resolves this public symbol to 0x8046Ab80 while displaying the 32 DWORD 16 binding dump after the address. However, only the four rows of the only one is meaningful, they respectively correspond to four SDT members in the list 2-1. For clearer, they will be displayed in black. If you look closely, you will find that the fifth line is very similar to the first line. Is this another SDT? This is a good opportunity for the LN command to test the kernel debugger. In the Quatern 2-2, after the hex DUMP is displayed after the KeserviceDescriptable, I entered the LN 8046Abc0 command. Obviously, the debugger knows the address 0x8046Abc0, which transforms this address to the corresponding symbol KeserviceDescriptAbleshadow, which is the second SDT maintained by kernel maintenance. The significant difference between the two is: The second SDT contains the entrance address of Win32k.sys. The third and fourth members of the two tables are empty. Ntoskrnl.exe provides a function keaddsystemServiceTabel () to populate these two members.
Note that I cut off the output information of the LN command, only the basic information is retained.
Starting from the address 0x8046Ab88, it is the hex dump of KeserviceDescriptable, where ServiceLimit member can be found, and it can be seen that its value is 0xF8 (decimal 248), which is the same as we expected. The values of ServiceTable and ArgumentTable points to address 0x804704d8 and 0x804708bc, respectively. Use the ln command to look at the two addresses to get their symbols: KiserventAble and KiargumentTable. Both symbols are exported from NToskrnl.exe, but the debugger can identify them by viewing the symbol file of Windows 2000. The LN command can also be applied to the Win32K SST pointer, which gives its serviceTable and ArgumentTable members, and the debugger gives its corresponding symbol W32PServentAble and W32Pargumentable. These two symbols come from Win32k.sys symbol files. If the debugger cannot resolve these addresses, you can use the .Reload command to force reload all available symbol files and then pars. Example 2-2 The remainder is the 128-byte hex dump in Kiserventable and KiargumentTable. So far, if I said that the Native API's thing is correct, the NTClose () function address should be located in the 24th position of the Kiserventetable array, and its address is 0x80470538. At this address, it can be found that its value is 0x8044 C422, in the output of DD KiserventAble, the address is marked in a black body. Use the LN to view 0x8044422, it will see its corresponding symbol is ntclose ().
KD> DD KeserviceDescriptable
8046AB80 804704D8 00000000 000000F8 804708BC
8046AB90 00000000000000000000000000000000
8046Aba0 000000000000000000000000002
8046ABB0 00000000000000000000000000000000
8046ABC0 804704D8 00000000 000000F8 804708BC
8046ABD0 A0186BC0 00000000 0000027F A0187840
8046Abe0 0000000000000000000000000000000000000000000000000000000000000000000000000000000000
8046ABF0 0000000000000000000000000000000000
KD> LN 8046ABC0
(8046ABC0) NT! KeserviceDescriptableshadow
KD> ln 804704d8
(804704D8) NT! Kiserventetable
KD> LN 804708BC
(804708BC) NT! KiargumentTable
KD> LN A0186BC0
(A0186BC0) WIN32K! W32PSERVICETABLE
KD> ln a0187840
(A0187840) Win32k! W32PargumentTable
KD> DD KiserventAble
804704D8 804AB3BF 804AE86B 804BDEF3 8050B034
804704E8 804C11F4 80459214 8050C2FF 8050C33F
804704F8 804B581C 80508874 8049860A 804FC7E2
80470508 804955f7 8049c8a6 80448472 804a8d50
80470518 804B6BFB 804F0CEF 804FCB95 8040189A80470528 804D06CB 80418F66 804F69D4 8049E0CC
80470538 8044c422 80496f58 804AB849 804AA9DA
80470548 80465250 804f4bd5 8049bc80 804ca7a5
KD> DB KiargumentTable
804708BC 18 20 2C 2C 40 2C 40 44-0C 18 18 08 04 04 0C 10., @, @ D ........
804708cc 18 08 08 0C 08 08 04 04-04 0C 04 20 08 0C 14 0C ........... ....
804708DC 2C 10 0C 1C 20 10 38 10-14 20 24 1C 14 10 20 10, ... .8 .. $ ....
804708EC 34 14 08 04 04 04 0C 08 08 04 1C 18 18 18 08 18 4 ....... (.......
804708FC 0C 08 0C 04 10 00 0C 10-28 08 08 10 00 1C 04 08 ........ (.......
8047090C 0C 04 10 00 08 04 08 0C-28 10 04 0C 0C 28 24 28 ........ (... ($ ($)
8047091C 30 0C 0C 0C 18 0C 0C 0C-0C 30 0C 0C 0C 0C 10 0 ........ 0 ...
8047092C 10 0C 0C 14 0C 14 18 14-08 14 08 08 04 2C 1C 24 ...........,. $
KD> ln 8044c422
(8044C422) NT! NTCLOSE
Example 2-2 Checking Service Descriptive Table
Translation:
In Windows XP, KeserviceDescriptable and KeserVicedescriptAbleshadow are different from Windows 2000. In XP, the latter is located in front of the former, and in W2K, the latter is behind the former.
INT 2EH system service processing routine (System Service Handler)
The int 2Eh interrupt process routine hidden in the kernel mode is KisystemService (). Again again, this is an internal symbol, NToskrnl.exe does not export this symbol, but it is included in the Symbol file of Windows 2000. Therefore, the kernel debugger can correct the symbol correctly. From essentially, KisystemService () will do the following:
1. Retrieve the SDT pointer from the current thread control block (Thread's Control Block).
2. Determine the SST in the SDT (four SSTs in the SDT) by testing the 12th, 14th bits of the assigned ID in the EAX register. If the dispatch ID is located at 0x0000-0x0FFF, the ntoskrnl table will be selected; the Win32k table is selected at 0x1000-0x1FFF. 0x2000-0X2FFF and 0x3000-0X3FFF are reserved by the Table3 and Table4 of the SDT. If the assigned ID exceeds 0x3FFF, excess bits before dispatch will be blocked.
3. Determine the ID member of the corresponding SST in the selected SST by checking 0 to 11 bits of the assigned ID. If the ID is outside the range, the error code will be returned: status_invalid_system_service. In an unused SST, the ServiceLimit member is always 0 to generate an error code for all possible assignments. 4. Create the value of MMUserProbeAddress by checking the parameter stack pointer saved in EDX. This is an open variable that is exported by ntoskrnl.exe. Parameter pointers are usually compared to 0x7FFF0000. If there is no lower than this address, then Status_Access_violation will be returned.
5. According to the number of bytes of the parameter stack found in the ArgumentTable in the SST, copy all function parameters from the caller stack to the current kernel stack.
6. After returning from the service call, pass the control to the internal function KiserviceExit ()
Very interesting is that the int 2EH interrupt processing routine does not use global SDT (ie KeserviceDescriptable), but use thread exclusive pointers. Obviously, each thread can have different SDTs. When thread initialization, KeinitializThread () writes the KeserviceDescriptAble pointer in Thread Control Block. However, this default value may change later, if you change to KeserviceDescriptAbleshadow.
Win32 kernel mode interface (Win32 kernel-mode interface)
From the previous discussion of SDT, it can be seen that there is a second primary core mode interface associated with the Native API, and the main kernel-modinter. This interface connects the Graphics Device Interface, GDI of the WIN32 subsystem to the core component - Win32K (ie, Win32k.sys). This component is introduced with WINDOWS NT 4.0. Introducing this component is to overcome the inherent performance limits inherent in Win32 graphics engine (due to the initial design of the Windows NT subsystem). In Windows NT 3.x, the Win32 subsystem uses the client-server mode, so that you must switch from user mode to kernel mode to perform kernel calls. By moving most of the graphics engine to the kernel component - Win32k.sys, most of the performance loss caused by internal core switching is avoided.
Win32K Dispatch ID (Win32k Dispatch IDS)
Now this introduces Win32k.sys, and it is also the time to update Figure 2-1. Figure 2-2 is based on Figure 2-1, but Win32k.sys is added on the left of ntoskrnl.exe. At the same time, I also added an arrow from GDI32.DLL and User32.dll to Win32k.sys. Of course, this is not 100% correct, because the int 2Eh call in these modules actually points to Ntoskrnl.exe, and there is this interrupt process routine in NtoskRNL.exe. However, calling final is managed by Win32k.sys, which is why arrows refer to.
By it is mentioned earlier, the Win32K interface is also based on the Int 2Eh Dispatcher, which is very similar to the Native API. Only the difference is that Win32K uses another segment assignment ID. Although the dispatch ID associated with all Native API calls is located at 0x0000 ---- 0x0FFF, the Win32K assignment ID is between 0x1000 --- 0x1FFF. As shown in Figure 2-2, the main client of Win32K is GDI32.DLL and USER32.DLL. Therefore, the symbolized name associated with the Win32k assignment ID may be found by reflecting these modules (referring to GDI32.DLL and USER32.DLL). By refurbishing can be found in the Export Sections of these modules (GDI32.DLL and USER32.DLL), only a small subset of Int 2Eh calls, it seems to use the kernel debugger again. As shown in Example 2-3, I use the DD W32PServiceTable command to determine that the win32k.sys symbol is available, please use the .reeload command before you before this before you load all available symbol files. In the last three rows of Example 2-3, I use the ln command to display symbols related to the first entry address of W32PServentAble. Obviously, it can be seen that the Win32K function assigned to 0 is NTGDIABORTDOC (). You can repeat this process for all 639 IDs, but it is best to automatically perform symbols. Now, I have completed this job, and all the symbol names corresponding to all dispatched ids are included in the table B-2 of Appendix B. The symbols are mapped from GDI32.DLL and User32.dll to Win32k.sys. It is very simple: GDI symbols can be converted to Win32K symbols by adding NTGDI prefixes in front, and User symbols add NTUser prefix. However, there is a small number of exceptions. For example, if a GDI symbol begins in GDI, its prefix is reduced to NT, which may be to avoid NTGDIGDI character sequences. In some other examples, the case of characters will be somewhat different (such as enableeeudc (), it becomes NTGDIENABLEUDC (), or W. w on the tail with the symbol name (eg CopyAcceleratorTablew () After transformation, becomes NTUserCopyAcceleratorTable ()).
The detailed documentation of the Win32K API needs great efforts. These functions are almost three times the Native API. Perhaps someone will write a good reference manual for these APIs, just like the Native API manual written by Gary Nebbett. However, within this book, information about these APIs is sufficient.
