WINSOCK applications with completion port development, usually develop web applications are not a relaxed thing, but in fact, as long as you master a few key principles - create and connect a socket, Try to connect, then send and receive data. It's really difficult to write a web application that can be accepted, one thousand connected network applications. This article will discuss how to develop high-extension WINSOCK applications through Winsock2 on Windows NT? And Windows 2000. The main focus of the article is in the server / server model server, of course, many of them apply to both sides of the model. The API and the response size pass the Win32 overlapping I / O mechanism, the application can draw an I / O operation, overlapping operation requests are completed in the background, while the same time is drew the thread to do other things. The thread receives the relevant notice after the overlapping operation is completed. This mechanism is useful for those time-consuming operations. However, the functions like WSaasyncselect () and UNIX on Windows 3.1 are easy to use, but they cannot meet the needs of the response scale. The completion of the port mechanism is optimized for the operating system. On Windows NT and Windows 2000, the overlapping I / O mechanism to complete the port can truly expand the system's response scale. Completing a completion port is actually a notification queue that places a notification of the completed overlapping I / O request from the operating system. When an I / O operation is completed, a notification will be received by a worker thread that can process the results of the operation. After the socket is created, you can associate with a completion port at any time. Typically, we create a certain number of worker threads in the application to handle these notifications. The number of threads depends on the specific needs of the application. Ideally, the number of threads is equal to the number of processors, but this also requires any threads should not perform an operation such as synchronous read and write, waiting event notification, etc., to avoid thread blocking. Each thread will be divided into a certain CPU time, during which the thread can run, then another thread will be divided into a time slice and start execution. If a thread performs blocking operations, the operating system will deprive its unused time slice and let other threads begin. That is, the previous thread does not fully use its time slice. When this happens, the application should prepare other threads to make full use of these time films. The use of the port is divided into two steps. First create a completed port, as shown in the following code: Handle Hiocp; Hiocp = CREATEIOCOMPLETITONPORT (Invalid_Handle_Value, NULL, (Ulong_PTR) 0, 0); if (hiocp == null) {// error} After the port is created, Use this complete port to associate with it. The method is to call the createioCompletionPort () function again, the first parameter fileHandle is set to the handle of the socket, and the second parameter existingCompletionPort is set to the handle of the completed port just created. The following code creates a socket and associates it and the completed port created: Socket S; s = socket (AF_INET, SOCK_STREAM, 0); if (s == invalid_socket) {// error if (Createiocompletion) (Handle) S, Hiocp, (Ulong_PTR) 0, 0) == NULL) {// error} ???} The association operation of the socket and the completion port is completed.
Any overlapping operations on this socket will make a completion notification by completing the port. Note that the third parameter in the CreateiocompletionPort () function is used to set a "completion key" (Translator Note) (the translator note) (the translator)) is set to any data type. Whenever the notification is completed, the application can read the corresponding completion key, so the completion key can be used to pass some background information to the socket. After creating a completed port, after the one or more sockets is associated with it, we have to create several threads to process completion notifications. These threads continue to circulate the getQueuedCompletionStatus () function and return to the completion notification. Below, let's take a look at how the app tracks these overlapping operations. When the application calls a overlapping operation function, the pointer to an Overlapped structure is included in its parameters. Once the operation is complete, we can take this pointer through the getQueuedCompletionStatus () function. However, the order is based on the Overlapped structure pointed to by this pointer, and the application cannot distinguish which operation is completed. To implement tracking of the operation, you can define an Overlapped structure to join the required tracking information. Whenever the overlapping operation function is called, a OverlappedPlus structure (such as WSasend, WSARECV) is always passed through its LPOVERLAPPED parameters. This allows you to set some operating status information for each overlapping call operation. When the operation is over, you can get a pointer to your custom structure through the getQueuedCompletionStatus () function. Note that the Overlapped field does not require a first field of this extended structure. After getting a pointer to the Overlapped structure, you can take a pointer to the extended structure with the containing_record macro (the translator Note: The above two segments will be the OverlappedPlus structure, which will be the overlapped structure, I don't understand it, please master Enlighten me OVERLAPPED structure defined as follows: typedef struct _OVERLAPPEDPLUS {OVERLAPPED ol; SOCKET s, sclient; int OpCode; WSABUF wbuf; DWORD dwBytes, dwFlags; // other useful information} OVERLAPPEDPLUS; #define OP_READ 0 #define OP_WRITE 1 #define OP_ACCEPT 2 below Let's take a look at the situation of the worker thread in Figure 2.
Figure 2 Worker Thread DWORD WINAPI WorkerThread (LPVOID lpParam) {ULONG_PTR * PerHandleKey; OVERLAPPED * Overlap; OVERLAPPEDPLUS * OverlapPlus, * newolp; DWORD dwBytesXfered; while (1) {ret = GetQueuedCompletionStatus (hIocp, & dwBytesXfered, (PULONG_PTR) & PerHandleKey, & Overlap, INFINITE); if (ret == 0) {// Operation failed continue;} OverlapPlus = CONTAINING_RECORD (Overlap, OVERLAPPEDPLUS, ol); switch (OverlapPlus-> OpCode) {case OP_ACCEPT: // Client socket is contained in OverlapPlus.sclient // Add client to completion port CreateIoCompletionPort ((HANDLE) OverlapPlus-> sclient, hIocp, (ULONG_PTR) 0, 0);. // Need a new OVERLAPPEDPLUS structure // for the newly accepted socket Perhaps // keep a look aside list Off = allocateoverlappedplus (); if (! newolp) {// error} newolp-> s = O verlapPlus-> sclient; newolp-> OpCode = OP_READ; // This function prepares the data to be sent PrepareSendBuffer (& newolp-> wbuf); ret = WSASend (newolp-> s, & newolp-> wbuf, 1, & newolp-> dwBytes , 0, & newolp.ol, NULL); if (ret == SOCKET_ERROR) {if (WSAGetLastError () = WSA_IO_PENDING) {// Error}!} // Put structure in look aside list for later use FreeOverlappedPlus (OverlapPlus); / / Signal ACCEPT THREAD to Issue Another AccepTex SetEvent (HACCEPTTHREAD); BREAK;
Case op_read: // process the time read / ??? // repost the read if Necessary, Reusing the Same // Receive Buffer AS Before Memset (& overlapplus-> OL, 0, SIZEOF (OVERLAPPED)); RET = WSARECV ( OverlapPlus-> s, & OverlapPlus-> wbuf, 1, & OverlapPlus-> dwBytes, & OverlapPlus-> dwFlags, & OverlapPlus-> ol, NULL);! if (ret == SOCKET_ERROR) {if (WSAGetLastError () = WSA_IO_PENDING) {// ERROR}}}}}}}}}} // process the data sample, etc. Break;} //witch} // while} // workerThread where the contents of each handle key is the content of the completion port and the socket The word is associated with the completion key parameters set; the OVERLAP parameter returns a pointer to the OverlappedPlus structure used when the overlapping operation is issued. To remember, if the overlapping operation call fails (that is, the return value is socket_error, and the error is not WSA_IO_PENDING, then the completion port will not receive any completion notification. If the overlapping operation calls success, or if the cause is WSA_IO_PENDING error, the completion port will always be able to receive the completion notification. Sociographs of Windows NT and Windows 2000 have a basic understanding of Windows NT and Windows 2000's socket architecture for the development of a large-induced Winsock application. It is very helpful. Different from other types of operating systems, Windows NT and Windows 2000 transport protocols do not have a style like a socket, which can be directly conversible with the application, but use a more underlying API called transmission drive. TRANSPORT DRIVER INTERFACE, TDI. Winsock's core mode driver is responsible for connecting and buffer management to provide a socket emulation emulation to the application (implemented in the AFD.sys file) while responsible for conveying drivers with the underlying transfer driver. Who is responsible for managing buffers? As mentioned above, the application is talking to the transfer protocol driver via Winsock, and AFD.sys is responsible for buffering districts for applications. That is, when the application calls the send () or wsasend () function to send data, AFD.sys will copy the data into its own internal buffer (depending on the SO_SNDBUF set value), then send () or Wsasend () The function returns immediately. It can also be said that AFD.sys is responsible for sending data in the background. However, if the application requested the data exceeded the SO_SNDBUF set buffer size, the wsasend () function blocked until all data is sent. The case where data is received from the remote client is similar. As long as you do not need to receive a lot of data from the application, AFD.SYS will copy the data into its internal buffer.
When the application calls the RECV () or WSARECV () function, the data will be copied from the internal buffer to the buffer provided by the application. In most cases, such architectures are well operated, especially when the application uses traditional sockets, non-overlapping Send () and receive () modes. However, the programmer should be careful, although the SO_SNDBUF and SO_RCVBUF option value can be set to 0 (turn off the internal buffer) by setsockopt () this API, but the programmer must clearly turn off the internal buffer of AFD.sys. What consequences have caused the system crashes that may cause a copy of the buffer copy when transmitting and receiving data. For example, an application is turned off by setting SO_SNDBUF to 0, and then issues a blocking Send () call. In this case, the system core will lock the buffer of the application until the receiver confirms that the send () call returns after receiving the entire buffer. It seems that this is a simple method that determines whether your data has received all of the other party, but it is not the case. Think about it, even if the far-end TCP notification data has been received, it is also not mean that the data has been successfully given to the client application. For example, the other party may have a resource inadequate, causing AFD.sys to copy the data to the application. Another more tight problem is that only one transmission call can be performed each time in each thread, the efficiency is extremely low. Set SO_RCVBUF to 0, and turn off the AFD.SYS's receiving buffer cannot make performance improvement, which will only buffer the received data is buffered at a lower level than Winsock. When you send a Receive call, it is equally buffered. District copy, so you have wanted to avoid the conspiracy of the buffer copy. Now we should be clear, close the buffer is not a good idea for most applications. As long as you want to pay attention to a few WSARECVS overlapping calls at a certain connection, you usually do not need to turn off the receiving buffer. If AFD.sys always has buffers available by the application, it will not necessarily use the internal buffer. High-performance server applications can turn off the send buffer while do not lose performance. However, such an application must be very careful to ensure that it always makes multiple overlapping calls, rather than waiting for a overlap transmission to issue the next one. If the application operates in order to send one after another, it is wasted twice to send the neutral time, in short, it is necessary to ensure that the transfer driver will be turned to another buffer after sending a buffer. Area. Restrictions on resources When designing any server application, its strongness is the main goal. That is, your application should be able to cope with any burst, for example, the number of concurrent client requests reaches the peak, and the available memory temporarily is insufficient, and other short-term phenomena. This requires the designer of the program to pay attention to the problem of resource restrictions under the Windows NT and 2000 systems, and handle burst events. You can directly control, the most basic resource is network bandwidth. Typically, applications that use User Datashers (UDP) may limit the loss of the package to minimize the loss of the package to maximize the loss of the package. However, when using the TCP connection, the server must be very careful to control, preventing the network bandwidth overload exceeds a certain time, otherwise a large number of packages will need to be retransmitted. The method of bandwidth management should be determined according to different applications, which is beyond the scope discussed herein. The use of virtual memory must also be managed carefully.
Re-use the allocated memory by cauting to apply and release memory, or to apply the LOOKASIDE LISTS (a cache) technology, will help control the memory overhead of the server application (original "to let the server application left by the server application Small a little "), avoiding the operating system frequently swaping the physical memory of the application application to virtual memory (original as" to allow the operating system to always keep more application address space more in memory ")."). You can also make your operating system more physical memory by setting SetWorkingSize () this Win32 API. You may also encounter the shortcomings of the other two non-direct resources when using Winsock. One is the limit of the locked memory page. If you shut up AFD.sys buffer, all pages of the application buffer will be locked to physical memory when the application transmits and receives data. This is because of the need to access these memory for the kernel driver, during which the page cannot be exchanged. If the operating system needs to assign some other applications to some-page physical memory, there is no problem when there is not enough memory. Our goal is to prevent the procedure from writing a pathological, locking all physical memory, letting system crash. That is, when your program locks memory, do not exceed the system specified by the system. On Windows NT and 2000 systems, all applications can be locked in memory is approximately 1/8 of physical memory (but this is just a big estimate, not the basis for your calculation). If your app doesn't pay attention to this, when you make too many overlap transceivers call, and I / O doesn't work and complete, you may occasionally Error_INSUFFICIENT_RESOURCES error. In this case you want to avoid over-locking memory. At the same time, it should be noted that the system will lock the entire memory page in which your buffer is located, so the buffer is approached to the page boundary (the translator understands, the buffer is exactly more than the page boundary, even one byte, The page where the excerpt is located is also locked). Another limit is that your program may encounter the shortcomings of the system unsubstructed pool resources. The so-called unpized pool is a memory area that never swaps out. This memory is used to store data for access to various core components, where the kernel components are not accessible to the page space that is exchanged. The Windows NT and 2000 drivers can allocate memory from this particular unpized pool. When the application creates a socket (or similar to open a file), the core will allocate a certain amount of memory from the unpized pool, and when the binding, the connection socket, the kernel will never paginize Some memory is allocated in the pool. When you pay attention to observing this behavior, you will find that if you send some I / O requests (such as sending and receiving data), you will reassign more memory from an unbeatable page (for example, to track a pending I / O operation, you may need to add a custom structure to this operation, as mentioned above). Finally, this may cause a certain issue, and the operating system limits the amount of not paging memory. On the two operating systems of Windows NT and 2000, the specific quantity of unsimposed memory assigned to each connection is different, and future versions of Windows are likely to be different. In order to make the application's life period, you should not calculate the specific demand for the memory of the unpredictable pool. Your program must prevent the limit that consumes an unbeatable page. When the remaining space remaining in the system is too small, some kernel drivers that have nothing to do with your application will be crazy, even cause system crash, especially when there is a third-party device or driver in the system, it is easier to occur. Such a tragedy (and cannot predict). At the same time, you have to remember that other applications that can also be run on the same computer, so that the exemption of the resource is particularly conservative and cautious when designing your application.
The problem of insufficient resource is very complicated because you don't receive special error codes when you have mentioned situations, usually you can only receive a general WSAENOBUFS or ERROR_INSUFFICIENT_RESOURES error. To deal with these errors, first, adjust your application work configuration to a reasonable maximum (translator Note: The so-called work configuration means the amount of memory required to run in each of the applications, please refer to http: // msdn .microsoft.com / MSDNMAG / ISSUES / 1000 / BUGSLAYER / BUGSLAYER1000.ASP, about memory optimization, the translator's other translation), if the error continues, then pay attention to the problem of insufficient network bandwidth. After that, please confirm that you didn't send too much transceiver call. Finally, if it is still a mistake of insufficient resources, it is likely to have problems with insufficient memory pool in paging. To release the unpaged memory pool space, turn off a considerable part of the application, wait for the system to spend and correct this instantaneous error. One of the most common things to accept the connection to requesting the server is to accept connection requests from the client. The only API that uses overlapping I / O accepted connections on the socket is the acceptex () function. Interestingly, the usual synchronous acceptance function accept () return value is a new socket, and the acceptex () function requires another socket as one of its parameters. This is because acceptex () is a overlapping operation, so you need to create a socket in advance (but don't bind or connect it), and pass this socket to Acceptex (). The following is a small typical pseudo code using Acceptex (): do {- Waiting for the last ACCEPTEX to complete - Create a new socket and associate with the completion port - Set the background structure, etc. - Send an ACCEPTEX request} while (TRUE ); As a high response capability server, it must issue enough AcceptEx calls, waiting, and respond immediately once the client connection request appears. As for how many ACCEPTEXs are enough, depending on the type of communication traffic expected by your server program. For example, if the income rate is high (because the connection duration is shorter, or the traffic peak appears), then the ACCEPTEX that needs to be waiting is to be more than those who enter the client. Smart approach is to analyze traffic conditions by apps and adjust the number of AcceptEx waiting, rather than fixed on a certain quantity. For Windows2000, Winsock provides some mechanisms to help you determine if the number of acceptex is sufficient. That is, create an event when you create a listener socket, and register the socket with this event with this event by wsaeventselect () API and register the FD_ACCEPT event notification. Once the system receives a connection request, if there is no acceptex () in the system waiting to be accepted, then the above event will receive a signal. With this incident, you can judge that you have enough AccePtex (), or detect an abnormal customer request (under the document). This mechanism does not apply to Windows NT 4.0. One of the advantages of using Acceptex () is that you can complete two things to accept client connection requests and accept data (by transferring LPOUTPUTBUFFER parameters) through a call. That is, if the client transmits data while the connection is sent, your AcceptEx () call can return immediately after the connection creates and receives the client data. This may be useful, but it may also cause problems because all client data must be returned.
Specifically, if you pass the LPOUTPUTBuffer parameters while sending an AcceptEx () call, AcceptEx () is no longer a atomic operation, but is divided into two steps: accept the customer connection, wait for the received data. When a mechanism is missing to inform Your application: "Connection has been established, waiting for client data", which means that there is a possibility that the client only issues only a connection request, but does not send data. If your server receives too many connection connections, it will refuse more legal client requests. This is a common method of hackers to "denial service" attack. To prevent such attacks, the thread that is accepted should check the AcceptEx () socket from time to time by calling the getSockopt () function (option parameter to SO_CONNECT_TIME). The entry value of the getSockopt () function will be set to the time of the socket being connected, or set to -1 (representing the socket has not been established). At this time, the characteristics of WSAEventSelect () can be used very well to do this. If the connection has been established, the data is not received for a long time, then the connection should be terminated, the method is to close the socket provided to AccePtex () as a parameter. Note that if the socket has been passed to AccePtex () and starting, then your application should not turn off their applications. This is because even if these sockets are closed, the data structure of the corresponding kernel mode will not be cleared before the system performance is entered, or before the connection enters itself, or the listening socket itself is turned off. The thread that is called by acceptex () seems to perform the port association operation, processing other I / O completion notifications, but don't forget that the thread should try to avoid the execution of the obstruction type. A side effect of Winsock2 hierarchical structure is to call the upper structure of the Socket () or WSASocket () API may be important (the translator does not understand the original meaning, sorry). Each acceptex () call requires a new socket, so it is best to have an independent thread to call acceptex () without participating in other I / O processing. You can also use this thread to perform other tasks, such as event records. The last consideration for Acceptex (): To implement these APIs, do not require the Winsock2 implementation provided by other providers. This is equally applicable to Microsoft's unique API, such as transmitfile () and getacceptexsockaddrs (), and other APIs that may be added to the new version of Windows. On Windows NT and 2000, these APIs are in Microsoft's underlying provider DLL (Mswsock.dll) can be called by compiling connectivity to the MSWSock.lib, or dynamically obtain a pointer to function through WSAIOCTL () (option parameter is SiO_GET_EXTENSION_FUNCTION_POINTER). If the function is called directly without the function pointer in advance (that is, the compile time is static connection Mswsock.lib, the function is directly called directly), the performance will be affected. Because AcceptEx () is placed outside the Winsock2 architecture, it is composed of WSAIOCTL () acquisition function pointer each time it is called. To avoid this performance loss, applications that need to use these APIs should be used to obtain a function of the function from the underlying provider by calling WSAIOCTL ().
See Figure 3 sets of text architecture: Application || / || / // Winsock 2.0 DLL (WS2_32.dll) || / || // Layered / Base Providers RSVP | Proxy | Default Microsoft Providers (mswsock.dll / MSAFD .dll) || / || / | | | // TRAMSPORT Protocols TCP / IP | ATM | Other TransmitFile and TransmitPackets Winsock offer two specialized files and Memory data transfer has been optimized. Where TransmitFile () This API function can be used on Windows NT 4.0 and Windows 2000, while transmitpackets () will be implemented in future versions of Windows. TransmitFile () is used to transfer the file content through Winsock. Usually send files, first modify CREATEFILE () to open a file, and then call ReadFile () and wsasend () until the data is sent. However, this method is very efficient, because it is called each time the readFile () and wsasend () involves the conversion from the user mode to the kernel mode. If you replace it with transmitfile (), then only need to give it a handle of the file and the number of bytes to be sent, and the mode conversion operation involved will only occur once when CreateFile () is invoked, then transmitfile () A occurs again. This efficiency is much higher. TRANSMITPACKETS () is further more than TransmitFile (), which allows users to send only multiple files and memory buffers that can be sent once. The following function prototype: BOOL TransmitPackets (SOCKET hSocket, LPTRANSMIT_PACKET_ELEMENT lpPacketArray, DWORD nElementCount, DWORD nSendSize, LPOVERLAPPED lpOverlapped, DWORD dwFlags); wherein, lpPacketArray structure is an array, where each element can be either a memory buffer or a file handle the structure is defined as follows: typedef struct _TRANSMIT_PACKETS_ELEMENT {DWORD dwElFlags; DWORD cLength; union {struct {LARGE_INTEGER nFileOffset; HANDLE hFile;}; PVOID pBuffer;};} TRANSMIT_FILE_BUFFERS; wherein each field is self-describing type (self explanatory). DWELFLAGS field: Specify whether the current element is a file handle or a memory buffer (specified by constant TF_ELEMENT_FILE and TF_ELEMENT_MEMORY); CLENGTH field: Specifies the number of bytes that will be sent from the data source (if it is a file, this field value is 0 means sending the entire File); Unnamed Uniforms in the structure: Contains the memory buffer (and possible offset) of the file handle. Another benefit of using these two APIs is that you can reuse the socket by specifying the TF_REUSE_SOCKET and TF_DISCONNECT flags.
