WINSOCK app for developing a large response size with completion port
Usually develop network applications is not a relaxed thing, but actually as long as you master a few key principles -
Create and connect a socket, try to connect, then send and receive data. It's really difficult to write a one, one can be received, more
Thousands of connected web applications. This article will discuss how to develop high expansions on Windows NT? And Windows 2000 through Winsock2
Ability of Winsock applications. The main focus of the article is in the client / server model server, of course, many of them
Both parties to the model are applicable. API and response scale
With the overlapping I / O mechanism of Win32, the application can draw an I / O operation, overlapping operation requests are completed in the background, while the same time is drawn
Operation threads do other things. The thread receives the relevant notice after the overlapping operation is completed. This mechanism is for those time-consuming operations
Don't use it. However, the function like Wsaasyncselect () and unix on Windows 3.1 is easy to use, but
They cannot meet the needs of responding. The completion of the port mechanism is optimized for the operating system, in Windows NT and
On Windows 2000, the overlapping I / O mechanism for completing the port is used to truly expand the system's response scale.
Complete port
A completion port is actually a notification queue that places a notification of the completed overlapping I / O request by the operating system. When an item I / O
Once the operation is completed, a notification can be received for a worker thread that can process the results of the operation. And the socket is created,
Can be associated with a completion port at any time.
Typically, we create a certain number of worker threads in the application to handle these notifications. The number of threads depends on the application
Specific needs. Ideally, the number of threads is equal to the number of processors, but this also requires any threads should not perform such as synchronization.
Read and write, wait for event notification, etc. to avoid thread blocking. Each thread will be divided into a certain CPU time, during which the thread is
You can run, then another thread will be separated to a time piece and start execution. If a thread performs blocking operation, the operating system
The remaining time slice which is not used is deprived and the other thread will begin. In other words, the previous thread did not use its time film, when
When this happens, the application should prepare other threads to take advantage of these time films.
The use of the port is divided into two steps. First create a completed port, as shown in the following code:
Handle Hiocp;
Hiocp = CreateiocompletionPort (INVALID_HANDLE_VALUE, NULL, (ULONG_PTR) 0, 0); if (hiocp == null) {// error}
After completing the port creation, you want to associate the socket that will use the completion port. The method is called again
CreateioCompletionPort () function, the first parameter fileHandle is set to the handle of the socket, the second parameter
EXInsteningCompletionPort is set to the handle of the completed port that has just created. The following code creates a socket and associates it with the completed port created by:
Socket S;
s = socket (AF_INET, SOCK_STREAM, 0); if (s == invalid_socket) {// Errorif (CreateiocompletionPort ((Handle) S, Hiocp, (Ulong_PTR) 0, 0) == NULL) {// error} ?? ?
At this time, the association operation of the socket and the completion port is completed. Any overlapping operations on this socket will make a completion notification by completing the port. Note that the third parameter in the CreateiocompletionPort () function is used to set a "completion of the socket"
Completion Key "(Translator Note: The completion key can be any data type). Whenever the notification comes, the application can read
The corresponding completion key, the completion key can be used to deliver some background information to the socket.
After creating a completed port, after the one or more sockets is associated with it, we have to create several threads to process completion notifications. This
Some threads continue to cycle the getQueuedCompletionStatus () function and return to the completion notification.
Below, let's take a look at how the app tracks these overlapping operations. When the application calls a overlapping operation function, point to one
The pointer to the Overlapped structure is included in its parameters. When the operation is complete, we can pass the getQueuedCompletionStatus ()
Take this pointer in the number. However, the order is based on the Overlapped structure points to this pointer, the application cannot distinguish it.
Which operation is. To implement tracking of the operation, you can define an Overlapped structure to join the required tracking information.
Whenever the overlapping operation function is called, an OverlappedPlus structure is always transmitted through its LPOVERLAPPED parameters (for example
Wsasend, WSARECV and other functions). This allows you to set some operating status information for each overlapping call operation. When the operation is over,
You can get a pointer to your custom structure through the getQueuedCompletionStatus () function. Note that the Overlapped field does not require certain
It is the first field of this extension structure. After getting a pointer to the Overlapped structure, you can use Containing_Record
Macro takes out of the pointer to the extended structure (the translator's note: The above two sections will be the OverlappedPlus structure, and it will be overlapped structure.
I also don't understand, please enlighten me.
The definition of the OVERLAPPED structure is as follows:
Typedef struct _overlapped {Overlapped OL; Socket S, SCLIENT; INT OPCODE; WSABUF WBUF; DWORD DWBYTES, DWFLAGS; // Other useful information} OverlappedPlus;
#define op_read 0 # define op_write 1 # define op_accept 2
Let's take a look at the situation of the worker thread in Figure 2.
Figure 2 Worker Thread
DWORD WINAPI WORKERTHREAD (LPVOID LPPARAM) {ulong_ptr * perhandlekey; overlapped * overlap; overlappedplus * overlapplus, * newolp; dword dwbytesxfered;
while (1) {ret = GetQueuedCompletionStatus (hIocp, & dwBytesXfered, (PULONG_PTR) & PerHandleKey, & Overlap, INFINITE); if (ret == 0) {// Operation failed continue;} OverlapPlus = CONTAINING_RECORD (Overlap, OVERLAPPEDPLUS, ol); switch (OverlapPlus-> OpCode) {case OP_ACCEPT: // Client socket is contained in OverlapPlus.sclient // Add client to completion port CreateIoCompletionPort ((HANDLE) OverlapPlus-> sclient, hIocp, (ULONG_PTR) 0, 0); // Need . a new OVERLAPPEDPLUS structure // for the newly accepted socket Perhaps // keep a look aside list of free structures newolp = AllocateOverlappedPlus ();. if (newolp!) {// Error} newolp-> s = OverlapPlus-> sclient; NEWOLP-> opcode = op_read;
// this function preparesendbuffer (& newolp-> wbuf); RET = Wsasend (newolp-> s, & newolp-> wbuf, 1, & newolp-> dwbytes, 0, & newolp.ol, null); if ( RET == SOCKET_ERROR) {IF (WsagetLastError ()! = WSA_IO_PENDING) {// error}}
// Put Structure in Look Aside List for lating use freeoverlappedplus;
// Signal Accept thread to Issue Another AccepTex SetEvent (HACCEPTTHREAD); BREAK;
Case op_read: // process the data d // ???
// Repost the read if necessary, reusing the same // receive buffer as before memset (& OverlapPlus-> ol, 0, sizeof (OVERLAPPED)); ret = WSARecv (OverlapPlus-> s, & OverlapPlus-> wbuf, 1, & OverlapPlus- > dwbytes, & overlapplus-> dwflags, & overlapplus-> ol, null); if (RET == Socket_ERROR) {IF (Wsagetlasterror ()! = WSA_IO_PENDING) {// error}} Break;
Case op_write: // process the data Sent, ETC. Break;} // switch} // while} // workerthread
Where each handlekey variable, the completion key parameters set when the completion port is associated with the socket;
The OVERLAP parameter returns a pointer to the OverlappedPlus structure that is used to send overlapping operations.
To remember, if the overlapping operation call fails (that is, the return value is socket_error, and the reason is not
WSA_IO_PENDING, then complete ports will not receive any notifications. If the overlapping operation is successful, or the cause is
When a WSA_IO_PENDING error, the completion port will always be able to receive a notification.
The Scholars of Windows NT and Windows 2000 have a basic understanding of Windows NT and Windows 2000's socket architecture for developing a large-induced Winsock application.
Very helpful.
Different from other types of operating systems, Windows NT and Windows 2000 transport protocols do not have a style like a socket, can be
The interface directly talks directly, but uses a more underlying API called the transmission driver interface (Transport Driver)
Interface, TDI). Winsock's core mode driver is responsible for connecting and buffer management to provide a socket simulation to the application (
Implemented in the AFD.sys file) and is responsible for dialogue with the underlying transmission driver.
Who is responsible for managing buffers?
As mentioned above, the application is talking to the transfer protocol driver via Winsock, and AFD.SYS is responsible for buffering the application.
District management. That is, when the application calls the send () or wsasend () function to send data, AFD.sys will copy the data into it.
The internal buffer (depending on the SO_SNDBUF setting value), then the send () or wsasend () function returns immediately. Can also say this,
Afd.sys is responsible for sending data out in the background. However, if the application requests data exceeds the buffer set by SO_SNDBUF
Size, then the wsasend () function will block until all data is sent.
The case where data is received from the remote client is similar. As long as you don't have to receive a lot of data from the application, there is no SO_RCVBUF
Set the value, AFD.SYS will copy the data first into its internal buffer. Data when the application calls the RECV () or WSARECV () function
Copy from the internal buffer to the buffer provided by the application.
In most cases, such architectures are well operated, especially when the application is written in a traditional socket, and the non-overlapping Send () and receive () mode are written. However, the programmer should be careful, although you can choose SO_SNDBUF and SO_RCVBUF through the setsockopt () API
Item value is set to 0 (turn off the internal buffer), but the programmer must clearly turn off the internal buffer of AFD.sys will cause any consequences, avoid
The system crashes that the buffer copy of the buffer when sending and receiving data.
For example, an application is turned off by setting SO_SNDBUF to 0, and then issues a blocking Send () call. In this case
In the case, the system core will lock the buffer of the application until the receiver confirms that the send () call returns after receiving the entire buffer.
It seems that this is a simple method that determines whether your data has received all of the other party, but it is not the case. Think about it, even far
The end TCP notification data has been received, in fact, does not mean that data has been successfully given to the client application, such as the other party may have resources.
The situation of foot, causing AFD.SYS to copy data to the application. Another more tight problem is that only each thread can only be performed.
One transmission is used, the efficiency is extremely low.
Set the S_RCVBUF to 0, and close the AFD.SYS's receiving buffer can not make performance improvement, which will only force the received data in ratio.
Winsock's lower level is buffered, and when you send a Receive call, you should also copy the buffer, so you have wanted to avoid buffering.
Conspiracy in the district will not succeed.
Now we should be clear, close the buffer is not a good idea for most applications. As long as you want to pay attention to you at any time
Keep a few WSARECVS overlapping calls on a connection, then there is no need to turn off the receiving buffer. If AFD.sys always has an application
The buffer provided by the order is available, then it will not necessarily use the internal buffer.
High-performance server applications can turn off the send buffer while do not lose performance. However, such applications must be very careful
To ensure that it always issues multiple overlapping calls, rather than waiting for a overlap transmission to issue the next one. If the application is pressed
One of the orders of sending it will be sent, then wasting the idle time twice to send the middle, in short, it is necessary to ensure that the transmission driver is issued.
After sending a buffer, you can immediately turn to another buffer.
Restrictions on resources When designing any server application, its strongness is the main goal. That is to say,
Your application should be able to deal with any burst, for example, the number of concurrent customer requests reaches the peak, and the memory can be temporarily inadequate.
Other short time phenomena. This requires the designer of the program to pay attention to the problem of resource restriction conditions under Windows NT and 2000 systems.
Processing sudden events.
You can directly control, the most basic resource is network bandwidth. Typically, applications that use the User Data Right Agreement (UDP) may be
Pay attention to the limitation of bandwidth to minimize the loss of the package. However, when using TCP connections, the server must be very careful
Make it, prevent the network bandwidth overload exceeds a certain time, otherwise it will need to retransmit a large number of packages or cause a large amount of connection interruption. About bandwidth management
The method should be determined according to different applications, which exceeds the scope discussed herein.
The use of virtual memory must also be managed carefully. Apply and release memory by cautiously, or apply Lookaside Lists (a high speed
Cache) technology to reuse the assigned memory, will help control the memory overhead of the server application (original "to make the server application
The footprints left by the order "), avoiding the operating system frequently exchange the physical memory of the application application to virtual memory (original"
The operating system always keeps more application address space more in memory "). You can also pass
Setworkingsetsize () This Win32 API allows the operating system to assign more physical memory to your application. You may also encounter the shortcomings of the other two non-direct resources when using WINSOCK. One is the limit of the locked memory page. If you put
AFD.SYS buffer shutdown When the application transmits and receiving data, all pages of the application buffer will be locked to physical memory. This is because
These memory needs to be accessed for the kernel driver, and these pages cannot be exchanged in this period. If the operating system needs to give other applications
Assign some of the pageable physical memory, and there is no problem when there is enough memory. Our goal is to prevent writing from a pathological
To locate all physical memory, let the system crash. That is, when your program locks memory, do not exceed the memory division specified by the system.
The page is limited.
On Windows NT and 2000 systems, all applications can be locked in memory is approximately 1/8 of physical memory (but this is just a big
The estimated estimate is not the basis for calculating memory). If your app doesn't pay attention to this, when you make too many overlap transceiver calls,
And when I / O didn't come, it may occasionally an error in Error_INSUFFICIENT_RESOURES. In this case you want to avoid
Overlock memory. At the same time, it should be noted that the system will lock the entire memory page in which your buffer is located, so the buffer is approaching the page boundary.
Is the price (the translator understands, if the buffer is exactly over the page boundary, then it is one byte, the page of this byte is also located.
Locked).
Another limit is that your program may encounter the shortcomings of the system unsubstructed pool resources. The so-called unpispired pool is a piece never swap out.
The memory area, this memory is used to store some data for access to various kernel components, where the kernel components are not accessible to those
Switch out of the page space. The Windows NT and 2000 drivers can allocate memory from this particular unpized pool.
When the application creates a socket (or similar to open a file), the core will allocate a certain number of memory from the unpized pool.
And when the binding, connecting sockets, the kernel will allocate some memory from an unbeatable page. When you pay attention to observing this behavior, you will find
If you send some I / O requests (such as sending and receiving data), you will reassign more memory from an unbeatable page (for example, to track a certain
I / O operation, you may need to add a custom structure to this action, as mentioned above). Finally, this may cause certain
Question, the operating system limits the amount of unsimposed memory.
On the two operating systems of Windows NT and 2000, the specific quantity of unpamped memory assigned to each connection is different, future versions
Windows is likely to be different. In order to make the application's life period, you should not calculate the specific demand for the memory of the unpredictable pool.
Your program must prevent the limit that consumes an unbeatable page. When the remaining space remaining in the system is too small, some of which is not with your application
The kernel driver of the relationship will be crazy, even cause system crash, especially when there is a third-party device or driver in the system, it is easier to occur.
Such a tragedy (and cannot predict). At the same time, you still have to remember that the same computer may also run otherwise other than the other consumption
Applications, so when designing your application, it is especially conservative and cautious about the amount of resources.
The problem of insufficient resource is very complicated because you will not receive special error code when you have the above case, usually you can only receive one
Poamed WSAENOBUFS or ERROR_INSUFFICIENT_RESOURCES error. To handle these errors, first, put your application
Configuration adjustment to a reasonable maximum (translator Note: The so-called work configuration means the amount of memory required to run in each of the applications, please refer to
Http://msdn.microsoft.com/msdnmag/issues/1000/bugslayer/bugslayer1000.asp, about memory optimization, the translator's other translation), if the error continues, then pay attention to whether it is a problem with the insufficient network bandwidth. After that, please confirm that you didn't send it.
Too many transceiver calls. Finally, if it is still a mistake, it is likely to have problems with insufficient pool of memory in paging.
. To release the unpaged memory pool space, turn off a considerable part of the application, wait for the system to spend and correct this instantaneous
error.
One of the most common things to accept the connection to requesting the server is to accept connection requests from the client. Use overlapping I / O to accept the connection on the socket
The API is the acceptex () function. Interestingly, the usual synchronous acceptance function accept () return value is a new socket, and
The acceptex () function requires another socket as one of its parameters. This is because acceptex () is a overlapping operation, so you
You need to create a socket in advance (but don't bind or connect it), and pass this socket to Acceptex (). The following is a small
Typical use acceptex () pseudo code:
Do {- Wait a new AcceptEx to complete - Create a new socket and associate with the completion port - Set the background structure, etc. - Send an AcceptEx request} while (true);
As a server-responsive server, it must issue enough AccePtex calls, waiting, once the client connection request is issued
Engrave. As for how many ACCEPTEXs are enough, depending on the type of communication traffic expected by your server program. For example, if entering
The case where the connection rate is high (because the connection duration is short, or the traffic peak appears), then the ACCEPTEX that needs to be waiting is certainly more than that.
There are many cases where the client is connected. Smart approach is to analyze traffic conditions by the app and adjust the AcceptEx waiting
The number, not fixed to a certain amount.
For Windows2000, Winsock provides some mechanisms to help you determine if the number of acceptex is sufficient. This is, in the creation of listening
Create an event when socket, pass the wsaeventselect () API and register the FD_ACCEPT event notification to bring your socket and this event
Associate. Once the system receives a connection request, if there is no acceptex () in the system waiting to be accepted, then the above event
A signal will be received. Through this incident, you can judge that you have enough acceptex (), or detect an abnormal
Customer request (under explanation). This mechanism does not apply to Windows NT 4.0.
A major advantage to use acceptex is that you can complete the client connection request and accept data by calling.
LPOUTPUTBUFFER parameters two things. That is, if the client transmits data while sending a connection, your AcceptEx () call
You can return immediately after the connection creates and receives the client data. This may be useful, but it may also cause problems because
Acceptex () must be returned by all client data. Specifically, if you send an acceptex () call while passing
The LPOUTPUTBUFFER parameter is no longer an atomic operation, but it is divided into two steps: accept the customer connection, etc.
Wait until the data is received. When a mechanism is missing to inform Your application: "The connection has been established, waiting for the customer
Data ", which will mean that the client only issues only a connection request, but does not send data. If your server receives too many of this type of connection, it refuses to connect more legal client requests. This is a common method of hackers to "denial service" attack.
To prevent such attacks, the thread that accepts the connection should pass from time to time by calling the getSockOpt () function (option parameter so_connect_time)
Check the socket in acceptex (). The option value of the getSockopt () function will be set to the time of the socket being connected, or set
To -1 (the representative socket has not been established). At this time, the characteristics of WSAEventSelect () can be used very well to do this. in case
It is found that the connection has been established, but it has not received the data for a long time, then the connection should be terminated, the method is to shut down as a parameter.
The socket to Acceptex (). Note that in most non-emergencies, if the socket has been passed to Acceptex () and begins
But not yet established, then your application should not turn off them. This is because these sockets are turned off, for improving the system
Performance Considers, before the connection is entered, or before listening to the socket itself is turned off, the corresponding core mode data structure will not be dried.
Pick clean.
A thread that issues acceptex () seems to be the same as that of the completion of the port association, handling the thread of other I / O completion notifications.
However, don't forget the thread, you should try our best to avoid obstructive operations. A side effect of Winsock2 hierarchical structure is to call socket () or
The upper structure of the WSasocket () API may be important (the translator does not quite understand the original meaning, sorry). Each acceptex () call needs to be created
A new socket, so it is best to have an independent thread to call acceptex () without participating in other I / O processing. You can also use this
A thread performs other tasks, such as event records.
The last consideration for Acceptex (): To implement these APIs, do not require the Winsock2 implementation provided by other providers. at this point
Other APIs unique to Microsoft are equally applicable, such as transmitfile () and getacceptexsockaddrs (), and others may be added
Enter the new version of Windows API. On Windows NT and 2000, these APIs are implemented in Microsoft's underlying provider DLL (Mswsock.dll)
, Can be called by compiling the connection with the mswsock.lib, or via WSAITL () (option parameters
SiO_GET_EXTENSION_FUNCTION_POINTER Dynamically obtains a pointer to function.
If the function is called directly without the function pointer in advance (that is, the compile time is static, the mswsock.lib is compiled, in the program
Direct calls directly), then performance will be affected. Because acceptex () is placed outside the Winsock2 architecture, it is urgent every time you call it.
Get the function pointer through WSAIOCTL (). To avoid this performance loss, applications that need to use these APIs should be called WSAIOCTL ()
The pointer to the function directly from the bottom of the provider.
See the Figure 3 socket architecture:
Application || / || / // Winsock 2.0 DLL (WS2_32.dll) || / || / // Layered / Base ProvidersrsRSVP | Proxy | Default Microsoft Providers (mswsock.dll / msafd.dll) || / || / // Windows Sockets kernel-mode driver (AFD.SYS) || / || / // TRAMSPORT Protocolstcp / IP | ATM | Other
TransmitFile and TransmitPackets Winsock offer two functions that have been optimized for file and memory data transfer. TranitFile () This API function can be used on Windows NT 4.0 and Windows 2000, while transmitpackets () will be implemented in future versions of Windows.
.
TransmitFile () is used to transfer the file content through Winsock. Usually the way of sending files is, first call CREATEFILE () open
A file, then call readfile () and wsasend () until the data is sent. But this method is very efficient, because
For each call readFile () and wsasend () involve the conversion from user mode to kernel mode. If you change
TransmitFile (), then just give it a handle of the file and the number of bytes to be sent, and the mode conversion operation involved will
Just occurs once when you call CreateFile () open file, then transmitfile () then occurs once. This efficiency is much higher.
TransmitPackets () is further more than transmitfile (), which allows users to send only multiple files and memory that can be sent once.
Buffer. The following function prototype: BOOL TransmitPackets (SOCKET hSocket, LPTRANSMIT_PACKET_ELEMENT lpPacketArray, DWORD nElementCount, DWORD nSendSize, LPOVERLAPPED lpOverlapped, DWORD dwFlags); wherein, lpPacketArray structure is an array, where each element can be either a memory buffer or a file handle , This structure
It is defined as follows: typedef struct _TRANSMIT_PACKETS_ELEMENT {DWORD dwElFlags; DWORD cLength; union {struct {LARGE_INTEGER nFileOffset; HANDLE hFile;}; PVOID pBuffer;};} TRANSMIT_FILE_BUFFERS; wherein each field is self-describing type (self explanatory). DWELFLAGS field: Specify the current element is a file handle or a memory buffer (respectively through constant TF_ELEMENT_FILE and
TF_ELEMENT_MEMORY designation); CLENGTH field: Specifies the number of bytes that will be sent from the data source (if it is a file, this field value is 0 means sending the entire file); the structure in the structure: the memory buffer containing the file handle (and possibly The offset).
Another benefit of using these two APIs is that you can reuse the socket by specifying the TF_REUSE_SOCKET and TF_DISCONNECT flags.
Whenever the API completes the data transmission operation, it will be disconnected at the transfer layer level, so this socket can be re-supplied.
Acceptex () is used. This optimized method programming will alleviate the pressure of the thread that is specifically acceptable (former explanation)
and).
These two APIs also have a common weaknesses: Windows NT Workstation or Windows 2000 Professional Edition, the function can only
Handle two call requests, only in Windows NT, Windows 2000 Server Edition, Windows 2000 Advanced Server Edition or Windows
Full support is obtained in 2000 Data Center.
Look down together
In the above sections, we discussed the development of functions, methods, and resources that may be encountered by developing high-performance, large responding applications. What does these mean you? In fact, this depends on how you construct your server and client. When you can be on the server and guests
When you design a better control, you can avoid bottleneck problems.
Look at a demonstration environment. We want to design a server to respond to the client's connection, send request, receive data, and disconnect.
Then, the server will need to create a listener sleeve and associate it with a completion port and create a work thread for each CPU.
Create a thread to specifically use AcceptEx (). We know that the client will send data immediately after issuing a connection request, so if
We are ready to receive buffers will make things easier. Of course, don't forget to poll the acceptex () call from time to time.
Word (using the so_connect_time option parameter) to ensure no malicious timeout connection.
There is an important issue in this design to consider, we should allow how many acceptex () will wait. This is because every time
When acceptex (), we need to provide it with a receive buffer for it, then there will be a lot of locked pages in memory.
Each overlap operation will consume a small portion of the unpaged memory pool, while also lock all buffers involved). This problem is difficult to answer
There is no exact answer. The best way is to make this value can be adjusted. You can get it by repeated performance testing.
The best value in a typical application environment.
Ok, when you measure clearly, the following is the problem of sending data, the focus of considering what you want the server to handle how many concurrent processes
connection. Typically, the server should limit the number of concurrent connections and the transmission call waiting for processing. Because the number of concurrent connections is,
The more memory cells consumed; the more sending calls waiting for processing, the more memory pages being locked (careful than exceeding the limit)
. This also requires repeated test to know the answer.
For the above environment, you usually do not need to turn off a single sleeve buffer because there is only one operation of the data in acceptex (), and
It is not too difficult to ensure that the connection to each arrival is not too difficult. However, if the client is interacting with the server
Change, the client will need to send more data after sending a data, and it is not very wonderful in this case.
Non you want to guarantee that overlapping reception calls are issued in each connection to receive more data.
in conclusion
The Winsock server that develops a large response scale is not very terrible, in fact, it is to set a monitor sleeve, accept connection requests and heavy
A stacked hair call. By setting a reasonable number of overlapping calls, preventing the memory pool from being exhausted, this is the most important
Challenge. According to some of the principles we have previously discussed, you can develop a larger responsible server application.
