Concurrently
l iteration and concurrency
The iterative server first handles all requests that have accepted before the service is provided for the subsequent customer request. When processing a request, iterate the server or the other requests will be issued into the queue or abandon. Iterative Server is particularly suitable for one of the following circumstances
Ø Short-term long service, and the long-term change is small. For example, the standard Internet Echo and DayTime service.
Ø The service is not running. For example, a remote file backup service runs lightly in the system load.
The design and implementation of iterative servers is relatively simple because he always performs his service in a single process address space, as shown.
The service implements the pseudo code as follows:
Void ITERATIVE_SERVER (Void)
{
Initialize Listener Endpoint (s)
For (Each New Client Request) {
Retrieve Next Request from an Input Source
Perform Requested Service
if (Response Required)
Send Response to Client
}
}
This iterative structure enables the process of each request to serial in a relatively rough level, and a bit similar application is performing a SELECT operation. This rough hierarchy makes the application unable to take advantage of the determined system processing resources (such as multi-CPU) and operating system features (such as support for parallel DMA). The iterative service structure blocks the client when processing the request, allows the client to not perform other actions. The client's blocking also makes the implementation of the time-out retransmission mechanism become difficult, and greatly increases the additional traffic of the network.
The concurrent server simultaneously and sends a plurality of requests from the customer, as shown above. Different requests are processed separately on the operating system and hardware platforms used, concurrent servers, or separate processing requests on different CPUs or on a single CPU. If this server is a single service server, multiple copies of the same service can run simultaneously. If this server is a multi-service server, multiple copies of multiple services can also be run. The concurrent server is ideal for IO boundary operations and processing time variable long-term long-term long service. Compared to the iterative server, multi-threaded concurrent servers allow more fine synchronization techniques, such as database lock, and so on. This service design requires concurrent control mechanism, such as Semaphores or Mutex Locks, to ensure a stable and reliable operation and sharing of data between each active process / thread. There are many ways to construct concurrent services, such as multi-thread / multi-process. A commonly used concurrent design is a thread mode, where a service distribution thread is distributed to a different request to a different thread's service server. Here is its work pseudo code:
Void Concurrent_Service_Dispatcher (Void)
{
Initialize Listener Endpoint (s)
For (Each New Client Request) {
Receive Request
Spawn New Worker Thread and Pass Request to this Thread
}
}
Void worker_thread (void)
{
Perform Requested Service
If (Response Required) Send Response To Client
Terminate thread
}
This model is easily modified to support other concurrent server patterns, such as each connection thread mode.
Void Concurrent_Service_Dispatcher (Void)
{
Initialize Listener Endpoint (s)
For (Each New Client Connection) {
Accept connection
Spawn New Worker Thread and Pass Connection to this Thread
}
}
Void worker_thread (void)
{
For (Each Request On The Connection) {
Perform Requested Service
If (Response Required) Send Response To Client
}
}
Each connection thread provides a good support for customer request to priority queuing services. For example, higher priority customers can bind to threads with higher priority so that client requests from higher priority will give priority.
l process and thread
The concurrent server can be implemented with multi-process and multi-threaded methods.
Multi-process: A process is an entity of an operating system, which provides a context for the execution of the program instruction. Each process contains and manages some resources such as virtual memory and IO manipulation, while isolating other processes. A plurality of processes communications use shared memory. Disadvantages are unable to implement fine control, and some applications cannot be implemented.
Multi-thread: Most OS platforms support multithreading in a process. A thread is a simple instruction sequence in the contemplated context in a process. A thread contains and manages some determined resources such as Run-Time Stack, priority, and specific thread data, and more. If there are multiple CPUs, the service running on the multi-thread can be handled in parallel.
Multi-threading more processs can reduce the following concurrent overhead:
1. Create and context of threads: The status information whose thread is retained is less than the process, so the additional overhead of creating threads and context switches in the thread is smaller than the process. For example, when switching between the same process, the resource range of the process ranges, such as virtual address mapping and cache, does not need to be changed.
2. Synchronize: When scheduling and executing the application thread, there is no need to switch between core mode and user mode. Similarly, the thread synchronization is much smaller than the cost of synchronization between the processes. This is because the synchronous objects in the thread are usually local (no core layer conversion). It is better than the global shared memory between the processes (involving the system core layer).
3. Data copy threads can share information through local process memory intervals, which is usually more frequent than the process of transmitting mechanisms using IPC messaging mechanisms. This reason is that the copy of the data between the different threads of the process does not need to pass the system kernel. Typically, use a process of shared address space to implement line-up communication than inter-process communication between the shared memory or the local IPC mechanism is much more effective.
Although many operating systems support multithreading, it does not mean that all applications must be multi-threaded. In fact, there are many restricted places using multithreading:
1. Robustness: In order to reduce the overhead of the context switching and synchronization, the protection between the thread is very small. Because each thread is not very good in the same process space, a defective service implementation in the process can affect the service running in the process other threads through global sharing data. This is likely to lead to unpredictable consequences, thereby causing the entire process even causes the web server to hang. In addition, some of some operating systems may have a negative impact on the entire process when used in threads. For example, the exit call in the UNIX system will close all threads in the calling process.
2. Another problem with access to multi-thread is that all thread sharing in a process uses the same user ID and access to the protected system resource. It is necessary to prevent accidental or intentional access to unauthorized resources, a lot of network services such as Telnet run in different separation processes.
3. Performance A regular confusion concept is an application that uses multi-threading will increase performance. In many cases, multithreading does not provide improvement in performance. For example, calculating applications running on a single CPU (UNI-Processor) cannot be beneficial from multi-threads. The reason is that the computing task does not need to be inter-inter-communication. Similarly, very fine lock will produce a high degree of synchronous overhead, which will cause applications that cannot make full use of parallel processing. In some cases, multithreading will significantly improve system performance. For example, the IO operation application of single CPU (the advantage here from multi-threaded service overcomes the communication service and disk operation service interveir to run) L active distribution and needs to respond to delay process / thread distribution strategy
There are a variety of different strategies when distributing processes and threads. Different strategies can optimize the performance of concurrent servers in different environments. These strategies will be discussed below to adjust the developers to adjust the concurrency level of the server based on customer needs and available OS processing resources.
Positive distribution: This policy launches one or more system processes or threads when the service is created. This thermal start-Started execution mode can initiate multiple services before the request can be in response to improve the response time of the system. The number of startup depends on a series of factors: the number of CPUs, the current machine load, and customer request queue time, etc. The picture below is a schematic of one of his implementation.
This policy is required to distribute a new subordinate process or thread when the customer's new connection creation or customer request arrives. Its examples include each request thread and each connection thread. This policy can minimize resource overhead when distributing threads / processes and start-up service costs. The following figure shows the schematic of this policy. The defects in this strategy are: this strategy will reduce the performance of the system in real-time systems and large load conditions.
Delayed distribution The application of delayed distribution strategy is usually assigned initial threads or process sets with active distribution or time distribution policies. Of course this set is usually quite small, such as a single thread or process. When the request arrives, the existing threads and processes will process these requests. If this request is processed more than a specific time, another thread / process will be activated to respond to the request arriving below.
Typically, the selection of the strategies listed above is compromising between resource consumptions increasing in the reduced launch load.
l CPU allocation and thread model
Planning scheduling is the main operating mechanism of the operating system to ensure the use of system resources. Because the thread is a unit scheduling and executed in the multi-thread process, the system resources most concerned here are CPUs. Today, most platforms have a series of scheduled mechanisms to manage applications created by applications. Not all platforms allow you to change the system resource allocation of the thread, but in any case, you should know how you can do when you work under your application platform and how the system is scheduled for resources. This makes it possible to optimize your application.
N: 1 User Thread Model: Early thread implementation is to manage only by the operating system process control machine, which is fully managed by the user space library. Therefore, the OS kernel is hardly known threads. The kernel is just a scheduling process, and the library in this process manages the N threads. This design is called "N: 1" model, sometimes called the user thread model. The advantage of this thread model is that the system core does not have any ready-made life cycles and context switching processes in the same process. Therefore, the establishment of threads, deletion and context switching efficiency are high. But this model has two problems, what is their reason is that the system kernel is ignorant of threads.
1. Regardless of how many CPUs in the system, each process will only be on a CPU. All threads of the process will compete for this CPU, which is available in the schedule of the process.
2. If a thread has a blocking operation, such as reading and writing, all threads of the process will block until this operation is completed.
1: 1 The core thread model is to solve the problem of N: 1 model, a more advanced thread model being adopted. He is 1: 1 model, where the OS kernel supports threads. Each thread is established directly by the system kernel, the kernel is responsible for scheduling each of these threads to the system (if there are multiple lithography) CPUs. This model is also called a core thread model. Linux and Windows NT / 2000 use this model. This model solves the problem of the above N: 1 model because the system kernel participates in the entire life cycle of the thread includes establishment and scheduling.
N: M Mixed Thread Model: Some operating systems now provide a mixed model of N: 1 and 1: 1, for example, Solaris. N: M model. This design supports the mixing of user threads and core threads. But the application creates a thread, the thread library will establish a user thread, but the core thread is created only when the application needs or when it is displayed.
No matter how powerful tools, abuse and wrong use will hurt yourself. The same is true, then how do you choose the right thread model in these threads? Here are these considerations to get, when you create a new thread.
Ø Separate CPU (CPU-INTENSIVE TASK) Task If your newly created thread comes from letting a heavy task have its own space, make him dispatch in their own range and to compete in other threads in the application At this time you should consider the core thread mode, that is, 1: 1 mode. This strategy avoids the thread range scheduling overhead, and makes the OS to make full use of multiple CPUs (of course, there must be, huh).
Ø Simplified System Design If your main purpose is to logically divide your application into several separated tasks, you don't need to use core thread mode. If you use user thread mode, while reaching logical decomposition services, you can avoid system core overhead when thread creation and scheduling.
l Task-based and messaging-based concurrent architecture
The concurrent architecture includes various units (such as layers, functional functions, connection, and messages) of application services, and various logical / physical CPU configurations. The concurrent structure of web applications is one of the factors affecting application performance (including protocol, bus, memory, and network interface features.). The three basic parts of the concurrent system include:
1. CPU, this is the underlying execution agent of application code
2. Data and control messages for sending and receiving between one or more applications or network devices.
3. Service processing, based on the execution of the message completion task
Based on this classification, there are two types of concurrent architecture based on tasks and based on messages.
Task-based concurrent architecture This design is assigned an organization CPU based on the unit of service function in the application. Here, the task is active, the message being handled is passive. The following is a schematic illustration. The acquisition is obtained by performing service tasks on their respective CPUs and delivering data messages and control messages between task / CPUs.
Message-based concurrent structure: This design can allocate organizational CPUs based on messages from network applications and interfaces. In this architecture, the news is active, and the task is passive. As shown in the following example. By ensuring a series of data and control messages on each of the different CPUs, it is concurrent to perform concurrency. Each request thread, each connection thread and thread pool are based on this message-based concurrent structure.
Concurrent structure
The choice will affect the key elements of the application performance, such as context switching, synchronizing, and data mobile overhead. Messaging-based concurrent structure is usually more effective than task-based concurrent structures. This is why he is widely used. However, it seems that the task-based concurrent structure is more simple in implementation.
