TWTCAD parallel computing program description
The parallel computing part in the program mainly uses Remoting technology in .NET, the overall calculation code is split into partial code that can be calculated in parallel, and these partial code is encapsulated in different objects, and these objects can be remote through Remoting technology. Distributed in the network, that is, these objects can be created in other computers in the network, generating a proxy in the local computer. There are several clients that generate several remote object agents in the server side, and each agent has a separate thread to maintain, and the agent is executed concurrently in the server side, and the actual remote object is in parallel in the network. (Note concurrent and parallelism).
The static structure and dynamic structure of the program have been described in Visio, which will be described here. Some special parts and techniques in the program will be explained.
I. Management of remote objects
Each remote object corresponds to a proxy on the server side. The management of the remote object is the management of the agent, and a thread is created separately in the server side for the server, no thread is called for remote object agent. The synchronization mode is used, that is, it is bundled, but this does not affect the operation of other agent management threads, so the program is integrally shown or concurrent execution. Due to a large number of multi-threaded process technology, its management issues are very complicated. Here, the task solver defines an abstract interface, and then defines a clientagent for multi-threading, encapsulates a thread and task solver interface to implement multi-threaded processing on all task solve processes.
Second, the transmission and storage strategy of grid data
First, the remote object can save the status, which solves the distributed storage problem of the data grid.
The grid data is finally transmitted to the server before the client is calculated, and the server is saved on the server. When all the grids are calculated, the server will put these grid data according to static tasks and dynamics. The assignment policy of the task is redistributed and transferred to each client. Theoretically, multi-threaded transmission can improve the utilization of the network, although the load on the server will be increased. However, after analysis of the transmission process of the grid data, the storage space of the grid data is very large, and the access to the grid data in multi-threaded transmission mode is the distributed distribution of the distributed distribution, that is, each thread. The data accessed is not simultaneous, which violates the local principles of the data, there is no such problem for large memory servers, but for small memory servers, this will inevitably cause the scheduling of the disk, make Data is frequently exchanged between physical memory and page files, thereby reducing the speed of the server. However, if a single-threaded transmission is used, although the scheduling problem of the disk is solved (the number of disk scheduling is minimized), the transmission speed is too slow, so the program is used in the program, and the static task uses a single-threaded transmission. The dynamic task uses multi-threaded transmission. (The data per client receives is the same).
However, in the actual test, it is found that when the client is less, this solution works very well, the grid transmission speed is very fast, but when the number of clients increase (> = 5), the transmission speed of the public grid part The average utilization rate of the server-side network is very low. After analyzing, when the client is too much, the server-side concurrent thread is significantly increased, and the frequent scheduling between threads may cause the system to reduce the efficiency of the system. On the other hand, the client is too much to cause concurrent line transmission public grid. Thus, the large-range distributed grid is transmitted, which has made frequent dispattrans of the disk. Therefore, all grid transmission in the final scheme uses a single-threaded serial transmission, so although the disk scheduling Numofclient is repeated when transmitting the public grid, the efficiency is fixed, and the volatility is not large. Third, the restructuring problem of dynamic tasks:
According to our task assignment algorithm, the purpose of dynamic tasks is to adjust the client's load balancing problem, so that the client's wishes waiting time as little, but actual testing, with the calculation, this adjustment is gradually weakened, and even causing Client load offset issues. Dynamic task sporadic in each grid, the task for IJK is 0 can be deleted, avoiding unnecessary calls, but there are only one IJK in some grids, but to communicate network calls, network transmission Time is much greater than the actual calculation time, although it looks at the load equilibrium state on the client surface, but actually calculates the efficiency, because most of the time is used for network transmission, this situation has deteriorated as the calculation!
In order to solve this problem, you must start from two aspects, one is to adjust the specific gravity of the static task, and set the static task specific gravity in this program to 0.9. On the other hand, the cross-chain table in the dynamic task must be adjusted. In the entire to a list of IJK, the following strategy is used in the program: first sort the dynamic IJK cross chain list, then start from the last element, if This element IJK is less than 500, then merge it into the previous element, so that the dynamic task is reorganized as a large number of large tasks, while reducing the number of tasks, reducing the overhead of network calls. However, the task cannot be multi-merged, ie if all tasks IJK numbers and still less than 500, the task can eventually be merged into one, but a bottom line for the number of tasks: Numofclient / 2, when the number of tasks Even if the number of IJK is small, it will not be merged even if the number of IJK is small.
