CS / CSS architecture application
Software Performance Test Model Analysis
Author: Xia Haitao
Key words: software testing, performance test, C / S architecture test, performance test model analysis
Summary:
This article is part of the classification model in the performance test methodology in the performance test methodology in the knowledge accumulation. Its purpose is to describe the characteristics, metrics and planning implementation of performance testing, metrics and planning implementations, metrics and planning implementation processes for performance testing of CS / CSS architecture applications that have been refined, abstract, abstract, in order to use similar system architectures. The performance test work in the engineering project has played a reference and reference.
1. Basic concept of the CS / CSS system architecture
1.1 system architecture definition
Although the B / S structure, J2EE architecture is increasingly embedded, but the traditional C / S structure-based application is also widely used in various industries. In particular, commercial bank counters - core account systems in the financial industry. On the one hand, because traditional commercial banks generally have a large number of devices that need to be multiplexed, on the one hand, on the one hand, on the one hand, on the one hand, because they have a large number of high-cabinet services that have high real-time requirements, use traditional C / S structures or C The application efficiency of / S / S structure is more guaranteed.
The C / S structure is a Client / Server structure. Traditional C / S structures are generally divided into two layers: clients and server. The basic working principle of this structure is that the client sends a SQL request to the data server, the server returns data and results. The client is responsible for implementing the user interface function and encapsulates some application logic. The server-side database server primarily provides data storage capabilities, and partial application logic is provided through triggers and stored procedures.
The C / S / S structure is the three-layer structure of the client / application server / database server, and the application server is added, and the application logic is usually implemented. It responds to the request of the user to execute a business task and deal with the database server, and the technical implementation is usually used to select the middleware product, such as the Tuxedo and IBM CICS of the BEA. (In fact, the application of J2EE architecture is also a three-layer or multi-layer structure, which is not included here.)
Compared with the two-layer C / S structure, the three-layer C / S structure is mainly manifested in: security, efficiency, easy maintenance, scalability, complicity, and good openness.
1.2 System Architecture Schematic
1.3 Characteristics of performance test in CS / CSS system architecture
1.3.1 Performance Factors of CS / CSS System Architecture
Due to the following characteristics of the CS / CSS system, test engineers have a lot of difficulty for implementing performance testing of a CS / CSS system:
l The various parts of the entire system use a variety of operating systems, and there is different performance;
l Use a variety of databases across all aspects of the entire system architecture, which is also different in performance;
l Applications are multiple, subordinate multiple types, distributed on different devices, including applications, third-party applications;
l System in the system, components are connected, communications through different protocols;
l The internal interface of the system, the performance bottleneck; the overall performance of the system often depends on the worst part; you need to test and jointly test separately.
l The performance indicators of the system are not only related to the application system architecture, but also related to the business model of the specific industry;
l Industry applications using this architecture are often a 7 × 24-hour system;
l Industry applications using this architecture may have more high cabinets, which will affect the selection and conversion of performance metrics;
l Communication in each link is substantially exchanged in the way, the format is often complicated.
Such a system is therefore a test for the depth and breadth of the test engineers. For such a system, how to use what test strategy, how to analyze test requirements, how to select the conversion calculation model of performance metrics, how to determine test content and round, how to design performance test case, etc., and plan and implement performance test Other many problems in the middle need to follow a system method to solve. 1.3.2 Basic Policies of Performance Test in CS / CSS System Architecture
1. Determine the range of test work
First, you can analyze the most prone to bottlenecks in the pressure test, thereby destaining the test strategy or test environment, making the pressure test results true reflecting the performance of the software. For example, the server's hardware restrictions, the database access performance settings often become an important factor in restricting software performance, but these factors are obviously not the user's most concerned. We have to adjust these factors before testing. .
In addition, users are more concerned about which link in the entire system also affects the range. If some links are new systems, and some links are already a mature system just a slightly change, so that the local performance test of the new system requires the system and comprehensive.
2. Analyze the performance test needs of customers
Customers have clearly proposed performance indicators, or only provide user usage methods and historical transaction traffic data, and we need to perform performance benchmarks in our own The purpose of performance test is to verify system performance or want to determine the ideal configuration of the target system? Do you want to use test results to predict the processing power of different models? Whether it requires a targeted answer if there is a problem such as performance tuning processes such as performance testing.
3. To make performance testing plan and program
Pay attention to the problem of the problem proposed in the test requirements analysis phase in the test plan and program.
4. The determined test passes the plan, the performance test plan, the result should be approved by customers
To confirm with the customer, the system's performance indicator is the standard of standard; for the planning and results of the various parts and steps in performance testing, even the performance testing process, it is necessary to determine whether the customer is confirmed and signed according to its importance. Getting customer recognition is the most important.
1.3.3 Performance measurement and performance detection in CS / CSS systems
u Performance measurement
1. Must be carefully planned for performance measurements before the performance test:
Software performance measurement technology is wide. You can include log, event count, event duration, sample and other performance measurement techniques.
l Determining the performance measurement: What should we test?
l What kind of measurement tools are used in planning performance tests.
2. Representative of measurement
l The measurement results should be able to reflect important factors affecting performance: workload load, software and computer system environment.
3. Measureable repeatability
l The workload load, software, and computer system environment can be controlled to repeat the test process.
u Performance detection technology
When performing performance measurements, standard commercial tools can be used, but data that are often provided by standard tools cannot meet the requirements. Performance probing is to insert code probes in the key point of the program to measure the execution characteristics of the software. Thus the following goals:
- Performance data acquisition is more convenient
- Data of data is improved
- More controllable data collection
According to SPE (software performance engineering), software detection needs should be part of the software architecture. Design software probes when designing software.
Therefore, during the performance test in the planning project, it is recommended to consider the design of the island performance detection in software design, and prepare better performance measurements in performance testing.
1.3.4 Type of Performance Test under CS / CSS System
Generalized performance tests include many types. Such as:
• Scalability / Load Testing (large-scale / pressure test)
By increasing the pressure on the measured system, until the performance indicator, such as the response time, more than a predetermined indicator or a certain resource has reached a saturation state. This test can find the system's processing limits, providing data for system tuning. • Performance Testing (Performance Test)
Whether the performance of the production and operation of the production operation and the performance of the test system are used to meet the production performance requirements. For example, with the actual production of structural test, the maximum throughput and the best response time are obtained to ensure the smoothness, safety, etc.
• Configuration Testing (Configuration Test)
Find the optimal allocation principle of various resources of the system by testing.
• Concurrency Testing (concurrent test)
Testing multiple users simultaneously accessing the same application, whether there is a deadlock or other performance issues when the same module or data record is recorded.
• Stress Testing (extreme test)
The test system can handle the session capabilities that the system can process, such as CPU, in-saturation, saturation, and whether the system will occur if there is a saturated state of saturation.
• Volume Testing (Capacity Test)
The test system can handle the maximum session capability.
• Reliability Testing (reliability test)
After loading a certain business pressure (e.g., the resources of 70-90% of the usage rate) to the system, run for a period of time.
• Failover Testing (failed test)
For systems with redundant backups and load balancing, the user will receive much impact if the system locally causing the user can continue to use the system.
In the actual performance test under the CS / CSS system, you need to select and combine the performance test types according to the specific situation.
1.3.5 Component of Performance Tests under CS / CSS System
Typically in a CS / CSS system, several hierarchies such as user interface layers, service logic layers, and data service layers are used, respectively, respectively, corresponding to customers, application servers, database servers. For example, the client carries the counter service, deploying the outlet (including the character teller or graphic teller), but also includes self-service services on the self-service device; the application server is mainly routing function, business Processing functions, and channel integration, and core business processing systems include trading platforms, business logic, core processing, data processing, and more.
Since business logic is distributed in different links, there are many internal interfaces of the system, and the performance bottleneck is more, and the overall performance of the system will often depend on the worst part. Therefore, testing for overall performance of the entire system may need to do its respective internal performance testing for each link.
Examples of the following CS / CSS system financial industry application:
Figure 1-3 CS / CSS Financial Industry Application
In order to test the performance of the entire system, it is necessary to perform internal performance testing in advance, such as the pressure test of the background host, the pressure test of SNA Gateway, the pressure test of the front system, the pressure test of the front-end system, external system access Pressure test, etc.
In an internal pressure test, in order to exclude the other parts of the system, it is necessary to isolate the respective portions, drive, and piles to use software test tools or self-compilation. In the internal pressure test of each portion, performance measurements can be performed using the various performance test types described in the previous section according to the specific case.
2. Metrical calculation model of performance test in CS / CSS system architecture
2.1 Defining Metrical Standards
When performing a model analysis of performance test, you must first identify key performance goals. It should be obtained by communicating with customers, these goals should be to solve the performance problem of customers, that is, customer performance test demand reflects the key performance goal. Performance goals should be clear and measurable. For example: support 2000 concurrent users; continuous operation of 30 days without stopping. After clarifying the pass / fail criteria for key performance goals and performance tests, you need to define how to measure critical performance goals and performance tests. Metrical standard items affect the choice of test methods and test tools. For example, if you want to measure the response time of 100 users, it must be clear which standard items below the metric:
l Each response time of a user
l In the case where 99 users have been accessed, the response time of the 100th user
l The two indicators must metrologize
2.2 Performance benchmark and test intensity estimation
In fact, key performance goals are not always very clear. Customers often only have some expected proportions of historical data and business growth. But these data is also useful for us, they can serve as performance benchmarks used in our design and test.
For performance testing, the design performance benchmark is first proposed when designing. The so-called performance benchmark is to think: How many people use this system? What is the maximum number of users in use? How much time interval in the peak period of users, what is the system response time in how large levels? How big is the amount of data growth? What is the increase in data growth and time long? What is the condition of network implementation? How to control the CPU and memory growth when processing? All of these, then optimized according to the performance reference implementation and hardware configuration, and the test is only validated whether the system reaches the desired design goal.
But the current situation is often: after designing, the performance test is required, what is testing, what is the best, but there is no standard answer to the exam, but ask everyone to look. Or, although the customer has made critical performance indicators, it is not considered when designing, depends on performance tests to give actual performance data, and then optimize. Performance tests can be obtained based on performance benchmarks, especially based on critical performance goals that have been calculated and converted. This requires testing engineers to repeatedly instill this concept to designers. Otherwise, performance testing includes tool research always stay in the discussion stage. Don't consider the optimization problem after the completion of the coding. If the project is implemented, the optimization has not been completed, it is difficult to ensure customer satisfaction.
There is no target design, just like the traffic problems between the city, we complain that the builders lack vision, and software designers lack imagination.
For the transformation of performance reference to key performance targets, it will be described in the following examples.
200 users use client software to perform business processing (and distribute at least 200 concurrently), the total traffic of 20000 million services per year is: 20 million business / year.
The following assumptions are used in the test intensity estimation:
◇ The annual business volume is concentrated in 10 months, 20 working days per month, 8 hours each day; ◇ Adopt 80-20 principle, 80% of the business day of 20% Completed within, 80% of the business per day is completed within 1.6 hours;
Estimate results of test pressure:
Last year, there were about 20 million business in the whole year, 15% of which were processed for 3 requests per business; 70% of business handles 2 requests for application servers for each business; the remaining 15% of business Each service submits 1 request to the application server. According to the previous statistics, the annual business increment is 15%, considering the need for business development in the next three years, the test is subject to 2 times the existing traffic. The total number of requests is: (2000 * 15% * 3 2000 * 70% * 2 2000 * 15% * 1) * 2 = 80 million times / year.
The number of requests per day is: 8000/200 = 400,000 times / day.
The number of requests per second is: (400000 * 80%) / (8 * 20% * 3600) = 55.6 times / sec.
Under normal circumstances, the ability of the application server handling the request should be at least: 56 times / second.
Or, ignore the number of requests submitted, subject to the number of business transactions, define the number of transactions per second, and "throughput".
If the increase in trading volume in the next few years (15% annually), it can be summarized as:
• Throughput: (T4 * 80%) /(1.6*3600)
- T4 = t0 * (1 15%) ^ 4
- T0: Current average daily transaction volume of 20 million
- T4: Trading volume in the next 4 years
In addition, sometimes the determination of key performance indicators and specific applications are related. For example, there will be batch, monthly and other batch processing, and often use a batch to characterize performance indicators.
2.3 Metrical standard items and collectable measurement data item conversion
Only use clearly collected data can truly reflect the performance of the system. E.g:
l Run the number of successful transactions per second
l The response time of a single client (the difference of the timestamp, the time to send the request, and the time to receive the response)
l CPU usage
l Network traffic occupancy
l Memory usage
l hard disk usage
2.4 Decomposition of pressure
For a complex system consisting of many links, if you want to simulate the integral of the actual environment, you need to decompose the overall pressure.
Such as: The figure below is a joint performance test for actual systems. The upstanding throughput of the background host system is 480 / sec. . Since the communication gateway and the host are one-to-1 relationship in this environment, the pressure of the communication gateway should reach 480 pens / sec. And a communication gateway corresponds to three front machines, so each front machine is to reach 160 / sec to deliver the host's throughput, which may make the entire system full load operation. Taking into account the other levels. Since there are other service systems other than the host, a "X" represents the pressure required for other service system on the pressure of the front machine. When a level of equipment is short, it is often necessary to reach the pressure of its decomposition, often need to use software tools, directly addressed directly to the previous level.
Figure 1-3 Combined performance test example
3. Planning and implementation of performance test in CS / CSS system architecture
3.1 Human Resources Plan in Test Plan
Since performance testing is more complex, the program of human resources in performance test programs is also more important. It is necessary to fully consider the human resource demand of various work tasks in testing organizations, test procedures, test design, implementation, and implementation, test reports, etc.
Under normal circumstances, the performance testing of an engineering project is taken as the following roles and responsibilities:
Test Analyst: Responsible for analyzing test strategies, writing test plans, formulating test programs, tissue testing;
Test engineer: test case design, realization; environmental coordination; analysis of test results, write test summary reports. Tests (usually testing engineers): test execution; record, collect, and organize test record data for testing processes.
Software engineers (sometimes consumed by test engineers): responsible for writing, debug client testing software and simulation software; database management system installation, OFS configuration, and system preparation.
System engineer (sometimes concurrently determined by test engineers): Hardware maintenance and operating system installation, configuration, responsible for testing.
The person in charge of the superior test: is responsible for approving the test plan and test summary report.
Customer: You can participate in the test if necessary, and make specific test requirements; you can request a pause test; important test conclusions should be signed and approved.
3.2 Select the appropriate test tool for the project
In the performance test, you must have a suitable test tool support.
Test tools used in performance test include:
• Load analog type tool
• Performance Measurement Tools
• System-level measurement tool: CPU, memory usage statistics
• Statistics class: Response time, throughput
• Analyzed class: Code level measurement tool, such as statistical function call
For test tools, there is a difference in the demand for each specific project, and there is no universal solution. Moreover, the introduction of tools takes time, funds, and human investment.
The choice of test tools requires consideration of the needs of the measured system in performance testing, as well as the feature that the test tool needs to be completed. In general, the options include:
u real production system
U commercial pressure test tool
u custom pressure test tool
The first choice is limited to resources and exact factors that are generally not adopted in pressure testing, which will not be discussed here. The main consideration of the main consideration of the next two choices include:
N Can I meet the need to serve as analog programs in the pressure test, load simulation
N Can I provide detailed, accurate performance measurement data
n Test tool limit factors in cost, including time and money
N test group's level of testing tools
There are a lot of good automation performance test tools. Such as: MI's Loadrunner, Mi Astra LoadTest, EMPIRIX E-LOAD, Rational TEAMTEST, etc. Among them, MI's LoadRunner is most famous and common.
Sometimes the program of self-editing or custom pressure test tools is also used during performance testing, mainly based on the following reasons:
First, due to the characteristics of the test system itself, it is difficult to find the commercial test tool that the simulation program needs, even if there is a test tool close to this, the risk of the testing process will increase the risk of the testing process.
Second, sometimes due to the maturity of the relevant technologies, the option to customize the pressure test tool does not have uncontrollable risks in design, implementation, or in the test tool, and in the testing process, it meets the timely aspects of the test needs during the test. Customized test tools have an unparalleled advantage.
Finally, considering the productization of the pre-system in the future, the need for further testing for the system will last for a long time, and the custom test tool can be better and more completely satisfied. At the same time, it can learn from this custom test tool for projects with the object system to use the same system architecture, and quickly establish new test tools quickly.
3.3 Test Preparation Work
Preparation of performance tests, can include preparations for test data, test tool preparation, test environment preparation, test implementation, etc.
3.3.1. Preparation of Test Data
For industry applications, especially financial industry applications, the most important thing in the preparation of test data is the selection of trading. The choice of the transaction has the following principles: internal pressure test: Try to select a few most consumable transactions, cover all trading forms (such as session, batch, asynchronous), so that you can maximize check Out of the performance bottleneck;
Overall joint pressure test: Since the general integrated pressure test needs to completely simulate the actual production situation, the sample selection of the transaction is relatively complicated. It is often necessary to collect and predict performance test transactions, more importantly, the distribution of transaction transmission ratios.
Distribution of transaction transmission ratios such as an actual financial project:
Table 1-1 Financial transaction proportional distribution
trasaction code
Original ratio
Transmission ratio
Trading code 1
0.49%
16.90%
Trading code 2 0.05%
1.72%
Trading code 3 0.01%
0.34%
Trading code 4 1.00%
34.48%
Trading code 5 0.98%
33.79%
Trading code 6 0.37%
12.76%
First select the top 50 transactions in the actual original transmission ratio. Then add all the ratios (certain less than 1), as the new base, respectively, respectively, the ratio of the use of tool simulation transmission, respectively, can be obtained.
In addition, the acquisition of major actual transaction data is usually extracted from the database; it is also possible to use certain rules to produce some data.
3.3.2. Preparation for Test Tools
Typically, it is necessary to reserve sufficient time for the preparation of the test tool. For commercial test tools, the recording and modification of the script is usually required, and the setting of the scene; for self-editing tools, there must be design, encoding process. Moreover, it usually has its own profile and the data file used, requires configuration or data format conversion.
3.3.3. Test environment preparation
The problem you need here is that it is possible to generate embedded test data in the background (acting) database, to ensure that it is sufficient and can be generated or easy to clean.
In addition, the test environment is ready, the inspection environment before execution is also very important, and a lot of useless work can be avoided.
4. Performance test report
4.1 Test structure data sorting and analysis
u test report documentation
1. Test files that should be generated in the test
Test record (test person in charge and personnel signing participating in the test);
Test summary report.
2. Contents that must generally be included in the test summary report
Test software name, test item, test environment; pressure test conclusions for test software: response time, maximum / minimum, failure, the maximum / shortest time of normal continuous operation, and the relationship between the number and failure, etc.
In short, you must include data, change curve analysis, and conclusions of predefined key performance indicators.
5. Problems need to pay attention to in performance testing under the CS / CSS system architecture
5.1 Note that the number of license, database of the middleware, etc.
Sometimes the test results are not expected and is not necessarily the problem of being subject to the object. It may be that the License of the middleware or the limitation of the number of concurrent users used by the database system, sometimes it is necessary to comprehensively analyze problems. .
5.2 Data aggregation problem
The data selected in performance test should be dispersed on the difference, similar to the actual production data, and the system is fully tested under different data. If you are using a more single data, the side may not use the local function of the system, resulting in unreliable performance test data; additionally, if the system has an optimized processing function for the same data processing, the performance test data is unreliable.
