Performance test plays an important role in the quality assurance of software, which is rich in testing content. China Software Evaluation Center summarizes performance tests into three aspects: Applications to client performance tests, testing and applications on network performance testing on server-side performance testing. Typically, in three ways and effective, reasonable combinations can achieve comprehensive analysis and bottleneck predictions for system performance.
Applied on client performance test
The purpose of the application in the client performance test is to examine the performance of the client application, and the entry of the test is the client. It mainly includes concurrent performance testing, fatigue strength testing, large data quantity testing, and speed testing, which is the focus.
◆ Computation performance test is the focus
The process of concurrent performance test is a process of load testing and pressure testing, which gradually increases the load until the system's bottleneck or cannot receive performance points, the system concurrent performance process is determined by integrated transaction execution indicators and resource monitoring metrics. Load testing is determined to determine the performance of the system under various workloads. The target is that the corresponding output item of the system components when the load is gradually increased, such as by volume, response time, CPU load, memory usage, etc. The performance of the system. The load test is an analysis software application and a support architecture that simulates the use of a real environment, thereby determining the performance process capable of receiving. Stress Testing is a test of the maximum service level of the system by determining the bottleneck of a system or the performance point that cannot be received.
The purpose of concurrent performance test is mainly reflected in three aspects: based on real business, choose representative, critical business operation design test cases to evaluate the current performance of the system; when expanding the functionality or new application When the program will be deployed, the load test helps determine if the system can also handle the desired user load to predict the future performance of the system; to confirm the performance bottleneck and optimize and optimize the performance bottlenecks by simulating hundreds of users, repeated execution and operation tests Adjust the application to find bottleneck problems.
When a company is organized or entrusting a software company to develop a set of application systems, users often have doubts, can users often have a large number of concurrent users to access the same time? Class issues are most common in online transaction processing (OLTP) mode database applications, web browsing, and video ongoing systems. This problem resolves to use the scientific software testing tools and advanced test tools.
◆ Example: Telecommunications billing software
As we all know, around 20 days is the peak period of the public payment, thousands of toll outlets in the city start at the same time. The charging process is generally divided into two steps. First, according to the phone number proposed by the user, it will produce its current expenses, then collect cash and modify this user to the paid state. A user looks simple steps, but as a hundred thousand terminals, when doing this operation, the situation is big, so many transactions occur simultaneously, the application itself, operating system, center database server , The inner server and network equipment's endurance is a severe test. The decision makers cannot consider the system's suffering after problems, and foresee the software's concurrency, which is the problem that should be solved in the software test phase.
At present, most company companies need to support hundreds of users, various applications, and complex products assembled from components provided by different suppliers, unpredictable user loads and more and more complex applications, making companies concerned Poor delivery performance, the user suffers from slow response, system failure, etc. The result is the loss of the company's income.
How to simulate the actual situation? Find several computers and the same number of operators to operate at the same time, then take the stoppage to record the reaction time? Such a hand workshop test method is unreal, and cannot capture the internal change of the program, which requires the auxiliary of the pressure test tool.
The basic strategy of testing is the automatic load test, and the application is tested by simultaneously performing business simultaneously on a virtual user on one or several PCs, and the application is tested, and each transaction processing is recorded. Piece server peak data, database status, etc. Through repeatable, real tests can thorough metric the scalability and performance of the application, determine the problem and optimize system performance. In advance, the system's endurance is known in advance, providing a powerful basis for the configuration of the end user to plan the entire operating environment. ◆ Preparation before concurrent performance test
Test Environment: The configuration test environment is an important phase of the test implementation. The appropriateness of the test environment will seriously affect the authenticity and correctness of the test results. The test environment includes a hardware environment and software environment, hardware environment refers to the necessary servers, clients, network connecting devices, and printer / scanners such as auxiliary hardware devices; software environment refers to operating systems, databases And other applications consisting of applications.
A fully prepared test environment has three advantages: a stable, repeatable test environment that guarantees the correct test results; guarantee the technical needs of the test implementation; guarantees the correct, repeatable, and easy to understand.
Test Tools: Concurrent Performance Test is a black box test executed at the client, generally not manually, but is carried out using tools. At present, there are many mature concurrent performance test tools, and the choice is mainly the test demand and performance price ratio. The famous concurrent performance test tool has Qaload, Loadrunner, Benchmark Factory and WebStress. These test tools are automated load test tools. Through repetitive, real tests, the scalability and performance of the application can be thoroughly measured, can automatically perform test tasks throughout the development lifecycle, automatically perform test tasks throughout the development life cycle, can simulate There are hundreds of users to perform critical businesses to complete the test of the application.
Test data: In the initial test environment, you need to enter some appropriate test data, the purpose is to identify the data status and verify the test case used to test, and debug the test case before the official test begins, and the wrong test start. drop to lowest. When testing to a critical process link, it is necessary to make a backup of data state. Manufacturing initial data means that the appropriate data is stored, and it is required to recover it, and the initial data provides a baseline to assess the results of the test execution.
When testing is officially executed, you also need to prepare business test data, such as testing concurrent query services, then requires considerable amount of data in the corresponding database and table, and the type of data should be able to overwrite all services.
Simulated real environment tests, some software, especially commercial software facing the public, often need to invest in real-world performance during testing. If the scan speed of the test anti-virus software is tested, the proportion of different types of files arranged on the hard disk should be as close to the real environment, so that the data tested has practical significance.
◆ Class and indicator of concurrent performance test
The species of concurrent performance test depends on the object monitoring of the concurrent performance test tool, as an example of Qaload automation load test tool. Software provides DB2, DCOM, ODBC, Oracle, NetLoad, Corba, Qarun, SAP, SQLSERVER, SYBASE, TELNET, TUXEDO, UNIFACE, WINSOCK, WWW, JAVA Script, etc. to support Windows and UNIX. test environment.
The most critical still is a flexible application of monitoring objects during the test. For example, the current three-layer structure is widely used, and the concurrency performance test of middleware is mentioned as a problem, and many systems have adopted domestic middleware. Select the Java Script monitoring object, manually write scripts, can achieve test purposes.
The concurrent performance test of the automated load test tool is used. Basic follow-up testing processes are: test demand and test content, test case development, test environment preparation, test script recording, writing and debugging, scripting assignment, playback configuration and loading strategy, test Perform tracking, results analysis and positioning problems, test reports and test assessments. Concurrent performance test monitoring is different, the main indicators of the test are also different, and the main test indicators include transaction processing performance indicators and UNIX resource monitoring. Among them, transaction processing performance indicators include trading results, transaction time per minute, transaction response time (min: minimum server response time; mean: average server response time; max: maximum server response time; stddev: transaction server response) The bigger the difference, the larger the deviation; Median: Medium-value response time; 90%: 90% server response time), virtual concurrent user number.
Application example: "Xinhua News Agency Multimedia Database V1.0" Performance Test
China Software Review Center (CSTC) According to the "Multimedia Database (Phase I) Performance Test Demand" and GB / T Database (Phase I) Performance Test Demand "and GB / T 17544" Software Package Quality Requirements and Test ", using industrial standard-level load test tools The Xinhua News Agency uses the "Xinhua News Agency Multimedia Database V1.0" for performance testing.
The purpose of performance test is to simulate multi-users and have access to Xinhua News Agency multimedia database, perform key retrieval services, and analyze system performance.
The focus of performance test is the main retrieval service for system concurrent pressure loads, perform concurrent testing and fatigue testing, and the system uses B / S operating mode. The concurrent test is designed in a specific period of time in the Chinese library, the English library, and the photo library, respectively, and multi-retrieval words, and variability, hybrid retrieval business, etc. concurrent test cases. The fatigue test case is a single search term for the test cycle for about 8 hours in the Chinese library. While concurrency and fatigue testing, the monitoring test indicators include transaction processing performance and UNIX (Linux), Oracle, Apache resources, and the like.
Test Conclusion: In Xinhua News Agency's Office Test Environment and Internal Network Test Environment, in the case of 100M bandwidth, the system is able to withstand the load pressure of 200, the system can withstand the number of load pressure, the maximum number of transactions / minutes reaches 78.73, run Basically stable, but as load pressure increases, system performance is attenuated.
The system can withstand the fatigue pressure of 200 and the number of users lasts for about 8 hours, which is substantially stable.
By monitoring the system UNIX (Linux), Oracle and Apache resources, system resources can meet the above concurrency and fatigue performance requirements, and the system hardware resources have a larger utilization room.
When the number of users exceeds 200, the HTTP 500, Connect, and timeout errors are monitored, and the Web server has an error in the web server, and the system should further improve performance to support greater concurrent users.
It is recommended to further optimize the software system to make full use of hardware resources to shorten the transaction response time.
◆ Fatigue strength and large data quantity test
Fatigue tests are the maximum number of concurrent users that can support in terms of system stability operation, continuously perform a period of time, and determine the process of processing maximum workload strength performance by integrated transaction implementation indicators and resource monitoring metrics.
Fatigue intensity testing can be tested in a tool automation, or manually writing program tests, which is larger than the proportion of the latter.
Under normal circumstances, the server can properly stabilize the number of fatigue tests for a certain amount of time for a certain amount of time, and obtain the transaction execution indicator data and system resource monitoring data. If an error occurs, the test cannot be executed successfully, then adjust the test indicator in time, such as reducing the number of users, shorten the test cycle. Another situation of fatigue testing is a fatigue test for a certain amount of time based on the performance of the current system performance, and uses the number of users in the normal business of the system. Big data volume test can be divided into two types: independent data quantity test of large data volume for some system storage, transmission, statistics, query and other services; combined with pressure performance test, load performance test, fatigue performance test Data quota test scheme. The key to the large data volume test is the preparation of the test data, and can rely on the tool to prepare test data.
Speed test is currently a hand-measurement speed for the business that is highly required, and can be compared to the average of the averages, and the response time, which can be measured.
Applied on network performance test
The test focused on performance on the network is to use mature advanced automation technology to make network application performance monitoring, network application performance analysis and network prediction.
◆ Network application performance analysis
The purpose of network application performance analysis is to accurately show how the network bandwidth, delay, load and TCP ports affect how the user's response time. Using network application performance analysis tools, such as Application Expert, you can find bottlenecks for applications, we can know the application behavior that occurs at each stage during the network to apply thread-level analysis applications. You can solve a variety of questions: Does the client have an unnecessary request for the database server? When the server accepted a query from the client, the application server spent an unacceptable time to contact the Database Server? The response time predicting the application before putting equipment; uses Application Expert to adjust the performance of the application in WAN, Application Expert enables you to quickly and easily simulate application performance, according to the response time of end users in different network configuration environments, users can use themselves Conditions determines the network environment of the application.
◆ Network application performance monitoring
After the system trial operation, you need to know what happened on the network in time; what is running, how to run; how much PC is accessing the LAN or WAN; which applications lead to system bottlenecks or resource competition, then network application performance monitoring And network resource management is critical to the normal stability operation of the system. Using the network application performance monitoring tool, you can achieve the effect of halving, in this respect, the tool we can provide is Network Vantage. It is popular that it is mainly used to analyze the performance of critical applications, the root of positioning issues is in the client, server, and applications or networks. In most cases, users are concerned about what applications occupy a large amount of bandwidth, which users have the largest network traffic, which can also meet the requirements.
◆ Network prediction
Taking into account the expansion of the future development, predict changes in network traffic, the changes in network structure are very important to user systems. Provision of network performance prediction data is performed in a timely manner based on planning data. We use the network prediction analysis capacity planning tool Predictor to: set the service level, complete daily network capacity planning, offline test network, network failure and capacity limit analysis, complete daily fault diagnosis, predict network equipment migration and network equipment upgrade to the entire network Impact.
The network topology is obtained from the network management software, and traffic information is acquired from the existing traffic monitoring software (if such software can be manually generated), the basic structure of the existing network can be obtained. Based on the basic structure, reports and charts can be generated according to changes in the network structure, and the network traffic changes, how these changes affect network performance. Predictor provides the following information: Based on the forecast results to help users in time to upgrade the network, avoid system performance decline due to critical devices; which network device needs to be upgraded, which can reduce network latency, avoid network bottleneck; according to forecasting The necessary network upgrades. Apply performance test on server
For tests that apply on the server, tool monitoring can also be used, or you can use the monitor command of the system itself, such as TUXEDO, you can use the TOP command to monitor resource usage. The purpose of implementing test is to implement server devices, server operating systems, database systems, comprehensively monitoring on server performance, and test principles are shown below.
(Temporary)
Figure: Performance Test Schematic Application on Server
UNIX resource monitoring metrics and description
Monitoring indicator description
Under normal state of the average load system, the last 60 second synchronization process
Average number
The number of conflicts monitored in Ethernet
Spending number between processes / thread swirls and threads
CPU utilization CPU usage (%)
Disk switching rate disk exchange rate
Receive package error rate receives the number of errors per second when the Ethernet data package
Pack input rate Ethernet packet number per second
Interrupt rate CPU number of interrupts per second
Output package error rate Send Ethernet data packet number per second error
The number of Ethernet packets outputted per second
Read the number of read memory pages per second in the memory page rate Physical memory
Write the memory page number from the physical memory in the physical memory in the physical memory
Terms or the number of memory pages deleted from physical memory
Memory page exchange rate writes memory page per second and reads from physical memory
Process Enter the number of processes entered by the exchange rate exchange area
Process Exchange Rate Exchange Rate Output Number
System CPU utilization system CPU usage (%)
User CPU Utilization User mode CPU usage (%)
Disk blocking disk blocks the number of bytes per second
