Some techniques and guides provided by testers
Level: primary
Chris Wilkinson
Software engineer, IBM
August 2003
The web service is in the core of distributed computing, and the interaction between them is usually difficult to test. Distributed development, large developer teams and expectations of increasing components of code may make the development of web services more easily hidden errors. These types of errors are extremely difficult to detect. The pressure test is an effective method for detecting such code errors, but only can play a role in situations where the pressure system is designed. This article will let you go deep into the basic requirements of this pressure system. testing method
Traditional test methods include some form of simple unit testing, usually executed by developers. Designing these tests requires internal knowledge of software, and these tests are almost always a very small, specific part of the product. These types of tests are ideal for simple Web services that are very small to interact with other code components, and even have no interaction.
Functional Verification is also a test process. In this process, the product source code understands the limited designer test to confirm the core functions of the product or service. Design This test is to prove that this core function is in line with a certain specification. For example, my online auction shows the correct bid in the input? Is my insurance broker system find the cheapest offer? If these tests fail, it is usually meant to detect a basic issue of the product (this problem is usually directly fixed). This test is also a simple web service that allows you to check whether the service can perform its respective features correctly.
System Test is usually done in the functional verification phase and verifies the core function. It tends to look for the entire system as a whole - find out how Web services operate as part of the system, and how Web services interacts with each other. Since the system test is carried out at the end of the development life cycle, it is usually not assigned enough time to complete. The system test phase is often ignored because of the post-transfer schedule and the development of the important phases of the development, and some generally can be discovered, and fewer errors cannot be detected. Even if this error is found, the reason why the wrong error is also changed, and it is trying to fix them. Therefore, when looking for code errors, you must design the system test application to be as efficient as possible. System testing is usually composed of three parts, they are:
Performance: This involves determining the process of relevant product statistics. For example: How many messages are there each second? How many users can I receive at the same time?
SCENARIO: This is the process of recreating the exact configuration required by the customer. Therefore, any problems found in the case can be detected before the customer uses the product.
Pressure (or workload balance): It is different from the other two parts because it is designed to make the software overload operation by applying a large workload. If the pressure test is efficiently performed by maintaining high intensity using the product (but does not exceed the performance statistics determination), it can often discover many hidden errors, and these errors are used in any other technologies mentioned above. It is not found (these errors are often the most difficult to repair). From the detection code error, it can prove that the highest efficiency in these three system test components is the pressure test portion. However, since this process is often confused with other elements or functional tests of the system, the method involved in this process has not been properly processed or implemented.
Error under pressure
With pressure testing, you have a mistake to find a lot of errors that use other test methods. There are two types of error:
Memory Leak: A phenomenon of extremely difficult detection. Memory leaks often happen in the released products, the reason is simple, and it is difficult to design test use to detect them. With a simple function test, it is almost impossible for memory leaks because there is no need to use the product before the product is completed. Memory leakage usually requires operational to repeat very many times to make memory consumption to achieve the extent to which you can pay attention. Although the Java program is more difficult to introduce a memory leak error, as long as the program remains a reference to the object, the object is still possible to be instantiated and the memory it occupies will never released. Concurrency and synchronization: The pressure test is very outstanding on the problem of finding concurrency, because in any test life cycle, it has applied many different code paths and timing conditions. The general rule is that the longer the pressure test runs, the more code path combination and timing conditions involved and applied. Of course, this also makes it difficult for these problems (errors can occur after 5 minutes or 5 days). Dead lock, thread leakage, and any general synchronization problems typically only be detected during the pressure test phase. These types of problems are difficult to discover by executing unit testing. Developers will not always consider his or her code will interact with elimination of elimination of other places (these code may not be written in the execution unit test). Existing pressure test tool
There are many useful tools that claim to pressure testing products are currently under development. It is widely used in those tools for web services. However, there are many simple HTML / SOAP builders that simulate many client connections and thus generate high loads for the web server (this is useful for finding the Web server, but for finding web services Not so useful). These tools are more useful for basic pressure tests, but they often extends only the functional verification phase to repeatedly perform the same functional tasks. If enough time and resource is available, you can achieve more efficient testing by creating a custom build pressure test system. Since designers of the pressure system typically have more understanding of the products and web services to be tested, they will be able to ensure that the pressure system can be used for which specific code areas.
Design pressure application
Designing a pressure test system attempting to test the Web service, let them run the code in a certain way. These styles beyond the functional verification, the purpose is to figure out that the Web service that is tested can not only do what we think it can do, but it continues to run normally in the case of being applied to some high-strength stress. Pressure tests must apply four basic conditions for web services. Many established pressure systems apply these conditions. Effective pressure test systems will apply these key conditions:
Repetition: Perhaps the most obvious and most easily understood pressure condition is the repetition of the test. In other words, the repetition of the test is to perform an operation or function over and over again, such as repeating a web service. Functional verification tests can be used to make it clear that the operation can be performed normally. The pressure test will determine if an operation is performed normally and can continue to be normal at each execution. This is critical for the introduced whether a product is applicable to a certain production. Customers typically reuse products, so pressure testing should discover code errors before the customer. Many simplest pressure systems only achieve this condition, but simply extend function verification tests multiple repetitions and cannot form a valid pressure test. When used in combination with some of the following principles, you can find many hidden code errors.
Concurrency: concurrency is a behavior of multiple operations simultaneously. In other words, there is a plurality of tests at the same time, such as a number of web services simultaneously calling simultaneously on the same server. This principle does not necessarily apply to all products (such as stateless services), but most software has a certain and multi-threaded behavioral elements, which can only be measured by executing multiple code examples. Functional testing or unit testing is hardly combined with any concurrent design. The pressure system must transcend functional tests and traverse multiple code paths. As for how to do this depends on the specific product. For example, a Web service pressure test requires analog multiple clients. Web services (or any multi-thread code) typically access some shared data between multiple thread instances. The complexity increased due to additional aspects often means that the code will have many errors caused by concurrency. Since the introduction concurrency means that the code in a thread may be interrupted by the code in other threads, the error is only discovered when an instruction set is performed in a specific order (e.g., at a specific timing condition). Combine this principle with the repetition principle, you can apply many code paths and timing conditions. Magnitude: The pressure system should be applied to another condition of the product takes into account the load in each operation. The pressure test can be repeatedly executed, but the operation itself should be added to the product to increase the product. For example, a web service allows clients to enter a message, you can use a client that simulates the ultra-long message to make this separate operation for high intensity. In other words, you add the level of this operation. This amount is always applying, but can determine it by finding the value of the product can be measured and modified by the user - for example, the size of the data, the length of the delay, the transfer of funds, the input speed, and the input change, etc. Wait. A separate high-strength operation itself may not find a code error (or only the functional defects), but when combined with other pressure principles, you will increase the opportunity to find problems.
Random changes: the last point, any pressure system has fewer randomness. If you randomly use the countless variations introduced in the previous pressure principles, you can apply many different code paths at each test run. Below is a few examples of how to change the test in the test lifecycle. When using repetition, you can change the time interval between repetitions, the number of times, or change the repeated web service, or you can change the sequence of repeated web services. Using concurrency, you can change the Web service that performs together, the number of Web services running at the same time, or can also change the decision to run many different services or run many of the same instance. Summary may be the easiest to change - each time you repeat the test, you can change the variables in the application (for example, a message or digital input value of various sizes). If the test is completely random, because it is difficult to consistently react down the mistakes, some systems use random variations based on a fixed random seed. In this way, the opportunity to reproduce the wrong mistake will be larger with the same seed. A stress test usually combines all of the above principles and operates as long as possible within the allowable range. The longer the test is allowed, the more code paths can be traversed, and the more errors are found. Of course, once you find an error, you must diagnose and fix it. Since a code error can be displayed after a pressure test run, the system must guarantee that all available debug information is generated when an error occurs - otherwise the same time must be expected to reproduce this error.
Conclude
Test is a partial part of the software development process, and an important part of the part often misinterpreted or ignored is a pressure test. Follow the principles of the detailed description above, you can design and implement an effective pressure test system to find some coverage problems associated with your code. Whether using pre-written tools or creates a fully dedicated pressure system, the pressure test is used to find the nature of the Web Service (or any other program) problem, and can eventually improve your software product quality. Reference
Read Alan Booth and Andrew Citron published on developerWorks, "Stress Testing Your Software Without Stressing Your Tester". (February 2001)
You can find some resource links for software testers on Software Testing Institute.
Software Testing and Quality Engineering Magazine has many useful articles.
You can also find many useful links in the software test and resource center.
About author
Chris Wilkinson is a software engineer in the Hersley Laboratory of IBM in the UK. He has previously been engaged in the development and use of stress systems for more than two years, which is designed to find code errors in various IBM products, including IBM WebSphere Application Server and IBM WebSphere MQ Integrator Series. He is currently the development of internal components as developers mainly engaged in IBM WebSphere Application Server. You can contact Chris via chris.wilkinson@uk.ibm.com.
