This article is a series of articles (as listed below) in a distributed, J2EE-based project (as listed below).
Part 1: Project introduction; high-level plans Part 2: Risk management; requirements management Part 3: Model creation and access control; demand analysis Part 4: Use example refine; production report; tools and technology Section 5 : System architecture and design Part 6: Detailed design; early development; two-way engineering; early unit testing Part 7: Continue development; early construction; demo Part 8: unit test strategy; function test; GUI test script Part 9 : System construction and testing; defect tracking; product delivery Part 10: Project is completed; conclusion; future work
In this article, we are a software company Lookoff Technologies Incorporated, our customers AudioPhile Speaker Design, Inc. (ASDI), which hires us to achieve their initial IT needs. For more detailed information, see Part 1.
So far, we have been close to the first phase of the ASDI project. ASDI has gained a series of system demonstrations we offer, and they are very satisfied with the product. (In fact, we have some concerns because the first phase of the project is already so available system, we are worried that ASDI will postpone or cancel the next second phase of the project.) The ultimate factor of the customer is full of us. System testing and acceptance tests that meet demand.
Part 9 Snapshots in Part 9 Demo:
Rational ClearQuest - Tracking and Managing System Tests in Integration and Test Cycles Rational SiteLoad - Rational Robot for Load Testing Rational Robot on our web application by simulating a certain number of concurrent users to the system access to our web application - recording and tested for load test B2B interface Playback VU script is created or updated product:
ClearQuest Defect Database - Created Use to track the defects SiteLoad and Robot test scripts that can be shared by all team members in an accessible network storage - created for automated testing
Packaging development At this time, our coding work has been significantly reduced. In the stage of modifying and refining the product, our team's work is mainly for some small changes. When integrated with the test (I & T) team discovers software defects, they fill in and prioritize defects in the Rational ClearQuest database based on defective defect tracing database modes. These defects are reviewed by the engineering team. Team leaders and project engineers typically determine the priority of defects and maintain a plan that describes which person will be repaired for a given building version.
The constructed frequency performs a complete system to build a frequency - build a system from scratch in a clean environment - making us greatly close to the end of the first phase project. Originally planning once a month, but these builds are sometimes performed once a week. This is not possible to build a clean build environment for a large project or a team of high-skilled developers, which is not feasible from the source of software, build systems, and test systems from the source code base. However, thanks to our tight intensive, our good document established process and we use Rational test tools to complete our test implementation, we can increase the system's construct to a week.
Automated scripts are particularly feasible to make such frequent buildings. At least we run the script to test the system part of the deficiencies we have resolved. We can usually be further, most of the system or all of the scripts. We are very lucky that we can test the system's module by automated test scripts; Rational's test tools can be very good in line with our technology, however, some other technologies may bring challenges or fundamentals to test scripts The test script cannot be used. The intervention of the integrated test (I & T) team is now, we arrived at the stage of system construction, and the I & T team is fully investment and leads this process. In the development cycle of the project (in the sixth part), we introduced the I & T team into the project team and worked in full-time, so they were ready. In our previous project, we have later introduced the I & T team into the project, almost close to the end of the development cycle, but this will have a certain issue. We will finally realize that the I & T team needs a certain preparation time, just like a technical member of other project teams. Although the I & T team in assembly system is slower than the system team earlier, this is part of our expected learning process, and the engineering team can continue them. Development work.
The I & T team is expected to build the goal of the project based on their understanding of the progress of the engineering team. They discuss constructs with project engineers to ensure that the construction can meet their expectations. For example, if it is not ready for some components or subsystems, the engineering team does not prepare for functional testing, for assembly, check each compiled code, matched interface and third-party tools (JDK, library and purchase The tool) is consistent in the thread and subsystem team members, and constructs just simple exercises. For a very mature construction, the team will load test or functional tests based on system-specific aspects.
The end of the development cycle, I & T's leadership is a very important role in a project team - even more important than team leadership or project managers. I & T Leaders To test the designated plan, understand the weaknesses and strengths of the system area, and constantly monitor defect analysis data. At the same time, he also manages the complete tracking mapping of demand, components, threads, test scripts, and defects, which helps him plan and prioritize his team's movements.
Repairing defect repair defects are not tight tasks. Every defect will trigger the following questions:
Is this really a defect? What type of defect is it? It can belong to any of these categories: decorative, bad, data loss, document, operation, installation, loss characteristics, performance, stability, unsoules, or unfrigated behavior. Do we need to fix it now or postpone him? Is the relevant demand not correct? How can we fix this defect? What kind of speed should we fix it? What other parts of the software will be affected? Who should fix it?
Whenever I & T team find a defect in the system, they will submit it in the ClearQuest database, and they can get a lot of details mentioned above in the database. They connect the database in ClearQuest and fill in the defects in the submission form, as shown in Figure 1.
Figure 1: Submit a defect (click to enlarge)
The defect database can be shared by members of all engineering teams and can be accessed over time through the network. For example, the owner of the defects submitted in Figure 1, his work is in the search capabilities of the product, and he is divided into the same group with a member of the search team responsible for user information. In order to maintain a tracking of any test problem that is allocated, the I & T team uses a ClearQuest query. Figure 2 shows the results of the search team, and the defects submitted in Figure 1 are displayed in the result set. Query (the result is always reflected by the latest content of the database updated by the I & T team), which is filtered, including the defect of the search team, and sorted by priority and submission time. Figure 2: Filtered defect query result (click to enlarge)
Other teams query ClearQuest in their respective regions in a similar manner and receive appropriate filtered results. It is only the I & T team, project processor and team leaders can see the full defect list in project progress.
The ActionS button in Figure 2 provides modifications, allocation, closing, copying, postponed or deleting options that are currently selected by the query results. According to the position of the project team member, different people can perform different operations:
Engineering team members can only modify or distribute defects. For example, the leaders of the search team can assign a defect to the designated member of his team. The I & T team can perform all the actions: submit defects, modifications, allocation, closes, copy, postpone, or delete defects in the database. Project engineers also have permission to implement all the actions, although the shutdown action is always performed by the I & T team.
System Test System Tests Test the entire system by using recorded test scripts. This test is not only important to the customer's acceptance test, but also its deep inspection of the system and provides insights in the test script. No matter how tightly track changes, there will often be some small changes that exceed our expected scope to bring some conflicts to unproductive ways affect other code snippets.
We usually build system buildings in a clean environment established by the I & T team, so we thoroughly test the build documentation. If any error is encountered in the build document, a problem (in the "Document" classification) is submitted to the defect database with any identified test defects.
From the unit test phase (discussed in Section 8) to the present, we tested the system's functional requirements and the non-functional demand of the system. Now we have invested more energy than in the early test phase. The main non-functional fields we test are availability and performance (load test), and we will discuss below.
Availability suggests that we have used us only usability experts. Although she is included in the planning and simulation of the early user interface, she helped the manual interface, but she did not join us and other members of the I & T team. Her current work is to work together as a user, find out the availability, and submit issues into the defect database (usually in the "unfriendly behavior" category).
Some of the availability issues must be delayed because they exceed the scope of the first phase or to handle them very cost-effective. However, many easy repairs will have been discovered to bring confused small availability issues. This is some of our most cost-effective testing activities, because usually from the customer's point of view, just some small code changes can greatly improve products. Recommendations for availability include new error messages, improvements in layout, adjustment buttons, and menu, documentation, and modifications of screen workflows.
Load Test Our system has no harsh performance requirements, but we want the system to be available under maximum load. We have made some load tests early, but this test type will reach the highest point when the end of the development cycle. We want to ensure that the system can surpass Asdi expectations. We hope that we can get the next job in the form of the second phase of the project, and we don't want to make performance problems. We carry out load tests for both parts of the system: web application interface and B2B interfaces based on SSL / XML-based Command Gateway (described in section 5). Web load test
For the Web load test, we used Rational SiteLoad. This tool enables us to record scripts consisting of a series of Web transactions, and then copy these steps as multiple virtual users. We and ASDI reviewed the expected load mode to determine how many users accessed the web application, and we decided to test 20 users' loads.
By using SiteLoad, we can easily simulate the concurrent users of 20 systems, and accurately statistical related system performance. When we launched SiteLoad, it launched our browser and prompt us to create a test or run an existing test (see Figure 3).
Figure 3: SiteLoad's home screen (click to enlarge)
When we choose to record a new script, SiteLoad pops up a Java-based browser and records all the actions we do on it. For example, when we browse our partsearch.jsp page in this browser, SiteLoad is loaded from the server (as shown in Figure 4) and records our actions, including any of our data values and buttons. action. We designed this specific test to perform many database query operations based on multiple parameters. This is obviously a very simple test because the database can cache queries; other tests will be more stringent and challenging.
Figure 4: SiteLoad records the action of the browser (click to enlarge)
We can also set some usual performance requirements for our test. We decided that when the script of the PARTSearch.jsp page was played back, we would like to load at least 90% of the page (see Figure 5) in four seconds or less. Although this does not meet high performance requirements, this is enough for our system availability and overall quality.
Figure 5: Set the performance requirements of SiteLoad (click to enlarge)
Figure 6 shows how we configure SiteLoad to simulate 20 concurrent users to repeatedly perform the actions of the PartSearch.jsp page we recorded. Siteload can make more complex performance modeling to help us find our system's "performance wall", but we choose to directly carry 20 concurrent users in our first attempt to test the maximum test. If we encounter problems, we will reduce the number of initial users to 5, and add one user every minute or two minutes. We can also set a standard for termination tests, but we don't do this, because we are visualized by the process of monitoring tests to understand what kind of behavior in our system.
Figure 6: Set SiteLoad User Characteristics (Click to enlarge)
When the test is running, SiteLoad displays and constantly updates the statistics displayed in Figure 7. In this example, a histogram indicates the performance results of our test script; almost all pages are loaded in 8 seconds (in other words, only 0-20% performance limitations in our 4 seconds) in). Use options available in the green menu bar near the screen, we can choose to view more detailed test reports, such as page access, CPU load, and average load time.
Figure 7: Siteload test results (click to enlarge) B2B load test
For the SSL / XML B2B load test, we use Rational Robot to record the virtual user (VU) script. We entered the command we want Robot to monitor and the most ended by a script, this script is very different from the GUI script generated by Robot (discussed in Section 8) very different. Unlike the GUI script, VU script records and receive low-level information related to transmission information. By running VU scripts from multiple machines, we can simulate the B2B client session of concurrent operating systems. ASDI suspects that there will be excess two concurrent sessions, so we can complete some steps that transcend this demand to ensure good system performance.
The test completion is checked in the last two weeks in our plan, the engineering team has conducted system testing and passed all the needs, and the I & T team closed all the questions. In addition to some of us have discussed with customers, it is considered to be postponed. Unestable problem. For us, the next milestone is a test completion check (TRR), we have implemented two TRR: one is internal, one is with customers. The inside TRR will be implemented:
Are all documents to complete? Are all code to be checked (Check IN) and tested? Do you have any code review and unit test lists for future verification? Is there any test problem that is not postponed? Are all changes requests to be closed or merged into demand? Does all models of RATIONAL ROSE are documentated and are delivered? Are all aspects of the system demonstrated it?
In addition to checking the above project, we review and pre-exploit a system presentation, the entire presentation as a final display of the product serves TRR to the customer. We are proud of our products, and we want to make sure all key features of the system are displayed because we know that the seniors of ASDI will attend this external TRR.
During external TRR, we are based on the same schedule as the internal TRR. We reviewed the list of checks with ASDI to show that every thing was completed and ended this demonstration. It is not surprising that more ideas appear in the final demo, in the future considering we commented down.
Acceptance Test For the first phase of the ASDI agreed project has been successfully completed, the system must test through some final acceptance tests. We have a reason to believe that the system can pass these tests, because we have written a set of scripts to check if all the needs are overwritten for the end-to-end test of the system. These scripts are created using Rational Robot and is checked by ASDI. The only thing we can think of to prevent our acceptance test is the change in the last moment of the engineering team, may affect other code snippets. But before we started the acceptance test, we received an amazing news, and the ASDI told us in the external TRR to let us handle the test. We believe that we pointed out that the intent of the test script is very clear in the acceptance plan, but now we realize that our wording we plan is vague.
When we state that CAT (customer acceptance test) in the external TRR will be quite short, and we can perform a very fast execution and check the execution result of the script, asdi means that they want to see the test execution of one step by step so that they know what happens. What is it. Although we don't want this, it seems to be fair and feasible to us. We document all test procedures and plans for our test script, even when the script is upgraded, we also maintain our test plan, so for us, do not have problems with hand-performing test. We are not prepared to handle how long it takes to test. Now let's take a manual test based on our documentation testing process, we realize how often the automated test saves us.
We have found that some details must be provided during our test. We are not always able to get enough information to create a clear and repeated test. We also realized that we updated the test script but did not modify the test plan. After a small change of the test plan, we deliver a test plan for a quick check; we agree that the change is very small, and other TRRs do not need other TRR.
The acceptance test occurred in our development environment, starting to clean up according to the build document, and trigger the implementation of the test process. These tests will probably spend a total of two to one and a half days. Three members in our team have implemented these tasks (I & T leaders, project engineers and team leaders), and three people from ASDI (QA, project manager and technical leaders).
We are proud that there is no software defect in the acceptance test. There are only some unimportant problems, usually in the "document" and "unfriendly behavior" categories. All the needs are all satisfied, and customers are very happy at the end of the test.
Summary This may be our team's ending of the first project that has not been taken from the end of development and testing. The factors that contribute to this is that we have better tools, familiar with technology and an engineering team that works in the early stage of the project.
Especially the test process has a big success. The Rational Tool is the most impressive perhaps the function of the test. This is our first introduction of automation testing, and we are so surprised to work for it. The maximum pain in the automated test is a script modification of the corresponding demand changes; however, this responsibility is passed to the integration and test team, so it will not affect our work of our engineering team.
Planning the future now we have installed the software in the customer's environment and give them some time to assess the system. Although there is no formal guarantee phase, ASDI has been submitted to the ClearQuest database. Finally, a protocol for project second phase must be reached.
Main risks At this time we feel that there is no major risk. We are confident that all big problems have been resolved, and we are fully prepared for any issues that may appear in this project.
About the author Steven Franklin has a very broad background in software design, architecture, and engineering process, which is often used in large, distributed information management and control systems. He started using Rational tools in 1997, his main interest in XML, J2EE, wireless and software engineering technology. You can contact Steven via Steve@sfranklin.net.
