Author: Cem Kaner
Translation: PiaoCl
Automated black box testing, GUI level recession test tools are very popular today, according to these myths, your programming experience is not very rich, and can also build a strong test kit. These tools (according to the instructions) are easy to get started. Maintenance test kit is not a problem. Therefore, myths continue to stage, replace those annoying software test engineers with one of these test tools, and develop managers save a lot of money and burden, so that the software is running normally.
The seller of testing software, the competent teller who does not understand the test has strongly promoted such myths, and even those test managers, test engineers think. It is better to use test tools.
Some companies have begun to enjoy the benefits of success, but some companies have failed because there is no effective use test tool.
In February, 13 experienced software test engineers in the Lawst gathering, discussing successful models and test kits for maintenanceability failures of the black box test script for the decline test phase. We discuss the focus of the actual experience, which has begun to start a certainly effective solution to an automation problem. Our goal is to summarize a set of effective processes in a short time in a short period of experience. In order to improve efficiency, we will discuss it under the experienced-Brian Lawrence led.
Other participants: Chris Agruss (Autodesk), Tom Arnold (St Labs), James Bach (St Labs), Jim Brooks (Adobe Systems, Inc.), Doug Hoffman (Software Quality Methods), Cem Kane (Kanel.com), Brian Lawrence (Coyote Valley Software Consulting), Tom Lindemuth (Adobe Systems, Inc.), Brian Marick (Testing Foundations), Noel Nyman (Microsoft), Bret Pettichord (Unison), Drew Pritsker (Pritsker Consulting), and Melora Svoboda (Electric Communities). So the participants have only one purpose, expressing personal views, not being bound by the company's view.
Listening to other participants' test experience:
What's the question?
There are many defects in the automated recession test. I only list a small part. James Bach (one of participants) lists many other problems in Test Automation Snake Oil.
Examples of examples:
This is an example of automation based on GUI in the recession test.
Design test case, then run him
If this script failed, you will write a bug report, give the defect state to fixed, then start from the beginning. If this automation script is running successful, it is correct. Run the test script here (or from a script, or from a certain type of capture tool). Caught the screen file output after the test, save the test case and this output file. The next contrast output file and this saved output file, if the script is running, the test is passed.
The first question: Do not cost this. Create, verify, write automation test scripts and hand-to-create execution tests to spend 3 to 10 or even more (until success). Many testers are willing to automate, but for testers, only one script is running, then this efficiency is very low.
Many testers recommend 100% automated test cases. I strongly disagree with this requirement. Many of the black box test cases I have created only once. To automate each test case, I have to spend a lot of time and money. Use these time I can perform a lot of test test cases. Why is it low to cover high cost tests? Second question: This method has additional risks. We know that the cost of correction defects will grow over time. As the product is getting closer to the shipping date, the more work is more, and as a trial version of the user to write a manual and buying or selling raw materials. The more it is discovered that the more people of the major defect will be wasted. If you spend a lot of time to write a test script before the test time, you know that it is very expensive to finalize the defect.
The third question: These tests are not strong enough. These tests that have been automated have passed. But how much your new defect can be found in this way? As far as I heard the assessment of about six percent. This number is a growth trend when you create a test case, which is just a manual test, and there is no relationship with the automation test.
Fourth question: Practice, many test organizations are just simple running scripts. In the previous period, simple design and programs are not possible to run more complex test cases. Later, although these tests are simple, especially compared to experienced testers, this is useless test method.
