1 Introduction
This article focuses on the main aspects and impacts of the organization of the automated test framework. The intent of this article is to provide some guidelines that can successfully implement automation tests.
2. Test the myth of automation
There are a lot of myths about the automation test. Some of them are real, while others are incorrect ideas, these incorrect ideas will seriously threaten the success of the implementation of automation tests. This article will introduce you to several myths that we face with the test automation:
2. We are pressing in time - the project has been behind - let us use the automation test!
This situation will not be a reality. In fact, the right idea should be - our time is urgent - we should never use automation tests.
If the project has been caught in trouble, automated functional testing is not recommended. The project is likely to be stroked because the preparation and implementation of a large number of test frameworks will be cut. I am hungry, I will focus on the following things:
Optimize the process of testing. Survey and suggest test methods and processes on current work. It is recommended to learn from the relevant ideological and processes of RUP. Introducing or forming the unit / component test. This is a good way to get benefiting quickly. If the official component test is used, I suggest you can use the Rational PurifyPLUS to perform unit or component testing. It is worthwhile to use Rational PurifyPlus as early as my experience. In an introduction and Rational PurifyPlus project, it is usually 30% performance improvement at the level of the component. When the project team can answer the answer to the following questions: YES ": The project can be properly delayed. There is an accurate goal that can be achieved by implementing an automated test. The project has the necessary conditions for establishing the appropriate test framework.
Then, you can implement test automation in a time urgent project. However, it is difficult to occur in this case.
All in all, I can only say "Sorry, the silver bomb does not exist".
2.2. Test automation is capturing and playback
In the past days, the automated test tools are only seen as a tool for capture and playback. The current myth is still in the idea of many testers. And in fact, automation tests are far more than caught and playback is so simple. Tests automated according to maturity can be divided into 5 levels.
2.2.1. Level 1: Capture and playback
This is the lowest level of automated testing, and this is not the most useful way of use of automated testing.
Benefits: Automated test scripts can be generated automatically without any programming knowledge. Disadvantages: You will have a large number of test scripts, while the corresponding test scripts must be re-recorded when demand and application changes. Usage: When the test system does not change - small-scale automation.
2.2.2. Level 2: Capture, editing, and playback
In this level, you use the automated test tool to capture the features you want to test. Put any write-down test data in the test script, such as name, account, etc., completely delete from the code of the test script, and convert them into variables.
Benefits: Test script begins to make more improved and flexible, and can greatly reduce the number of scripts and maintenance. Disadvantages: There is a need for a certain number of isolation. Frequent changes may cause "pasta code", and change and maintenance is almost impossible. Usage: When the regression test is performed, the application is tested has a small change, such as the change of the calculated code, but there is no change in the GUI interface.
You can use this technology to explore your predetermined test design by quickly preparing some test scripts to test your ideas. When I did a product when I didn't have a product such as a document such as a demand or design model, I used this technology when I need to get the first impression of the stability of a series of internal buildings. Usually, if appropriate software configuration management (SCM) is combined with good internal design, the technology of using the level 2 is sufficient. 2.2.3. Level 3: Programming and playback
This level is the first level of effective use of multiple build versions. You need to ensure the security of the project before the actual investment begins to ensure the security of the project. If there is no appropriate training tester for the test automation, it will not have the ability to reach this level. All test functions in the automated test tool must be well understood and to master the knowledge of test scripting languages.
Benefits: You identified the design of the test script. Appropriate design is necessary. The habit of encoding must be appropriate. It is very good to use the same coding habits as developing. This will begin to build a bridge between testing and development. Automated testing can be started in the early days of the project. You can start the design of the test script in the early days of the project. Interpretation of developers and investigations they believe that there may be problems. Make sure developers pay attention to the scheme that can be tested. Disadvantages: Requires testers to have good software skills, including design, development, etc. Usage: Mass test suite is developed, implemented and maintained by professional automation tests.
Level 3 allows you to use automated tests and build different regression tests (reuse existing automated test cases). According to my experience, before seeing more practical rewards, in order to achieve this level, there must be a lot of work and activities that affect the project must be done. Therefore, the value of fast establishment and certification of automated tests is critical. Find a tedious test (for example, edge testing and specific function test cases are a good candidate who first performs automated tests). First create a small amount of test cases that can test some basic functions (such as login and create users, etc.).
2.2.4. Level 4: Data driven test
This is a professional test level for automated testing. You now have to use all the test functions provided by the test tool. You have a powerful test framework, which is based on the test function library that enables you to quickly create a test script based on the test system. The cost of maintenance is relatively low. You will use a lot of real data in your test.
Benefits: You can maintain and use well and effectively simulate test data in real life. Disadvantages: Software development skills are foundations and require access to relevant test data. Usage: Mass test suite is developed, implemented and maintained by professional automation tests.
Level 4 requires some very good test data. A tester must take some time to identify where to collect data and what data collected. Using the data in real life is the most basic to get a complete return from the test. Use the appropriate data will provide you with the ability to discover only in the later period of the project or have a customer discovery. Now you can run a lot of testing by using realistic data.
2.2.5. Level 5: Test Automation using action words
This is the highest level of automated test. The main idea is to separate the test case from the test tool. This level requires a team with a high-skilled test personnel that combines the very deep level of knowledge of test tools with the deeper programming capabilities they have. This team is responsible for generating and maintaining the functionality of testing in test tools, enabling test tools from external sources, such as Excel tables or databases. This test concept was initially developed by CMG. Other possible open source solutions have been developed by Carl Nagle and SAS Institute as compared to the CMG solution. With DDE concepts, the focus is when creating test cases in the Excel table, placing some types of templates that include specific action words that are used. The execution process is to read the test case from the Excel table and convert the test case to the form of the test tool to understand, and then use different test functions to perform the test. This concept becomes more and more, because the test is very useful with the use case.
Benefits: The design of the test case is separated from the test tool - paying attention to the test cases in design. Allow fast test examples and better estimates based on the use case. Disadvantages: A test team with tool skills and development skills is required to provide and maintain test engineering (framework). Usage: Professional test automation combines skills to use optimization
If the tool does not have the possibility of using built-in object mappings, this program is excellent for eliminating most of the maintenance costs related to GUI. This solution has been created in some organizations, and some of them have achieved high degree of automation (60%), and they have got a huge return. If the test framework is appropriate, we can use Excel to generate actual test cases.
This level is very good for organizations or projects in accordance with the formal basis usage. How much test is estimated is required, and the work that needs to be done is also very simple when the use case is appropriate. You can concentrate on generating the first test case (main process) that contains the "object mapping" required. Relying on the complexity of the test application, usually takes about half a day to day. Each test case that is required is probably taken from 15 to 20 minutes, as usual test cases can copy existing test cases and make the necessary modifications, usually this modification is limited. The action word framework can become possible by using the use case to make the close parallel test case.
2.3. No training
All of our people have certain experience in some aspects, we don't have time to spend training using new tools. Training and guidance is critical when an organization or project team is not very familiar with the automation tool. If we allow organizations or project teams to implement automated tests without any knowledge that should be made, it will definitely end. The budget for implementing the automated test scheme will be exceeded, the test will be delayed and worse is that the automated test will be abandoned. Organizations and project teams need to avoid some understanding of defects, especially the maintenance costs of automated tests and frustration generated when the tester is trying to confirm how tool works. You need to make sure your test process is appropriate - if the test process is unreasonable, introducing an automated test will only bring greater chaos to software organizations or project teams. Therefore, I recommend that organizational or project teams that wish to implement automated test programs should establish a "training camp" before implementation, and focus on training test teams can use a prototype project.
Develop a implementation plan for the first prototype project, the following description of the minimalization of prototype projects:
The first state we want to achieve what - the return of the factor expectations (the first automated test work is expected to verify what) find a "simple" test pain, try to solve it through the automation test, this can be used as the same Time to make the test run on multiple platforms Example illustrate the resources and time you need ... At the beginning, you will have a successful confidence - let people know the progress you show. This will attract more attention and resources.
2.4. Must 100% automation?
From a management perspective, 100% of automation goals are just a cost that may be achieved from theory, but actually reaching 100% of automation cost is very expensive.
A 40-60% of the degree of use of automation is already very good. This level will increase the test-related maintenance cost. Due to the complexity of each build version of the demand variation, you will spend more time on the change test case to make them work correctly. In this case, by informed the management of 100% of the automation goals of the management is quite expensive to establish a reasonable expectation value is wise. The general rule for deciding a test case is that this test case must be run more than 4 times. This number is based on the user's good skills for test tools and has a good test framework. If the situation is not such, the entire number can be 10-20 times or higher. An example, tested in a project spending and two-week time converts the 23-day task of manual testing into an automated test. In completion, the project can run the same number of test cases on multiple platforms 4 hours.
2.5. Test Framework
Test Framework is important for the proper basis of the successful test automation. Many considerations must be resolved to make test automation more efficiently. The focus must be:
Maintenance cost maintenance cost is one of the most important issues of successful use of automated testing. Maintenance costs directly contact the maturity of automated tests already mentioned earlier. Organizations or projects must be at least at least 3 to use high test libraries in maturity to make maintenance and update test functions easier. What type of data will be tested? To generate test data for each test case or use data in the test application. In many cases, a test data is created, deleting them is impossible. Can the testability automation test plan effectively test? For example, an object being properly named (not just index ID). A simple example is that all dialogs have the same #ID and the same title, which is simply displayed text information. When the test should cover a variety of languages, the test of the dialog is a challenge. What kind of skills should testers should have in the creation of automated testing? If they have a good development background, then maturity level 3 (cmm3) is sufficient. If they have little or have no experience in developing, we are forced to find or train a group of automated test experts and directly to the maturity level 5, and the creation of the maturity level 5 test is separated from the actual test. . A good building process automation test has added some constraints on the "Building Team". In order to achieve high utilization (regression test) of the automated test, it is required to have a high construction frequency. Just run automation tests per week is not a good automation test usage. Increase the regression test to a new problem that will help quickly discover new problems every day and make developers easier to discover the root causes of the problem, because the feedback time of the test is relatively short (the developer can remember what they did yesterday). The definition of ownership of the different test libraries is important. A good solution will divide the tissue of the test library into three levels: Level 1 - the global level of this usual level. Test feature stored at this level can be accessed by all items. GM and usual features login, create a user is a good candidate at this level. Level 2 - The test function at this level is related to a particular test item, but usually useful in the project is useful in the project. Typical level 2 is the provider of the functionality of the level 1. Level 3 - Script function is directly related to a specific test script. I In this level, usually a first version of a test function is developed. Reuse of test functions during the creation of new test scripts is discovered and moved to level 2. Minimize the number of functions in this level because it will increase the maintenance workload.
There are still many questions about the test framework, but what is mentioned herein is some basic to be resolved.
3. Where to use the automation test
There are a lot of cases that use automation tests to reduce test costs. I will try to highlight different test techniques in automated testing.
Technical Description Note Unit Test / Component Test This test work is usually the duty of developers. Many different methods can be used, such as "test first", it is a test framework, developers write different unit tests before writing code. When the test passes, the code is also completed. By using formal unit testing, not only help developers produce more stable code and can be more good quality. Smoking test smoke test is a collection of functional test cases of non-test systems. The idea behind the smoking test is to ensure that the foundation is working, so that "big" test work can begin. When the construction process ensures that the construction has been prepared for the test, smoke testing is usually automated. Function / Integration Test The work of the test here focuses on validation between integration between different components. These types of tests are usually based on the more complex test of the test system, and a large number of edge tests are combined to create different error handling tests. System Testing - Using Case Test This test is to use the system to prove the system with a desired function by executing the user scene to simulate the real user to demonstrate the system with the desired function. There is no need to perform an automated test here. Installation tests, security tests are usually hand-made because the environment of the system is constant. The regression test regression test is actually a repetitive test. Usually if it is hand-made, this test is only executed one to twice on the end of the project. Here is a test of automation of automation here. You can perform an automated regression test after each build completion to verify that the change of the test system affects the other functions of the system. Performance test performance test includes the following different test forms: - load test - pressure test - concurrent test --... If there is no automated test tool, you will not perform high-intensity performance testing through analog users. 4. Why do I use an automated test?
I am a summary summary for when I should use the automated test and when I should use the manual test:
Using automated tests using manual testing
There is no strict time stress with a good definition of test strategy and test plan you until you try to test what you know.
For automated tests You have a test framework and candidate that can be identified to ensure that multiple test running build policies Multi-platform environments need to be tested. You have a hardware that you have running tests You have a focus on the automation process is tested. Automated test
There is no test process without a test, when is the clear blueprint of the test in a project, you are a newcomer, and it is not a full understanding of the functionality and design you or the entire project at the time of time. There are no resources in the team or have automated test skills.
If you are engaged in automated testing, you must remember that you want to pay attention to the use of automated testing and manual testing. First, the target of the automated test rate is 10/90 (10% automated test and 90% manual test). When these goals are implemented, the usage rate of automated tests can be improved. Remember that the test case for creating an automated test is more time than the test case for the creation of a handmade test. Don't use all your test times on the automated test case. At the same time, it is necessary to remember that every time it is expected to be treated for each of the discovered errors during the test.
5. The benefits of automated testing
If you are introducing an automation test in your organization, remember that there are many different aspects that are included. How many different views have been made in test work today. In order to be able to implement the automated test, you should ask these questions:
What is the test cover? - What are we not covered? What is the cost of "BUG" that we did not discovery due to omissions? What is the cost of destroying the functionality due to bad tests? What does "trivial" test is run every day, what does it mean for your project? If we can provide developers to develop feedback to developers every day, what is the impact of the project?
These issues can be met to be automated. You must start from the level 1 or level 2 of the automated test maturity, and start measurement results. According to my experience, it is very beneficial to feedback to developers and run test every day for the level 4 or level 5 to automated test maturity. Automated tests have the following contributions:
Reduce risk - You know what you tested and not testing what tests can start early in the project and extend the fast feedback over time. The automated test case can run at any time on multiple platforms. Estimated - You can better understand the concentration of excellent personnel on test progress and time to use - you can get an expert team, and spread their knowledge to other project joy - you and your team is getting success
Reference
1. Rational Training Course TST170: Principles of Software Testing for Tester
2. Rational Training Course TST275: Essentials of Test Management With Rational TestManager
3. Rational Training Course Prj480 Mastering The Management of ITERATIVE Development V2003.06.00
4. Reference: Software Test Automation by Fewster & Graham
5. http://safsdev.sourceforge.net/framesdataDriventestautomationFrameworks.htm
