Test automation framework
"When developing our test strategy, we must minimize the impact of testing because the application changes or testing changes."
1.1 Thinking "projects"
1.1.1 Testing Automation
Case study: expensive automation failed
1.1.2 Some test strategies guidance
1.1.3 Test Automation is a full-time job, not part-time
1.1.4 Test Design and Test Framework is completely independent
1.1.5 Test Framework needs to be independent of the application
1.1.6 Test framework must be easily extended, maintained, and effective for a long time
1.1.7 Test strategy / design vocabulary should be independent of the frame
1.1.8 Test strategy / design requires the removal of most testers from complex test frameworks
1.2 Data Drive Test Framework
1.2.1 Data Drive Script
1.2.2 Keyword or Table Drive Test Automation
1.2.3 Mixed Test Automation (or "All")
1.2.4 Business Keyword Drive Framework
1.3 Keyword Drive Automation Framework Model
The following automation framework model is completed after 18 months of planning, design, encoding, and constant debugging. But that is not to say that it takes 18 months to complete - in fact it has a working prototype spending about 3 months, that is, a person spent three months on this.
The focus of this model is to implement the "Keyword Drive Automation Framework", and there is no function including tracking requirements, tracking automated test results return value, and no other features needed during testing, it is only provided An engine that performs testing through keyword drive automation.
The market can be used in the market. Frames are usually more features and broader applications - of course, for this, you can also reflect from their prices.
1.3.1 Project Guidance
This informal project follows the following guidelines and practices:
1.3.2 Code and Document Standard
1.3.3 Our automation framework
1.3.4 Application Map
1.3.5 Component Functions
1.3.6 Test Tables
1.3.7 Core Data Drive Engine
1.3.8 support library
1.4 Automation Framework Workflow
We have seen the main features of our automation framework and we should now add it to our test. This section provides a test workflow model that can work very efficiently for this framework. Specifically, let us first define a high-level set (Cycletables) and provide more and more details to our Application Map and low-level Step Tables.
For example, we can fictize an imaginary test design about a security line identification of a Web site. We introduce this information and create the order of the table is some of our ideal workflows for our automation framework. It will also illustrate why we don't even need to prepare some applications that can actually run in order to show these tests.
1.4.1 High-rise Test - Cycle Tables
We will begin to define our high-level set (CYCLE TABLES), see Table 16. The tests required for checking the User Verification feature are listed here. There is still no app, but we will guide us to verify some users by entering some user IDs and passwords on a login interface. With this information, we can start designing some tests.
At this level, our table is just a keyword or the action we plan to do. These keywords say that when we are ready or understand more about the name of the SUITES.
We will try high-level table VerifyAuthenticationFunction test, see Table 16. It will not display all the tests that we need to perform on the Web site "User Verification" feature, but only to illustrate our test design process.
CYCLE TABLE: VerifyAuthenticationFunction
Keywords (Suite Tables)
Table Purpose
VerifyInvalidlogin
Tests with invalid userid and / or password
VerifyBLANKLOGIN
Tests with missing userid and / or password
VerifyValidLogin
Tests with Valid Userid and Password
Table 16
The above table illustrates three tests that we plan to run when reviewing the User Verification feature in our application. We can add or delete a test in this list, but this is why we currently believe that this feature can be fully tested. (But this is how we currently believe this functionality can be adequately verified.)
1.4.2 Intermediate Test - Suite Tables
When we are ready to provide more details for our "User Verification" test, we can start enrich our intermediate tablets. We must now expand on the three "groups" (suite) defined by our high-level VerifyAuthenticationFunction.
First, we plan to check the invalid login with invalid user IDs or invalid passwords or invalid user IDs. At this time, the first Suite Tables are called, and VerifyInvalidLogin see Table 17.
Second, we should use an empty user ID and password to make a simple test. At this time, the second Suites Tables are called, see VerifyBLANKLOGIN Table 18.
The last Suite Tables we must implement is to ensure that you can successfully log in to the system when we provide a valid user ID and password. This is named VerifyValidLogin, see Table 19.
Suite Table: VerifyInvalidLogin
Keywords (Step Tables)
UserID
Password
Launchsite
Login
Baduser
Goodpassword
VerifyLoginerror
Login
Gooduser
Badpassword
VerifyLoginerror
Login
Baduser
Badpassword
VerifyLoginerror
ExitLogin
Table 17
Suite Table: VerifyBLANKLOGIN
Keywords (Step Tables)
UserID
Password
Launchsite
Login
""
""
VerifyLoginerror
ExitLogin
Table 18
Suite Table: VerifyValidLogin
Keywords (Step Tables)
UserID
Password
Launchsite
Login
Gooduser
Goodpassword
Verifyloginsuccess
SHUTDOWNSITE
Table 19
Note that we have begun to find a large number of test tables that can be reused here. Before we have not completed the definition, we will find that we will use Launch, Login, VerifyLoginerror, and ExitLogin very frequently. Also you can imagine, VerifyLoginsuccess and Shutdownsite will also be widely used in all tests on this Web site, but we only pay attention to "User Verification".
You should also notice that our login keyword describes a Step Tables that will receive two parameters - the user ID and password we wish to provide. We still can't clear how to design the details of the automation framework to be implemented, but the test can be reused without receiving different data. These tables can be used when using different user IDs and passwords to log in to the system. 1.4.3 Application Maps
So far, we have not used Application Map yet. Almost all high-level and medium-end tests are completed before reference to a specific window assembly. But when we approach our bottom test design, we really need reference to specific application elements. Therefore, for test automation, we need to build our Application Map.
Component named:
This thing is to test the team yourself, unless the application has a complete design document that provides meaningful names for forms and components.
The entire team must discuss and consistently naming each form and component. Sometimes we can use developers to assign the name of the component in the code. But the team will often discuss for meaningful and clear names that can be explicitly labeled. But in any case, it must be defined, and the selected name cannot be changed. Otherwise, if this needs appears, a project will always have more than one name.
Test Design Application Map:
For underlying test design and manual testing, an Application Map suitable for people should be established, which is usually captured from various application windows on the screen. When each component is named, these screen captures can be used as a clear reference, and it may also indicate the type of each component (TYPE).
Although most controls can be relatively accurate, it is not all like this. There are also many controls that have no forms or cannot be identified by the application interface, such as a COM object. Therefore, the identification of different objects should also consider the type of object (TYPE) information.
Automation Framework Application Map:
For automated tests, you need to create an Application map for automated frameworks, and we will use the Test Design Application Map to name the object to the object. The Identification Syntax Suitable for the Automation Tool Will Then Be mapped to each name as discussed in section 1.3.4.
Our test design has so far been included in three interfaces: (1) login interface; (2) Login failure is the displayed error message dialog; (3) The home page of the Web site displayed when logging in. We have to consider the browser interface during the test.
We must now establish an Application Map - it contains all objects in the above three items that have their respective identification methods in the automated framework. For the sake of simplicity, our login interface includes two Edit components of the user ID and password, and the Submit and Cancel two buttons (button), see Figure 7
Figure 7
The error message dialog is actually a system warning prompt dialog, see Figure 8, there is a Label and an OK button containing the error message on the dialog.
Figure 8
The main interface of the application may contain a lot of different elements, but for our purpose and this example, we only need to identify this interface. Because we are not prepared to put all the components on the homepage, we only need to create a simple Application Map. Table 20 shows the content we are used to identify the right automation framework for the browser window and three pages.
Site Application Map
Objectname
Identification Method
Browser:
Browser
Windowid = WebBBROWSER
LoginPage:
LoginPage
Frameid = content; /; documenttitle = login
UserID
Frameid = content; /; documenttitle = login; /; textboxIndex = 1
Password
FrameId = content; /; documenttitle = login; /; textboxIndex = 2
SubmitButton
Frameid = content; /; documenttitle = login; /; buttonText = Submit
CancelButton
FrameId = content; /; documenttitle = login; /; buttontext = cancel
Errorwin:
Errorwin
Windowcaption = Error
ErrorMessage
LabelIndex = 1
Okbutton
ButtonText = OK
Homepage:
Homepage
Frameid = content; /; documenttitle = home
Table 20
The physical format of the Application Map determines how the Automation framework access it, we can use text files, database tables, spreadsheets, etc. to store Application Map.
Note that in the Application Map shown in Table 20, we allocate a section for each interface that needs to be identified. There are many components in our app, we need to avoid naming conflicts between them. For example, there may be a Submit button in many forms, and each form can have a button named SubmitBuuton because each of them is unique in some of the APPLICATION MAP.
We have now established a clear indication for all of our components, where ObjectName is called a flag of a component in the underlying Step-Tables, and Identification Method is specifically defined as information required to identify different components.
The syntax for these component identifications is different depending upon the automation tool deployed. The example syntax we used is not for any particular automation tool, but is representative of what is needed for automation tools in general. The Application Map designer will need a thorough understanding Of The Automation Tool's Component Identification Syntax in Order To Create Effective Application Maps.
New Component Type:
When we check an application and develop an Automation Framework Application Map, we may find a component type that does not exist in our hand in the Component functions in our hand. For this situation, We need to Either Implement a New Component Function Module for Each New Component Type, or Determine How Best The Can Be Handled By Existing Component Functions. For a full-time automation technician. 1.4.4 Bottom Test - Step Tables
When we already know the design situation of the application, and our Application Map has been developed, we can start seriously consider the specific details in our test. We can also develop Step Tables without Application Map, but if there is no Application Map, we cannot automate those tests.
Review all Suite described in the high-level VerifyAuthenticationFunction Cycle Table, we found that there are six calls to be reused, see the part listed in Table 21. We will implement them in Step Tables.
Step Tables Invoked by Suites
Step Table
Table Purpose
Launchsite
Launch The Web App for Testing
Login
Login with a user ID and password
VerifyLoginerror
Verify An Error Dialog on Invalid Login
ExitLogin
EXIT The login page via Cancel
Verifyloginsuccess
Verify Home Page Up After Successful Login
SHUTDOWNSITE
Logoff and close the broment
Table 21
1.5 summary
