Having worked on projects that are introducing test automation, projects where test automation is almost the only means to test and in projects trying to develop test automation, here's my two cents on the subject. All the projects have had the common definition of test automation, which is what I call "Press the button and go home" test automation. No human interaction required. This means that the tool should setup the test environment, perform the action part of the scripted test and then perform the result observation and validation of the scripted test. I say scripted test since no other kind of test fits this test automation practice. James Bach discussed this topic during his course and without trying to write to much of what he thinks regarding this his definition is more in the line of "anything that does something automatic with regards to testing" and by a rule of thumb he would not bother if the tool required took more than 48 hours to produce. The key here is not the 48 hour rule but the wider term which can lead to a more effective use of test automation. Before I continue with that, I just want to point out the reason for why you would want to do any kind of automation, and the main reason is that you want to speed up the test process. This is why the wider term is so effective because that includes all efforts made to by automatic means speeds up the process, i.e. a tool to calculate all different combinations of given set of variables can speed up the test design time, or a tool to compare log files speeds up the validation, all tools related to create load automatically on the system are also test automation tools.
Having said that, and you still feel like to focus on "Press the button and go home" test automation, at least consider the following. A scripted test generally consists of some test preparation, some action steps and some expected result parts. If you separate the different activities in your automation effort you will both be more resilient to change, and can help to speed up manual testing as well. The proposal is to create at least three different tools and make sure they work together as well as independently. The three tools are the following:
- Test environment control
- Test data generation
- Test data analysis
Test environment control is a tool that make sure that the test environment is in the required state for the test case. I.e. install and configure the correct software, bring the system to a known and desired state, configure and start other tools that are needed for the specific test etc.
The test data generation is a tool that generate actions in the system, i.e. simulates a user, simulate different failures. The generation tool should not validate the system but record all the actions it has performed in the log.
The data analysis tool is the tool which validates that the performed actions produced the desired results.
What I like about this is also that it can support any sapient test activities and you can work on realizing the parts which you need the most first.
No comments:
Post a Comment