Maintainable Automated Test Design

Design automated test scripts that are easy to maintain

All test automation engineers know that a single application change may break many automated test scripts, and these kinds of application changes happen constantly. To effectively deal with it, test automation engineers need to be in a position to respond to a simple application change in a matter of seconds (here, I mean the time taken to modify the test scripts, excluding execution).

The key to test maintenance (a common problem in test automation) is the test script, not the test tool. Tool vendors like to focus on their tools or deliberately blur the relationship between test scripts and testing tools, for the sake of commercial interests. It is the test engineer’s knowledge that ultimately determines the quality of the test scripts. Good testing tools should help tester engineers transform their knowledge into reality, efficiently. But first, the test engineers need to have that knowledge.

“Expensive Tools Do Not Produce Better Designs” — The Pragmatic Programmer book, Tip 59

To be able to maintain a large number of automated test scripts that can keep up with application changes, we need to design test scripts that are intuitive and can be updated efficiently by the testers (or even business analysts).

• Intuitive to read

  Generally speaking, testers are not tech-savvy. Therefore test scripts must be intuitive, and ideally readable. How intuitive should (or can) test scripts be? If we step outside the software development and consider how people talk about websites, we might hear instructions like this:

  • After login

  • you shall see an account summary page

  • on that page, click ‘select pay by credit card’

  • Enter your credit card details on the payment page

  • Record the receipt number on the confirmation page

  These steps set a good target for our test script syntax. More on this later.

• Easy to update

  The core reason for hard-to-maintain test scripts is duplication. It is very similar to the problems caused when software code is duplicated (often by copy-and-paste). It happens too often when changes are applied, and people forget to update all the references. In the context of test scripts, duplications are often widespread in the recorded test steps. When applications change frequently (almost invariably), duplication will lead to maintenance difficulties.

DRY stands for Don’t Repeat Yourself, that is, Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.
— “The Pragmatic Programmer” by Dave Thomas and Andy Hunt

You might have heard the term “DRY”. DRY in test automation means if one test step’s corresponding operation changed in a web application (such as the button ‘Confirm’ on the payment page being renamed to ‘Pay now’), only one place (in test scripts) needs updating, regardless of this test step being used by many test cases.

Maintainable automated test design

Let’s revisit the sample instruction of how people talk about the use of websites:

You might have noticed I highlighted the login and several XXX pages, which are examples of two key concepts for maintainable automated test design: reusable function and page object.