Maintainable Automated Test Design

Design automated test scripts that are easy to maintain

All test automation engineers know that a single application change may break many automated test scripts, and these kinds of application changes happen constantly. To effectively deal with it, test automation engineers need to be in a position to respond to a simple application change in a matter of seconds (here I mean the time taken to modify the test scripts, excluding execution).

The key to test maintenance (a common problem in test automation) is the test script, not the test tool. Tool vendors like to focus on their tools or deliberately blur the relationship between test scripts and testing tools, for the sake of commercial interests. It is the test engineer’s knowledge that ultimately determines the quality of the test scripts. Good testing tools should help tester engineers transform their knowledge into reality, efficiently. But first, the test engineers need to have that knowledge.

“Expensive Tools Do Not Produce Better Designs” — The Pragmatic Programmer book, Tip 59

To be able to maintain a large number of automated test scripts that can keep up with application changes, we need to design test scripts that are intuitive and can be updated efficiently by the testers (or even business analysts).

• Intuitive to read

  Generally speaking, testers are not tech-savvy. Therefore test scripts must be intuitive, and ideally readable. How intuitive should (or can) test scripts be? If we step outside the software development and consider how people talk about websites, we might hear instructions like this:

  • After login

  • you shall see an account summary page

  • on that page, click ‘select pay by credit card’

  • Enter your credit card details on the payment page

  • Record the receipt number on the confirmation page

  These steps set a good target for our test script syntax. More on this later.

• Easy to update

  The core reason for hard-to-maintain test scripts is duplication. It is very similar to the problems caused when software code is duplicated (often by copy-and-paste). It happens too often when changes are applied, and people forget to update all the references. In the context of test scripts, duplications are often widespread in the recorded test steps. When applications change frequently (almost invariably), duplication will lead to maintenance difficulties.

DRY stands for Don’t Repeat Yourself, that is, Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.
— “The Pragmatic Programmer” by Dave Thomas and Andy Hunt

You might have heard the term “DRY”. DRY in test automation means if one test step’s corresponding operation changed in a web application (such as the button ‘Confirm’ on the payment page being renamed to ‘Pay now’), only one place (in test scripts) needs updating, regardless of this test step being used by many test cases.

Maintainable automated test design

Let’s revisit the sample instruction of how people talk about the use of websites:

You might have noticed I highlighted the login and several XXX pages, which are examples of two key concepts for maintainable automated test design: reusable function and page object.

1. Reusable function

As we know, a user login function commonly consists of three steps:

1. Enter a user name in a text box

2. Enter a password in the password box

3. Click the ‘Login’ button

For this frequently used function, we prefer not to repeat these 3 test steps every single time (which would be a good example of causing duplication) in our test scripts.

Here is a sample test script (in Ruby, the design works for all languages, see the link below for examples in other languages) using a reusable function for user login.

login(“bob”, “pass”) 
# …
login(“wendy”, “boss”)

The function’s definition:

# driver is already defined 
def login(username, password)
  driver.find_element(:id, “username”).send_keys(username)
  driver.find_element(:id, “password”).send_keys(password)
  driver.find_element(:xpath, “//input[@value='Sign in']”).click
end

These reusable functions are defined in a test helper ( I often name it ‘test_helper.rb’ ), which is shared by all the test scripts, i.e. reusable.

2. Page Object Model (also known as Page Pattern)

“If you have WebDriver APIs in your test methods, You’re Doing It Wrong”
— Simon Stewart, lead of Selenium WebDriver project (link)

Page Object is a common test design pattern (in fact, listed as a best practice on Selenium documentation) that groups a web page’s operations into a ‘logic page object’ (feel confused? Stay with me, it will become clear after you see some examples). Here is a sample test script using a page object credit_card_page:

credit_card_page.enter_holder_name(“John”) credit_card_page.enter_card_number(“4242424242424242”)

The page name credit_card_page reveals the purpose of operations on the page, enter_holder_name and enter_card_name are two operations on this page.

require File.join(File.dirname(__FILE__), "abstract_page.rb")class PaymentPage < AbstractPage  #...  def enter_holder_name(holder_name)
    driver.find_element(:name, "holder_name").send_keys(holder_name)
  end  def enter_card_number(card_number)
    driver.find_element(:name, "card_number").send_keys(card_number)
  endend

Page Objects can be a difficult concept to understand initially for people who haven’t programmed before. It is important not to get them to learn Object-Oriented concepts and programming, which are not necessary. Instead, guide them with an easy-to-understand exercise, like the one in the “Practical Web Test Automation” book. From my experience of training and working with numerous manual testers, with a carefully designed exercise and a good testing tool, everyone can master it quickly (in minutes).

Maintain with ease

Besides syntax benefits (easy to read), test scripts that make use of reusable functions and Page Objects are much easier to maintain. This is because test steps are now in reusable functions and the operations are in page objects. Test scripts in (top-level) test cases are not referencing test steps directly. Instead, it becomes essentially a two-tiered arrangement.

When a change is introduced on a web page, its corresponding test steps need to be updated accordingly. If the affected test steps are used directly in many test cases, we need to update them one by one, which is tedious and error-prone. If test cases use page objects, there is only one place that needs to be updated — the function in the page class.

For example, after identifying the credit card page was changed in the new build, we just simply navigate to the definition of page class: CreditCardPaymentPage and update the test steps for affected operations there. Depending on the nature of application changes, test cases themselves might not need to be updated at all.

Sample test script: https://github.com/testwisely/agiletravel-ui-tests in several languages, such as RSpec (Ruby), Cucumber (Ruby), Mocha (JavaScript), and PyTest (Python).

Sample test project (opened in TestWise, can be in others as well) :

I have been using the same test project structure (very simple, right?) for over 10 years. The test suite for my own app WhenWise has about 500 Selenium tests, the same project structure: one test helper, and one “pages” folder containing many page classes (pay attention to the scroll indicator in the left project pane).

WhenWise regression suite: 558 tests; 183 page classes; test steps (lines): 28938

Can you identify the reusable functions and page classes in the above test script (displayed section)?

• reusable functions

  • business functions:sign_in

  • general web-testing functions: visit , try_for (for testing AJAX)

• page classes:

  • FindBusinessPage

  • BookResourceCalendarPage

  • BusinessSearchPage

  • ConfirmBookingModalPage

Work with Maintainable Test Design efficiently

To efficiently develop/refine your test scripts in maintainable test design, use Functional Test Refactoring.

---

Further reading:

• PageObject pattern by Martin Fowler