Record/Playback in Test Automation is Bad, mostly.

Avoid using record-n-playback tools. Hand-craft automated test scripts instead.

This article is one of the “IT Terminology Clarified” series.

The Record/Playback approach to test automation has been controversial for decades. I recall from my experience attending software testing conferences (2009–2013) and informal meetups that the survey results consistently revealed that testers generally dislike this method. Despite this, record-n-playback continues to resurface in various forms.

Often, I was tasked to rescue a failed test automation project implemented using inappropriate tools such as QTP, Ranorex, RFT, and Tosca. When I asked why these tools were selected, the response was always the same: “Because they can record automated test scripts.”

While it is true that recording tools make it easier to create automated test scripts, there is usually a significant downside unless the test scripts are only intended for single use (which is rarely the case).

Table of Contents:
· Recorded test scripts are harder to maintain
· Recorders don’t work (well) for certain operations
· A recorder may be of some value, but “playback” shall be strictly prohibited.
· Hand-crafting automated steps can be easy, efficient and fun!

Recorded test scripts are harder to maintain

In the realm of test automation, test creation accounts for a relatively small percentage, approximately 10%, based on my estimation. This is decided by the nature of software development, where changes occur frequently and consistently. A single modification (to code, test scripts, or infrastructure) can lead to end-to-end (E2E) tests failures. Consequently, genuine test automation professionals are primarily concerned with maintenance rather than creation.

Because test creation is of minor importance in the grand scheme, I never bother estimating story points for creating new automated tests.

Recorded automated test scripts come with a host of drawbacks:

1. The test script might only work for this software build

In some web apps (commonly Microsoft ones), GUID is used as the identifier for the web controls, like below:

<input id="1e40415d-677c-4743-b990-e3307466abfd" name="username-$1e40415d-677c-4743-b990-e3307466abfd" arial-label="Username"/>

A recorder usually generates a test step like the one below:

driver.find_element(:id, "1e40415d-677c-4743-b990-e3307466abfd").send_keys("NotWise")

It would work, but only for this build. However, the test would fail on the following internal release because the GUID would differ. 😱

2. Prone to change

As demonstrated in the previous example, using GUIDs in automated tests is evidently erroneous, a fact that most testers are likely aware of. While a slightly smarter recorder might avoid utilizing GUIDs, recorded test scripts are still sub-optimal than hand-crafted ones.

I know, some readers would argue: hand-crafting would be too much work. It is not, I will show you shortly.

For example, suppose one page has two controls with exactly the same name attribute (and this is the only identifiable attribute). A typical recorder generates a test step like this:

Page/Input[@name='nextBtn'][1].click # the last (2nd) button

It worked. A few days later, another button with the same name attribute was inserted. Then, the test scripts would fail.

A more reliable way is to use XPath to look for attributes in its parent (or even great-parent) node, like this.

# click the last button
driver.find_element(:xpath, "//div[@class='form-action']//input[@name='nextBtn']).click

3. Testers care less about the quality of recorded automated test scripts

Human factors play an important role here too. If a tester can produce an automated test script by clicking or entering texts into the controls (recording), he/she naturally won't care much about the quality of the scripts.

This mindset will affect others’ views on test automation, such as managers and developers.

Recorders don’t work (well) for certain operations

Modern web applications are significantly more dynamic, primarily due to the widespread use of JavaScript in websites. Consequently, some operations cannot be reliably recorded using automated tools, then, recording is ineffective.