My Innovative Solution to Test Automation: Debug Automated E2E Web Test Scripts Intuitively and Proficiently

The approach can save automated testers a few hours every day, which is a common feedback I received from mentees.

This is included in “My Innovative Solution to Test Automation and Continuous Testing” series.

Table of Contents:
· Debugging is required, as in coding, for E2E Test Automation
· Developer’s Debugging Approach is Not Appropriate
· The Right Approach to Debugging Automated E2E (Web) Tests
∘ A Programmer’s immediate reaction to a bug in the code
∘ An automated tester’s immediate reaction to a test failure
· The Code Debugging is too complex, slow and less intuitive for End-to-End Test Automation
· Case Study 1: An assertion failure
· Case Study 2: A waiting-related test failure
· How much more productivity will this debugging approach enhance for automated testers?
· How Did I come up with the idea?

Debugging is required, as in coding, for E2E Test Automation

Following the failure of commercial record-n-playbook test automation tools, e.g. HP Quick Test Pro, it has become standard practice to script automated test scripts using a conventional programming language. In other words, scripting automated tests is one form of coding.

As we know, a programmer spends more time on debugging than actually writing the code.

https://www.reddit.com/r/meme/comments/14pm80n/chatgpt_is_actually_a_problem_for_developers/

The debugging process applies to scripting automated tests, where

• a locator that is unable to find the element on the web page (wrong or the page changed).

• an AJAX or dynamic action causing the specified element not to be rendered on time.

• the locator works unreliably.

• …

The question comes, how does a test automation engineer debug automated E2E (web) test scripts effectively?

Developer’s Debugging Approach is Not Appropriate

During coding, developers use a debugger (commonly built into the IDE) for code debugging. Common debugging techniques include:

• Setting a breakpoint

• After the application pauses at a breakpoint:
  - Inspecting variables
  - Stepping over / stepping in
  - Resuming execution

A program paused at a breakpoint in a coding IDE

The app must be compiled with debugging enabled to support the above operations.

Can the above techniques be applied to debugging automated E2E test scripts? Technically, yes, but it is inefficient, slow, and doesn’t feel appropriate. Let me share a story.

One time, a senior JS programmer on the team began using my Selenium + RSpec test script to assist with his programming. Like many others, he gradually recognized the benefits of real end-to-end test automation and found it indispensable. He started customizing a few test scripts for his coding work.

One day, he called me for help in analysing why one test script wouldn’t work. I saw this in his script.

require 'pry'
#...

I asked: “Why did you include the ‘pry’ library?” (I know what Pry is for).

This senior programmer answered, “I wanted to debug the script, and to my knowledge, pry is the one to use in Ruby to do some interactive inspection, i.e. debugging”.

Then, I showed him a much easier way (the content in this article) to accomplish the task.

This story shows how little programmers know about E2E Test Automation.

The Right Approach to Debugging Automated E2E (Web) Tests