Auto-Waiting in Web Test Automation Clarified

The so-called “auto-waiting” mechanism is, in fact, inferior; it is just marketing. When it comes to real e2e test automation, Selenium WebDriver’s waiting mechanism is better.

This article is one of the “IT Terminology Clarified” series.

According to this WebDriverIO document, “One of the most common reasons for flaky tests are interactions with elements that don’t exist in your application at the time you want to interact with it. Modern web applications are very dynamic, elements show up and disappear.” While the description is correct, often fake automated testers shift “flaky” blame to the framework, which is very wrong. This is a human issue, flaky tests are poor-quality ones. Real test automation engineers (such as the ones at Facebook and Google) have been achieving successful test automation on dynamic websites using raw Selenium WebDriver.

The so-called “Auto-Waiting” feature to address ‘flaky tests’ is a marketing slogan by no-good automation frameworks to lure fake automated testers to a dark path.

Table of Contents:
· Background
· Selenium IDE, a recorder, was a Mistake
· Dynamic Apps make “Record-Dependent Automated Testers” Cannot even Fake anymore
· Selenium WebDriver Always Had Good Solutions to Handle Dynamic Web Apps
· Why Most Automated Testers Did not just use Selenium WebDriver Waiting?
· The Madness of “Creating Own Web Automation Framework”
· Tool Vendors
· Dissect Cypress/Playwright ‘Auto-Waiting’
· I Implemented ‘auto-waiting’ and then realized it was a mistake
· My Innovative Solution

Background

A bit of history background. Before the dynamic web apps (maybe ~2012), Selenium WebDriver had already been the most dominant web automation framework. The stories of Selenium’s success at Google and Facebook were well accepted. There was no other. The old-generation record-playback automation tools (most are commercial), such as HP Quick Test Pro and IBM Rational Functional Tester, were fading quickly out of the market.

Also, most software engineers, from the first time, started playing test automation (as Selenium WebDriver is 100% free) because the commercial tools were very expensive, typically costing US$10K+ per seat license.

Automated testers then (still a fair percentage now) associated record/playback with test automation. Many new-to-Selenium testers usually started with a recorder.

Now looking back, it was such a pity! If there was a movement to educate these automated testers to learn hands-on scripting Selenium properly, our testing and software industry would be much better now.

Selenium IDE, a recorder, was a Mistake

It was quite logical that the Selenium team released its recording tool: Selenium IDE. I thought it was a big mistake when I first saw it.

It turned out that my concerns were justified. Here are some quotes from Jason Huggins, the creator of Selenium v1, at the Agile Alliance Functional Testing Tools (AAFTT) workshop in 2009.

• “Selenium IDE is the place to start with Selenium, but it is Selenium on training wheels”.

• “Record/playback testing tools should be clearly labelled as “training wheels”.

• “What to do with the Selenium IDE, no self-respecting developer will use it.” Thinking of renaming the IDE to Selenium Trainer.

The naming of Selenium IDE is particularly bad, and the proposed renaming (Selenium IDE to Selenium Trainer) did not happen. This means, right-mindset testers who tried to google ‘Selenium IDE’ (a real functional testing IDE, like TestWise), would return this recorder.

Dynamic Apps make “Record-Dependent Automated Testers” Cannot even Fake anymore