A Long List of Failed E2E Test Automation Hypes I Foresaw

Lessons from Two Decades of Predicting What Doesn’t Work in E2E Test Automation

In 2025, I marked 20 years in the field of End-to-End (E2E) Test Automation and Continuous Testing—the backbone of CI/CD, Agile, and DevOps.

These fields have been filled with hype for decades. To cut through it, you only need to ask two simple questions:

1. How does your team actually perform system (functional) testing, especially regression testing?
   My guess of your answer: Manual testing.

2. Has your team truly achieved Continuous Delivery (to production, every working day or sprint) as defined by Agile principles?

   

   My guess of your answer: No.
   

Were my guesses correct? If so, then little has truly changed in the last two decades. Keep in mind, the primary target of E2E testing—web applications—has barely changed over the past 20 years.

My writing is grounded in years of successful, hands-on test automation experience. Unlike many so-called ‘experts,’ I’m not afraid to make predictions, challenge overhyped ‘new’ trends, and so far, my track record (listed below) has been 100% accurate.

• Record-n-Playback

• Headless Testing

• Cross-Browser Testing

• BDD with Gherkin-style test syntax

• ProtractorJS, developer-oriented E2E test automation

• Cypress, a flawed tool that attracts dishonest people

• AI Test Automation

1. Record-n-Playback

In the late 1990s and early 2000s, the test automation market was dominated by ultra-expensive commercial tools such as HP QuickTest Pro and IBM Rational Functional Tester.

After reviewing some of them, I quickly concluded that this approach to test automation was flawed. I included this prediction in my first book, Practical Web Test Automation (2009).

Check out my article, “Record/Playback in Test Automation is Bad, mostly.” (2023-03)

2. Headless Testing

Fake test automation engineers and senior developers who only pretend to understand automation still puzzle me with their fascination for headless testing—long after the hype had already faded. In meetings, they kept praising headless testing as if it magically solved the problem of slow or flaky test execution.

It started with PhantomJS, which was deprecated in 2017.

“Headless Chrome is coming. I think people will switch to it, eventually. Chrome is faster and more stable than PhantomJS. And it doesn't eat memory like crazy.” — the maintainer of PhantomJS.

However, his judgment was wrong again. It turned out Chrome headless was a separate implementation of the browser, before Chrome v112 (released in 2023). This is a big slap on the faces of fake automated testers, as so-called ‘headless browser testing’ previously violated the fundamental law of software testing, unnecessarily.

Check out my article, “Headless Browser Testing Clarified” (2021-09)

3. Cross-Browser Testing