Why Are Most E2E Test Automation Training Not Effective? Part 2: Wrong Content
Overloaded with Theory, Lacking Practical Hands-On
In this article series:
Part 1: Unqualified Instructor, often not even a Dedicated One
Part 2: Wrong Content: Overloaded with Theory, Lacking Practical Hands-On
Part 3: Expensive
Part 5: Wrong Expectations of Attendants
Part 6: Advice
Part 7: FAQ
For any training, its effectiveness depends primarily on two key factors: the instructor and the content. In Part 1, I discussed the first factor — the instructor. Now, this article moves on to the second: content.
Just like the first factor — most E2E test automation instructors are unqualified — the quality of most E2E test automation training content is also poor.
Let’s see the content of one so-called ‘Agile Test Automation’ training:
I will comment on those five main topics one by one.
1. “Agile Culture and Mindset”
Completely unnecessary. It's not the year 2000—Agile hardly needs an explanation these days.
End-to-end test automation is needed for all software development methodologies, including Waterfall, though it’s even more essential for Agile (embracing changes). However, based on my observations, over 95% of self-labelled Agile projects still rely heavily on manual E2E testing.
“95% of the time, 95% of test engineers will write bad GUI automation just because it’s a very difficult thing to do correctly”.
- this interview from Microsoft Test Guru Alan Page (2015), author of “How we test software at Microsoft”
2. “Test Automation Strategy”
Unnecessary, or at most a three-minute talk. Without hands-on automation skills, long talking about the E2E Test Automation strategy is pointless. Check out my article, 35-Word Functional Test Automation Strategy.
35-Word Functional Test Automation Strategy
Write 1+ Automated End-to-End Test for a user story; Add it to the regression suite; Run the regression suite daily and…zhiminzhan.medium.com
I doubt most attendees are in a position to influence a test automation strategy anyway.
3. “Continuous Integration”
A wrong topic to attendants who haven’t done at least ~15 automated tests. In other words, they don’t have the absolute real need for continuous execution yet.
Today, continuous integration primarily involves running unit and integration tests (white-box) on CI servers like TeamCity and Jenkins.
In contrast, end-to-end (E2E) tests (black-box) fall under Continuous Testing, executed on Continuous Testing servers such as SandCastle and BuildWise.
The inclusion of the “Code Analysis” subsection reveals that the training organizer lacks a fundamental understanding of E2E test automation. In black-box functional testing, code analysis is irrelevant.
4. Automated Testing
That is the main part, I guess. However, from its section summary, it seems just introductions, maybe with a few examples.
Test Levels
maybe testing pyramid?Mapping tests
Not sure what it means? My guess is that it refers to the outdated object identification utility found in older commercial test automation tools, most of which are now obsolete.ATDD and BDD frameworks
Cucumber is pretty much dead. Check out this article, “SmartBear’s Attempt to Commercialize Cucumber for E2E test automation Failed Badly”; By the way, the same fate for SpecFlow (see my article, SpecFlow is Dying. Another Prediction of Mine Proven Correct).UI Testing frameworks
This seems like an overview of multiple automation frameworks, comparing their pros and cons. If that’s the case, ChatGPT probably does it better.
What’s the real value for the attendees? Wouldn’t it make more sense to show how to write actual tests (step by step) using a specific automation framework (e.g. Selenium or Appium) relevant to the app they’re working on?
5. Automation Support for integration and system testing
I don’t know exactly what these mean. Integration usually is white-box testing, with nothing to do with test automation engineers. System testing, which is often known as functional testing, isn’t this what attendants coming for?
Exploratory Testing is manual testing, and can be mentioned, but should be listed as an item in E2E test automation training.
In summary, this course structure is designed for instructors’ convenience, anyone with basic presentation skills (such as reading slides) can deliver it.
Effective E2E Test Automation Training is Hands-On and Highly Interactive
Software testing is hands-on and focuses more on practicality than theory. While there are underlying theories, automated software testing prioritizes quick, hands-on execution and objective feedback.
For instance, take the user story “Login with Remember Me” as an example, I expect my QA engineers to complete one or two automated test cases for it within 10 minutes. I’ll then run the automated tests to verify it. This is the pace of a real agile project. In four words: “Show me the proof,” fast and repeatable proof.
Given the nature of E2E Web Test Automation,
Everyone, including young kids, knows how to use the web.
The web has not changed for nearly two decades, from a testing perspective.
Once a test automation engineer successfully does one successful web test automation, the same approach applies to all websites.
The most effective way to learn is through hands-on exploration with well-crafted exercises and guided support.
Of course, this “learning by doing” needs to be structured with focus. It is easy to say than done. Most test automation instructors are lazy, read through the textbook/slides, or just demonstrate with simple examples. People don’t learn E2E Test automation well this way.
My Web Test Automation Training
Some readers might ask, “Zhimin, it’s easy to criticize or judge the content of other training providers. What about your own?”
Related reading: