Why Are Most E2E Test Automation Training Not Effective? Part 1: Unqualified Instructor, often not even a Dedicated One
Real E2E Test Automation Engineers are rare, make sure your Instructor is one of them. Here, by qualification, I mean practical hands-on skills, not just a certificate.
Given the importance of E2E testing in agile projects and the lack of test automation education in universities, one might assume software companies routinely provide their staff with hands-on, instructor-led E2E test automation training (NOT pre-recorded courses like those on Udemy). However, the reality is quite the opposite.
Quite commonly, testers wanted to learn E2E test automation, but their managers usually completely ignored their requests, as depicted in this real story. Why? I can think of two main reasons:
Many test managers have experienced multiple failed test automation attempts, leading them to doubt the effectiveness of test automation (even though they can’t publicly oppose it, self-labelled ‘Agile’).
The quality of their experienced E2E test automation is very poor.
I encountered this situation several times while working on contracts. My quick and successful implementations usually raised some interest. Someone higher, a test practice lead, would organize a free, one-day training session for testers across various teams. However, it was quite common for some project test managers to prevent their testers from attending, believing that the testers would be better off spending that day on manual testing.
In this article series, I will delve into the second issue.
Part 1: Unqualified Instructor, often not even a Dedicated One
Part 2: Wrong Content: Overloaded with Theory, Lacking Practical Hands-On
Part 3: Expensive
Part 5: Wrong Expectations of Attendants
Part 6: Advice
Part 7: FAQ
The focus of this article is the core factor of any training or education: Instructor. Its importance is needless to say.
Before starting this series, I quickly searched three test automation training programs offered by mid-sized companies, and none even had a dedicated instructor. Yes, you read that right — crazy, right? You’d be better off skipping those.
Below, I’ll share my observations and thoughts on E2E test automation training with a named instructor (I’ve relaxed my standards quite a bit here).
The odds are that the Instructor doesn’t really understand E2E automation
In areas such as piano teaching or tennis coaching, it’s unimaginable for an instructor to lack knowledge of the subject or be unable to perform the basic task on request. Similarly, good presentation & personal skills alone are certainly insufficient to qualify as an E2E Test Automation instructor. Unfortunately, this issue is quite prevalent in the field of E2E test automation.
A large percentage of Test Automation Instructors do not really know E2E test automation. Why? Real Test Automation Engineers are Extremely Rare.
Let’s look at the quotes from some renowned experts.
“In my experience, great developers do not always make great testers, but great testers (who also have strong design skills) can make great developers. It’s a mindset and a passion. … They are gold”.
- Google VP Patrick Copeland, in an interview (2010)“95% of the time, 95% of test engineers will write bad GUI automation just because it’s a very difficult thing to do correctly”.
- this interview from Microsoft Test Guru Alan Page (2015)“Testing is harder than developing. If you want to have good testing you need to put your best people in testing.”
- Gerald Weinberg, in a podcast (2018)
What is the likelihood that your course instructor is among the “Gold,” “Best People,” or the top 0.25% (5% of 5%)? Please note that these low percentages are in large high-profile IT companies, where the overall staff quality is generally higher.
Don’t be misled by some high-profile, so-called world-renowned figures in software testing. I’ve met a few — they talk well about E2E testing on stages, but based on my conversations and observations, these ‘experts’ lack even basic hands-on test automation skills. I might share some stories in the future.
My Qualification
Some might question my eligibility to say the above. Here it is.
Over 18 years of hands-on work in test automation and continuous testing
Speaker at international software testing conferences, including the premium StarWEST conference.
Creator of TestWise IDE, demonstrated at Agile 2009 workshop; a finalist of Ruby Award 2010; featured in Ward Cunningham’s testing tools matrix; listed as the first testing tool in Lisa Crispin’s presentation “Trends in Agile Testing” in 2009.
Creator of BuildWise, winner of the 2018 Ruby Award (Japan).
Authors of 14 books on Test Automation and Continuous Testing
Implemented several highly acclaimed own apps, thanks to my Automated End-to-End Regression Testing process (which counts ~ 70% of the whole development effort)
150+ articles featured in leading software testing newsletters.
All the above, I consider myself a Level 4 (Expert, better than 99.9%) of the AgileWay Continuous Testing Grading. This isn’t meant to be boastful (there’s no point in showing off here); rather, I just want to add weight to say, “I agree with those experts, doing real test automation is not that easy, especially the test suite reaches a certain size”.
Don’t just take my (or anyone’s) word for claiming expertise in E2E Test Automation — verify it. Why? Because it’s quick and easy to do. After all, we’re testers, and true test automation engineers should welcome being tested.
I offer the following two services:
30-minute Test Automation Coaching for $1
Learn Web Test Automation from an accomplished Test Automation and Continuous Testing coachAgileWay’s Web Test Automation Creation Service: One Test Step for $1
A high-quality web test automation scripting service at an incredibly low price
in which, serious testers can verify whether I can do E2E test automation on the spot, for your web app.
Testing Instructor
Some readers might wonder, “How can we assess instructors who don’t offer public tryouts like you do?”.
You can test that out on people who claim to be proficient in E2E test automation.
Asking simple yet unavoidable questions.
During interviews, I usually started with the four simple screening questions.
Four Screening Questions to Filter Out Fake Test Automation Engineers in Interviews
Hiring a fake automation tester can be costly. These four simple screening questions may help prevent that
You probably can tell the fakers from their answers.
Below is a simple scenario question I asked in many job interviews. No one could answer it properly, but it is really a simple one.
One Simple Test Automation Scenario Interview Question that Most Candidates Failed
Simple, yet every candidate I interviewed failed
One large tech company here included the above scenario question in their interview process, but later dropped it. The reason, the testing lead told me later, “No one got it right”.
For such a simple test case, “User Change Password”, nearly every job candidate failed. Why? They never developed a proper test automation mindset.
2. Verify that they understand the importance of maintenance in E2E Test Automation.
Writing a few demo automated tests is easy. People did that over 20 years ago with QTP. The result: the industry-leading HP’s QTP is pretty much dead (sold to Micro Focus in 2017, then opentext) despite the fast-growing software testing market.
Based on my observation, test creation is only about 10% of the overall effort. The other two are Stabilization (including refinement and refactoring) and ongoing Continuous Testing.
Certainly, good test automation training will extensively cover test maintenance.
For some readers who have attended test automation training before, has the following topics: stabilizing test, test script refactoring, debugging, test execution from the command line and in a CT server?
I introduced this mindset in the first hour of my Test Automation Training.
Some test managers might think, “I’ve looked at several test automation training courses and couldn’t even find the instructor’s name, let alone ask him/her questions.” This highlights the severity of the situation. Unfortunately, this is just one of the many issues plaguing E2E test automation today, which I will discuss in Part 2.
Related reading: