A Few Silly yet Common Decisions Often Lead to E2E Test Automation Failures
Avoid those obvious mistakes.
A repost of my past article on Medium
Fact: most IT professionals do not understand E2E (UI) Test Automation. This is excusable. To my knowledge, no universities offer dedicated courses on E2E Test Automation despite its obvious importance in software development.
End-to-end (vis UI) Test Automation has alarming high failure rates.
“95% of the time, 95% of test engineers will write bad GUI automation just because it’s a very difficult thing to do correctly”.
- the interview from Microsoft Test Guru Alan Page (2015)
“Automated testing through the GUI is intuitive, seductive, and almost always wrong!” — Robert C. Martin, co-author of the Agile Manifesto, on his blog (in 2009)
Wise IT executives should avoid silly mistakes like the following:
∘ 1. Management and Tech Leads mandate a specific automation framework/tool or scripting language.
∘ 2. No rule says “coding language and e2e scripting language must be the same.”
∘ 3. The company mandates using a CI server, e.g. Jenkins, to execute E2E tests.
∘ 4. Putting other heavy-weight processes on automated test scripts
1. Management and Tech Leads mandate a specific automation framework/tool or scripting language.
Quite often, in a Job Ad for a test automation engineer, the automation framework/tool has been specified. When on board, you most likely find out that all previous tempts on E2E Test Automation FAILED. Who made this decision? A more important question is, Are they the same people who made the previous wrong decisions?
E2E Testing is black-box testing, there is NO mandate on a particular framework or scripting language, from a management’s perspective. A simple and logical way is to try reputable automation frameworks and scripting languages out (totally independent from coding languages) and choose a good one objectively and quickly. Prove it by results in days, not weeks or months. See my other article, Test Automation and Continuous Testing Competition Week.
The same goes for the test syntax framework. Cucumber was often brought in by a manager or senior architect who thought “BDD is a good idea”.
How wrong is that?!
Why Gherkin (Cucumber, SpecFlow,…) Always Failed with UI Test Automation? (2021–01)
A Practical Advice on Rejecting Gherkin for Test Automation (2022–04)
“Cucumber is Dying”, What did we learn? (2023–05)
2. No rule says “coding language and E2E scripting language must be the same.”
This has got to be the most common and frustrating cause that I experienced in test automation consulting. As I said earlier, for E2E Testing, the choice of scripting language should be independent of the code. I have written a number of successful E2E Test Automation using Selenium + RSpec (in Ruby language), for apps coded in various languages, including Java, C#, JavaScript, PHP, and Ruby.
For more on this topic, check my other article, There is no rule mandating that the ‘Coding Language and E2E Test Scripting Language Must be Identical’. In fact, Mostly Better to be Different.
3. The company mandates using a CI server, e.g. Jenkins, to execute E2E tests.
Keep reading with a 7-day free trial
Subscribe to The Agile Way to keep reading this post and get 7 days of free access to the full post archives.