End-to-End Test Automation Is Judged By Results, Ongoing Results
Other factors do not matter and shouldn't matter much, however, …
End-to-End testing for web applications involves verifying that a web app functions according to its specifications from the viewpoint of an end user. Automation refers to the practice of using automated scripts to carry out testing, without changing the inherent nature of the testing.
The details in the middle, such as the:
Programming language (C#, JavaScript, or Ruby),
Web framework (AngularJS, React, or Ruby on Rails),
Architecture (monolith or microservices), or
Deployment environment (on-premise or cloud)
are not relevant! E2E test automation engineers are not concerned with these details — they are for developers to worry about. E2E test engineers focus solely on interacting with the application as an end user would.
For web applications, which are built on the W3C standard, E2E test automation choices should be completely independent of the app’s tech stack or the developers behind it.
automation framework, e.g. QTP, Cypress, Playwright or Selenium WebDriver
test syntax framework, e.g. JUnit, Pytest, Cucumber or RSpec
test scripting language, e.g. C#, JavaScript, Python or Ruby
The bold ones are the combination my father has been using successfully since 2011, for a number of web apps implemented in various languages and on different deployment architectures.
So, how do we define End-to-End Automation Success?
“Running the whole test suite *daily* as valid and trustworthy regression tests.” — Zhimin Zhan
The keyword here is “Daily”!
Suppose you’ve achieved genuine test automation success; with your test suite running on a daily cadence and your team trusts them to pick up defects. Even if the tests are written in ProtractorJS — a framework that was officially deprecated four years ago. Does that really matter? Not much. The web hasn’t changed significantly in the past decade. As long as the testing team runs those tests (the whole suite) daily and the tests provide good feedback (valid, of course), it is fine! Actually, more than fine!
If you are doing well, regardless of what technology/tool, ignore anyone who tells you otherwise.
What is really important (more than the technologies/tools) is the ongoing aspect of E2E Test Automation. An app changes constantly and frequently. Hence, the majority of the E2E test automation efforts are in maintenance.
The value of the whole E2E test suite remains a long time after the development phase, in fact, it shines. Without solid automated E2E regression testing, how do you cope with defect fixes and change requests?
With good E2E Test Automation and Continuous Testing, it can lead to unlocking other great benefits such as “Daily Production Releases”.
That’s my stake in E2E Testing. In principle, I am not against any framework or tool. I judge by the real, ongoing results (like below).
and real benefits, such as enabling ‘ship frequently to production’ with confidence.
