Zhimin Zhan’s Quotes on E2E Test Automation & Continuous Testing
Based on 18 years of hands-on E2E Test Automation & Continuous Testing experience, opinionated.
This is a compilation of highlighted statements (by me and readers) from my 500+ articles. I will continue to expand it in the future.
Table of Contents
· End-to-End Test Automation (General)
· Web Test Automation
· Software Testing in General
· Misc
End-to-End Test Automation (General)
Working Automated Test ≠ Good Reliable Test
(source)
Efforts on E2E Test Automation
1. Test Creation, ~10% of total efforts
2. Test Stabilization and Refinement, ~30%
3. Ongoing maintenance, ~60%
(source)
The audience of Automated End-to-End Testing via UI is the whole team, not just automated testers and developers.
(source)
“Every software team member can benefit from running Automated End-to-End (via UI) Tests.”
(source)
Ruby is the Best Scripting Language for End-to-End Test Automation.
(source)
35-Word Functional Test Automation Strategy:
“Write 1+ Automated End-to-End Test for a user story; Add it to the regression suite; Run the regression suite daily and keep them valid! Starting on the first day and then every day!”Specific and correct Test Automation Practices will be deprived of this 35-word test automation strategy.
(source)
Engineering: proven and repeatable process that builds to last.
Test Automation Engineers, the word “Engineers” in your job title has meanings.
(source)
The App in an Agile project changes frequently, and will continue to do so. Test Automation Engineers, accept this simple fact. Don’t ever use “app changed” as an excuse for failing to maintain automated E2E test scripts.
Automated Test Scripts Shall be in the Syntax of a Scripting Language, Naturally! We call test scripts for a very good reason.
(source)
There is no rule mandating that the “Coding Language and E2E Test Scripting Language Must be Identical”. In fact, Mostly Better to be Different.
(source)
End-to-End Test Automation, e.g. Web Testing in Browser, is not limited to software engineers, but rather, to the whole team. For a real test automation implementation, A business analyst or manual tester can understand the test scripts, and run (even edit) them.
(source)
A simple rule of end-to-end test automation: if one step of a 100-step test script is not implemented or not working reliably, the whole test script is useless!
(source)
All automated tests, if valid, are valuable. The most valuable one is clearly the top tier (implied in the Testing Pyramid anyway): End-to-End UI testing.
(source)
Framework makes it work; Tool helps you be productive. Don’t let a tool to control you!
(source)
No one can master real test automation without lots of hands-on practice. Real test automation coaches can greatly speed up the learning process. But first, one needs to have a mindset of daily hands-on developing/maintaining test scripts.
(source)
End-to-End Test Automation is not Research. An automated tester is hired to perform testing tasks that a preschooler can probably perform manually in automated test scripts. So, get real work done on Day 1, then every day onwards.
(source)
Running all E2E (vis UI) test scripts frequently (as regression testing) and keeping them valid is where the real test automation work is.
(source)
Real Test Automation Engineers are extremely rare. This (so many fake automated testers) is the failure of our IT education on test automation: Universities do not teach; Companies do not provide training. Motivated engineers can learn, though.
(source)
Web Test Automation
Selenium WebDriver is the best web automation framework, better than any other one in the market, by a big margin, in nearly all aspects.
Fact: Web Test Automation has changed little over the last two decades. There is no reason for test automation engineers to fail, yet, 95+% failed.
Test Automation Engineers, our target, the Web, barely changed in the last two decades.
(source)
Test Automation Frameworks/Tools/Approaches
What is the main technical reason for Cucumber (i.e. Gherkin) failing on test automation? Test Script Maintenance.
(source)
If unable to test one operation on a web page, then it pretty much means the application cannot be tested. This is simply the nature of UI test automation. How come Cypress, with such severe limitations, became ‘popular’ in the past few years, puzzled me. (Update in 2023, cypress.io is dying).
(source)
Continuous Testing
Continuous Testing is the Holy Grail of Software Development.
(source : I might have read this from somewhere. If someone finds out its original source, please let me know)
Continuous Testing is the most important process in DevOps; Yet most so-called ‘Agile’ or ‘DevOps’ projects did not have it or got it wrong.
(source)
Continuous Testing is the foundation of both Agile and DevOps. In my opinion, this is the quickest and easiest way to determine a bad/fake Agile or DevOps project instantly.
(source)
Software Testing in General
It is neither possible nor necessary to achieve 100% coverage in software testing.
(source)
“You cannot test enough for software. But you can keep the existing automated tests valid and useful.”
(source)
Misc
“The reason that you and your team are hired is to develop a custom software solution. A generic AI bot won’t help you.”
(source)
While IT executives like to talk about “Release Early; Release Often” and “One Team, One Dream”, in the heads of many middle managers and software tech leads: “Another Day Another Dollar”. A good IT executive values E2E Test Automation and Continuous Testing as the highest priority. Once he/she does that, the team culture will be instantly better, as software development becomes transparent.
Zhimin Zhan would be struggling with maintaining 50 Cucumber E2E tests for the app under active development (change frequently). But he is very comfortable with 500+ User-Story-Level E2E tests in RSpec.
(source)
— —
Why is faking common in E2E test automation? Check out my other article,
Medium Curators marked this article as Boosted, which is “a marker on the story in our system to say this is high quality and we believe that it should go as far as possible”.
Related reading: