Real End-to-End Test Automation: Far More Than Just Quality Checks. Part B: Release Early, Release Often
Achieving Real Continuous Delivery Through Reliable E2E (UI) Test Automation.
In this new series, I will focus on the results, real results (not fake ones such as made-up velocity points) powered by real E2E (UI) Test Automation.
Part C: Dare to Explore Simpler & Better Options (coming soon)
Part D: Make Fixing Defects Much Easier, Quicker and Cheaper; Plus, Prevent Stuff-ups. (planned)
Part E: Team Morale Boost (planned)
Part F: Professionalism (planned)
Part G: The value continues to shine in the software maintaining phase (planned)
CIOs like to talk about “Release Early; Release Often” (RERO) on stage. Sure, to achieve that not only enable real Agile and DevOps, but it also brings great benefits, such as:
Frequent releases enable companies to quickly gather user feedback,
Releasing smaller, incremental updates allows companies to deliver value to customers more rapidly.
Frequent releases encourage teams to experiment with new ideas and technologies.
Regular updates signal to customers that the software is actively maintained and improved.
Breaking down development into smaller, manageable cycles reduces the risk of large-scale project failures.
…, and many more.
Frankly, during my nearly-30 years working in the software industry, no single case that people argue about the great benefits of “Release Early; Release Often”, however, the successful implementation was extremely rare, but possible.
Real Examples
1. FaceBook.
Recently, Mark Mark Zuckerberg publicly described his successful “formula” for Product Strategy: “Ship Frequently” (source: LinkedIn post).
Some might wonder how? the capability to achieve “ship frequently”. It is E2E test automation.
“Facebook is released twice a day, and keeping up this pace is at the heart of our culture. With this release pace, automated testing with Selenium is crucial to making sure everything works before being released.” — DAMIEN SERENI, Engineering Director at Facebook, at Selenium 2013 conference.
For many software professionals who are in doubt, check out this great presentation “Continuous Integration at Facebook” (Youtube).
2. LinkedIn
Wired, the most respected technology magazine, published “The Software Revolution Behind LinkedIn’s Gushing Profits” in 2013. Here, I just share a few quotes in that article:
See, quite clear, right. Like Facebook, E2E test automation enabled ‘frequent releases (to production)”.
My own company
After the above two examples, some may use the excuse, “These two are global tech giants, we don’t have resources (time/money/talent) to implement real E2E test automation, i.e., Agile, DevOps, CI/CD. So, we fake it.”
Real E2E test automation can be achieved in any size, even for a one-person side hustle project. Check out this article, “Showcase a 500+ End-to-End (via UI) Test Suite: E2E Test Automation is Surely Feasible for Large/Complex Apps”
An example: Case Study: Real "Release Early; Release Often" ClinicWise Development. First production release after 40-hour development; Then Daily Production Releases.
Sure, E2E test automation failures are extremely high (~99.5+%),
“95% of the time, 95% of test engineers will write bad GUI automation just because it’s a very difficult thing to do correctly”.
- this interview from Microsoft Test Guru Alan Page (2015), author of “How we test software at Microsoft”“Automated testing through the GUI is intuitive, seductive, and almost always wrong!” - Robert C. Martin, co-author of the Agile Manifesto, on his blog (in 2009)
mainly for two reasons::
Not taking it seriously (like the above three companies).
Not knowing how.
For the latter, Check out my new book: End-to-End Test Automation Anti-Patterns: Critical Lessons from Real-World Test Automation Failures.
By the way, paid subscribers to this newsletter (just $50 annually) can get this eBook for free.
