Effective E2E Test Automation Could Have Saved the Sonos CEO’s Job
Another example of the costly mistake of being negligent with software testing, especially E2E testing.
Today, I came across this news: The Sonos CEO has stepped down, due to the company's diszardous release of its software app.
The above are all major business websites. However, isn't this primarily a matter of software technical capability? Why do so few software-focused websites report on it? In this article, I’ll share my insights on the matter.
Highlights from CNBC’s Article
First, let’s see some highlights:
Many functional defects (user cannot even perform basic operations)
“In May 2024, Sonos launched a new app that was plagued with significant issues, with users not able to perform essential functions like accessing or searching their music libraries, setting sleep timers, or even downloading the app.”
Costly (one-time)
“The company said it would cost between $20 million and $30 million to fix these issues and decided to cut about 6% of its staff.”
Executives lost money too
“Spence, in October, had acknowledged mistakes surrounding the app’s release and said that he and seven other company leaders would forgo their bonuses.”
The damage to the company is long-lasting
“In November, the company forecast a decline of about 9% to 22% in its first-quarter sales from a year earlier”
What do we learn?
1. Software E2E testing at Sonos failed badly.
This is obvious, of course. Do you think the Sonos executives will accept the excuse, “We have done good unit/integration testing, even something like Cypress component testing”?
E2E testing (from an end-user perspective) is surely the most important.
It's somewhat sad that, throughout my work with various companies, I often had to convince engineers and managers of this.
In Agile, real Agile, E2E testing is done mostly (still some are manual) via Automation, via UI.
There are many posts online claiming “Automated E2E testing is brittle, so avoid that”. This cowardly statement is of course wrong. Of course, there are companies (big and small) that accomplished that, did they seek help?
“Facebook is released twice a day, and keeping up this pace is at the heart of our culture. With this release pace, automated testing with Selenium is crucial to making sure everything works before being released.” — DAMIEN SERENI, Engineering Director at Facebook, at Selenium 2013 conference.
Here are a few questions:
Has your company ever worked with a hands-on Test Automation Coach?
If not, how can you confidently assert that “Automated E2E (UI) testing is not possible?”If testing failed using Cypress (and it was blamed), have you considered trying raw Selenium WebDriver, which has a proven track record of success?
Remember, web test automation skills are universally applicable to all websites, as the core of the web has remained largely unchanged for two decades. I’ve consistently used the same tech stack to implement successful test automation across various projects, following the exact same approach.Is the testing team open to using Ruby, widely regarded as one of the best scripting languages and recommended by the classic Agile Testing book?
Check out “Why Ruby is the Best Scripting Language for End-to-End Test Automation?”
The above questions are often dismissed outright by test managers, architects, and principal software engineers—I’ve seen it happen countless times. However, when a wise CEO or CIO steps in and truly gets involved, the nonsense tends to become clear very quickly.
So, if E2E Testing is not doing well, it is the CEO/CIO’s fault.
2. The pain point is regression testing.
Please note the root of this whole debacle is the software update; various defects after changes.
Astute readers might have some thoughts on this sentence:
“The company said it would cost between $20 million and $30 million to fix these issues”
Why so much? The reason is that the company lacks proper regression testing, typically, many software companies conduct regression testing manually, which is costly (money and time).
The solution is, of course, Automated E2E (UI) Regression Testing, but the company did not have this capability at all, and very likely has a number of fake QA engineers there.
Please note doing good regression testing is not a big or extra demand, really, it is part of basic software engineering.
Advice to CEOs
End-to-End (UI) Test Automation serves as a safety net for both you (as CEO or CIO) and your company. Make it a top priority from a technical standpoint, like LinkedIn’s Founder as in the Wired Article: “The Software Revolution Behind LinkedIn’s Gushing Profits”.
For more insights, check out my other article.
Action Today for Wise IT Executives:
Find an internal talent.
Follow this simple and quick process: “Test Automation and Continuous Testing Competition Week”. This time, ensure that you (CEO) are directly involved to prevent any political games.Lure an external talent.
If your company is large and willing to offer millions in annual salary, consider luring top E2E test automation talent, just as LinkedIn executives have done.Seek a qualified hands-on E2E Test Automation Coach
For financially constrained companies, consider seeking training and on-demand mentoring services from a real E2E test automation coach.
Advice to Software Solopreneurs
Embrace E2E UI Test Automation. I spent ~70% of SDLC Effort is on Software Testing (E2E UI Test Automation) and Related Activities.
Related reading:
My eBooks:
- Practical Web Test Automation with Selenium WebDriver
- Practical Continuous Testing: make Agile/DevOps realBenefits of Real E2E Test Automation and Continuous Testing series: Executives, Managers, Business Analysts, Developers, Testers and Customers.
A Good Software Testing Culture Can be Easily Broken, Lessons learned from Google’s “$100 billion error” yesterday.
Simple Solution to Embarrassing Collapses of Census Website due to Poor Load Testing
🎖A Tale of a Deceptive End-to-End Test Automation Engineer (Boosted)
AgileWay’s Web Test Automation Creation Service: One Test Step for $1