Analysis of "4th Edition Software Testing & Quality Report" by TestRail. Part 1
Trends and Findings from a recent software testing survey.
The “Software Testing & Quality Report”, conducted by TestRail, gathered 2,751 responses this year. You can download the full report (PDF)
As with any survey conducted by a single company, some bias may exist. However, in this article, I’ll highlight a few objective findings that align closely with the trends and predictions I’ve observed over the years.
Software Testing Trends: Test Automated, Early!
The beginning of the report summary.
“The top future initiatives include:
Increasing test automation (43%): Teams are exploring AI-driven, self-healing tests that reduce maintenance.
Shifting left (39%): QA is moving earlier in the dev cycle to catch defects before they reach production.
Prioritizing better environments and test data (35%): Better infrastructure and reusable data are key to speeding up releases as unreliable environments slow down execution and reduce confidence in results.”
This is not new. Many software teams have been lacking the capability and determination to accomplish it.
Check out my related articles:
Benefits of Real E2E Test Automation and Continuous Testing series: Executives, Managers, Business Analysts, Developers, Testers and Customers.
Findings
Let’s take a look at some key findings (I selected 9) from the latest Software Testing & Quality Report (2025). Interestingly, many of these insights have been around for over a decade.
Finding 1: Selenium is still the best web automation framework.
The above combines test syntax frameworks (e.g., Cucumber, Pytest) with automation frameworks (e.g., Selenium and Playwright for web, Appium for mobile). Stripping away the noise, Selenium's dominance in web test automation remains undisputed, unchanged for the past 14 years since 2011.
Check out my article, “Selenium WebDriver Is Still the Best Web Test Automation Framework in 2024,” published in January 2024.
Many software testers have been misled by “fake experts” or commercial propaganda that evil mudsling Selenium WebDriver, despite it being free, open-source, proven, and the only framework compliant with the W3C standard. But the results speak for themselves.
If you're looking to apply web test automation at work or for your side project, learning Selenium WebDriver is still your best option.
Check out my articles:
Comments on the claims of “7 ways Cypress is different”. All False, Wrong or Lie
Correcting Wrong “Playwright’s Advantage over Selenium” series.
Finding 2: Software teams have always wanted more test automation, but often fail to take meaningful action.
As shown in the chart above (from the report), the demand for test automation rose from 56% to 63% last year, yet the actual implementation rate remained unchanged.
Even the stagnant 40% figure for test automation is questionable. In my experience, many software teams interpret it loosely, counting white-box tests, semi-automated processes, performance/load testing, and API testing as part of the total. Often, even UI-based E2E tests that fail nightly in a CI server are included in the tally. But to me, if a team can't consistently achieve a green run (100% pass) every one or two days, the effective level of test automation is 0%—because it simply can’t be trusted.
Finding 3: What is Test Automation for?
In 2010–2011, I spoke at two software testing conferences, and both conducted surveys on attendees’ topic interests. The top three—though the order varied—were consistently: Test Automation, Regression Testing, and Web (E2E) Testing.
Fifteen years later, it seems the interests of software professionals in test automation haven’t really changed.
Findings 2 and 3 confirm that the demand for E2E (UI) test automation has remained strong for over a decade. Finding 1 shows that for the most common type of application—web apps—the go-to solution has been raw Selenium WebDriver.
Yet, there has been a lot of noise and confusion, such as the examples below.
This is a cowardly act.
Check out my articles:
Stay tuned for Parts 2 & 3, six more findings.
Related reading:
