Selenium WebDriver is Still the Best Web Test Automation Framework in 2024
And beyond…
On 2023–12–27, A cypress tester posted a survey in LinkedIn’s Software Testing & Automation group (with 350K+ members).
Table of Content
· Web Test Automation Framework Survey
· The Result
· (Update 2024-12) Web Test Automation Framework Survey Late 2024
· The Analyse
∘ 1. Selenium WebDriver is a clear leader by a huge margin.
∘ 2. Playwright has taken over Cypress
∘ 3. Selenium, Playwright, and Cypress are really the three options for web test automation.
· Which one will you choose for the 2024 Test Automation Work?
Web Test Automation Framework Survey Late 2023
Below is the screenshot of the result on that day. (I will post the final survey results later). I show the initial day’s outcome to assess whether there is a significant variation in the final result. (the verdict: no variation)
The question is neutral. The (initial) result might surprise the author, whose profile (shown in the survey post) is “Cypress | JavaScript”.
The result does not surprise me, though. Michael Palotas did a similar survey during the TestFlix conference in October 2023.
Selenium: 59%
Playwright: 8%
Cypress: 9.5%
Appium: 9.5%
Others: 14%
This includes non-web test automation, e.g. Appium. Let me make a small adjustment to focus on just web test automation, excluding the portion of Appium and small of others, and apply them equally. (FYI, Appium uses Selenium WebDriver)
Selenium: 59 + 4.5% = 63.5%
Playwright: 8 + 4.5% = 12.5%
Cypress: 9.5 + 4.5% = 14%
Others: 10%
The State of Test Automation: Insights From The Field
It is quite close to the recent one on LinkedIn’s Software Testing & Automation group, right? If I apply the non-web proportionally, the result would be even more so.
I noted Michael Palotas’s survey but didn’t share or write an article about it. Having spent 3.5 years as an IT research scientist, I cultivated a practice of citing at least two independent sources.
The Result
The LinkedIn Survey, from a statistical point of view, is a good source:
Only approved members can vote
there are a large number of responses, ~ 1000
from a large audience base, ~350K
The final result:
(Update 2024-12) Web Test Automation Framework Survey Late 2024
On 2024 December 1st, another survey in the LinkedIn Software Testing & Automation group. The result:
I published this article on 2024-01-03 (firstly on Medium, still the original version). The latest survey (end of 2024) has confirmed that my prediction was correct: "Selenium WebDriver is Still the Best Web Test Automation Framework in 2024."
The Analyse
I draw the following conclusions from the above and my experience (17 years in test automation).
1. Selenium WebDriver is a clear leader by a huge margin.
You might have seen posts or comments about “Selenium is the past” or “WebDriver has been replaced by new modern frameworks” …, oh well, they are wrong.
2. Playwright has taken over Cypress
Combining Michael Palotas’s TestFlix survey, we can conclude that Playwright is eating Cypress’s share quickly (not affecting Selenium’s dominant position). Assuming the samples are the same, within ~ three months,
Playwright: 12.5% -> 16%
Cypress: 14% -> 11%
Selenium: 64.5 -> 70%
Playwright is not in a position to challenge Selenium, which is actually extending its lead.
3. Selenium and Playwright are the two options for web test automation.
First of all, we can exclude the others (all added up), merely 3%.
Some say, how about the 3rd place, Cypress (11%)? But did you notice the alarming rate drop? The playwright’s gain is on Cypress’ loss while Selenium strengthens its lead by a huge margin. Also, cypress.io is dying .
In recent years, there have been too many failed JavaScript Test Automation Frameworks, such as PhantomJS, WebDriverIO, Protractor, TestCafe, …, etc.
Before another hype, there would be Selenium WebDriver and Playwright.
Which one will you choose for the 2024 Test Automation Work?
Between Selenium WebDriver and Playwright, which one will you choose for a new Web Test Automation?
The correct answer, of course, is and always has been Selenium WebDriver.
I know that some percentage will choose the wrong option: Playwright, as many have done so, PhantomJS, TestCafe, and Cypress.
Some might argue that “Zhimin, your liking Selenium does not mean it is right”. This statement is not wrong. However, we are talking about the facts here.
Selenium WebDriver was released in 2011.
There have been many web test automation frameworks over the last 12 years, claiming to be “Selenium Killer”. Those so-called “Selenium Killer” frameworks either were dead, such as Protractor, or dying, such as Cypress.
Selenium WebDriver still leads others by a huge margin, based on the above two surveys.
I like Selenium WebDriver and have been using it successfully for various projects for over 12 years. I have been providing proof, like this one, in my articles all along.
You might not disagree with my view or still fixed with Cypress or others, but you can’t deny the fact that Selenium WebDriver is well-suited for Web Test Automation.
“Facebook is released twice a day, and keeping up this pace is at the heart of our culture. With this release pace, automated testing with Selenium is crucial to making sure everything works before being released.” — DAMIEN SERENI, Engineering Director at Facebook, at Selenium 2013 conference.
This is by no means exclusive, but as test engineers, please consolidate your view with the results (like mine above).
Since 2011, to my knowledge, the only test automation framework demonstrating success, according to the Definition of End-to-End Test Automation Success, is Selenium WebDriver. Based on my observations, all other frameworks, including WebDriverIO, TestCafe, Protractor, Cypress, and Playwright, have consistently exhibited a 100% failure rate. (check out this Medium Boost article, A Tale of a Deceptive End-to-End Test Automation Engineer)
Now, on Playwright. I am not totally against Playwright. It is better than Cypress, which had all sorts of problems (see this article: Comments on the claims of “7 ways Cypress is different”. All False, Wrong or Lie). Objectively speaking, it is a joke that a web test automation ‘framework’ has severe limitations.
My testing tool, TestWise 7, even added basic support for Playwright.
My opinion and a good record of predictions are based on my 17+ years of hands-on test automation and continuous testing. Frankly, whether you take my advice or not, I don’t care, just like this former mentee ignored mine to avoid Protractor a few years ago.
My tools (TestWise IDE + BuildWise CT server) support both Selenium and Playwright.
For people who want to get on the right track to learn raw Selenium WebDriver? Check out the following articles: