False ‘Selenium WebDriver Cons’
Expose the dirty tricks of fake “automated testers”, who want to hide their incompetence, and justify ‘juggling frameworks’, not doing real automated testing.
This article is one of the “Be aware of Fake Test Automation/DevOps Engineers” series.
Many test automation framework reviews listed the Pros and Cons of Selenium WebDriver, like the below ones:
Though there are generally more positives on Selenium, the cons listed are so untrue, which inevitably exposed these people’s ignorance to test automation. I am not belittling the authors here, if you had read “How Google Tests Software” book and listened to Gerald Weinberg’s “Testing is Harder than Development”, you would understand: real test automation engineers are so valuable and so rare, even the top companies like Google and Facebook found it hard to acquire. In a word, these authors made false comments on Selenium simply because they are not good test automation engineers (again, very very rare).
In this article, I will explain why those false “Selenium Cons” are wrong, in a simple, easy-to-understand and objective way.
Table of Contents:
∘ 'Steep learning curve'
∘ 'No tech support'
∘ 'No reporting capabilities'
∘ 'Not an All-in-One solution' — requires 3rd party tool bindings
∘ 'High Initial Cost'
∘ 'Slow Test Development' due to the script-based Approach
∘ 'Low readability of Test Scripts'
∘ Why do some IT professionals come up with those false claims against Selenium WebDriver? ‘Steep learning curve’
Untrue. Please check out my other article showing how easy to learn Selenium WebDriver in minutes, step by step. Here I will just highlight the key facts:
Selenium WebDriver dominates web testing in a relatively short time. Please remember, there was no sales pitch and no commissions.
Selenium WebDriver is a W3C standard. As per other W3C standards, such as HTML, CSS, and XML, they are all quite easier to learn. I worked at W3C Australian Office before, becoming a W3C standard is not easy, it will take many rounds of reviews by the experts. From the records, do you trust W3C experts or some guys in a vendor company?
Selenium WebDriver syntax follows a simple and intuitive pattern: find a web element and then drive it. This makes it extremely easy to learn. In my one-day West Test Automation with Selenium WebDriver training, I spent only a few minutes on Selenium syntax, and the participants grasped it quickly with my exercises. (the main topic of my training is to develop selenium tests efficiently and refactor them to be maintainable).
My daughter learned to write Selenium tests at the age of 12. Here I mean learning, for doing useful work, and she did. Has she mastered test automation? No, far from that, mastering any skills takes time and experience. (she had excuses for doing school homework, 😃) But she did far better test automation (judged by the results) than most of the test automation engineers I have seen, one reason is that she is using raw Selenium WebDriver with RSpec, follows the rules of test automation.
People say “Selenium is hard to learn” because they mix it with other factors, such as
not understanding test automation (it is easier to fake it, reality: few have seen real test automation)
mixed with programming (used programming practices wrongly for test automation)
bad test design
lack of hands-on experience (unable to do complex scenarios)
lack of general knowledge, such as XPath
trying to create an ‘own framework’ on top of Selenium, which always leads to failures.
fixated with the wrong programmer-centric tools. Trust me, developing/debugging automated test scripts are different.
The above are more human issues.
I often saw frustrations when new-to-automation testers’ face when they were trying to set up their machines for developing/running automated tests: Java, Eclipse, Maven, …, etc. That’s Selenium with Java. It does not have to be that hard. If using Selenium + RSpec (Ruby), setting it up using TestWise IDE is very easy, usually takes less than 1 minute.
‘No tech support’
Absurd. Selenium WebDriver is a W3C standard, just like HTML and XML. When a project considers developing a web app, no tech support from W3C surely won’t be the reason to reject HTML and use Adobe Flash instead (which is dead, by the way).
In terms of general support, Selenium WebDriver has been outstanding. The releases are quite stable, and chromedriver releases are always up to date with the latest Chrome browser.
You get better and more support for open-source frameworks, instead of from the only official vendor, such as:
Stack Overflow or Google search
Engage in external coaches/consultants on training and mentoring
Because it is open, you can be creative in working out solutions
On the contrary, the tech support from large vendors, from my experience, was usually quite poor: slow and unhelpful.
‘No reporting capabilities’
Irrational. Selenium WebDriver is an automation framework, not a test syntax framework, such as JUnit, RSpec, and Cucumber. We cannot accuse a leopard of not flying fast.
In fact, when using Selenium with a syntax framework, you can get lots of reporting capabilities. I have reviewed many proprietary tools such as Ranorex and RFT. Comparing those vendor-specific reports against the ones I got from Selenium + RSpec, is like comparing “brick” mobile phones of the 90s against the new iPhone.
‘Not an All-in-One solution’ — requires 3rd party tool bindings
Nonsensical. People saying this does not understand the nature of the software business. Yes, Selenium requires a common language runtime such as Ruby, Python, or Java. If this is considered a Con, does that mean you will buy a tool with a newly-created proprietary syntax ‘Z’, its language and implementation is totally hidden from you?
‘High Initial Cost’
Illogical. Selenium WebDriver is 100% free (as in both freedom and pricing) and open-source. For all my clients, after my one-day training, the team is able to work on real tests, without paying a cent. And there is no need to worry about licensing issues.
‘Slow Test Development’ due to the script-based Approach
Wrong, quite the opposite. The script-based approach is always better for real work. Unless what you want is a simple demo and the test scripts will be thrown away, then record-n-playback might be an option for fake test automation engineers.
Don’t forget, non-script-based approaches such as (HP QTP) have existed for decades, and HP sold QTP out to Micro Focus a few years ago. Micro Focus is now a selenium-level sponsor of Selenium WebDriver, listed on Selenium’s home page.
‘Low readability of Test Scripts’
False. I have seen many test scripts in bad testing frameworks/tools, such as UFT, RFT, Ranorex, Sahi, Protractor, Cypress and Playwright. The scripts are hard to read. Readability can be somehow subjective. Please remember, the audience of automated functional test scripts are not limited to programmers. A JavaScript programmer might believe one test script is readable, but a business analyst or manual tester probably disagrees.
Take a simple cypress (which some claimed good test script syntax) test as an example.
describe('My First Test', () => {
it('finds the content "type"', () => {
cy.visit('https://example.cypress.io')
cy.contains('type')
})
})A manual tester or BA might think it is bad. They really don’t like () => { and }) syntax, which a JavaScript programmer takes for granted. The above is just a super-simple example. With a long and complex test, the readability may degrade dramatically. Worse still, some JS programmers might use Promise in the script, like the one below:
it('waits for promises to resolve', () => {
let waited = false function waitOneSecond() {
// return a promise that resolves after 1 second
return new Cypress.Promise((resolve, reject) => {
setTimeout(() => {
// set waited to true
waited = true // resolve with 'foo' string
resolve('foo')
}, 1000)
})
} cy.wrap(null).then(() => {
// return a promise to cy.then() that
// is awaited until it resolves
return waitOneSecond().then((str) => {
expect(str).to.eq('foo')
expect(waited).to.be.true
})
})
})What a bad functional test script! This is an example from the official Cypress Documation. In other words, I don’t think Cypress knows its test syntax is bad for functional testing. (Check out my article: Why Cypress Sucks for Real Test Automation?)
How about Selenium WebDriver? If the Page Object Model is used well ( a Selenium best practice on the official Selenium doc), the test scripts can be very readable, more importantly, easy to maintain.
it "Sign up business validation" do
try_for(2) { driver.find_element(:id, "sign-up-link").click }
new_business_page = NewBusinessPage.new(browser)
new_business_page.enter_name("")
new_business_page.enter_email("verynew@biz.com")
new_business_page.enter_password("test02")
new_business_page.click_sign_up
try_for(2){ expect(page_text).to include("Name can't be blank") }
# TODO more validation steps ...
endOne test step line for one user operation. This makes it not only easier to read but also much easier to debug: checking with the failed test step based on the line numbers in the error trace. I have 495 and 611 selenium tests like the above for my two web apps WhenWise (3 years) and ClinicWise (7 years). These regression tests were running in our CT Server (BuildWise) frequently to enable real DevOps. We are capable of pushing software updates on the production multiple times a day if necessary.
If I go with any other framework, frankly, I wouldn’t be able to achieve that (I did most of the app development/testing in my spare time). In fact, I have never seen a single software project that is able to maintain 50 automated E2E UI tests (running reliably and frequently in CI/CD) in RFT, Ranorex, UFT, Sahi, Cypress and Playwright. The exception is raw Selenium WebDriver (and old Watir).
Why do some IT professionals come up with those false claims against Selenium WebDriver?
The answer is incompetence. Some Tech leads need reasons not to use Selenium. If they chose a vendor, they could blame the vendor when the test automation attempt failed. If it is Selenium WebDriver, they are unable to blame others but themselves. Their fake knowledge of test automation, CI/CD, Agile, and DevOps will be exposed.
So, some may wonder, there must be cons with Selenium. For example, a common complaint to Selenium is that it cannot run a specific test step against the current browser. The answer is “Yes, it can”. In fact, TestWise, a functional testing IDE, has the feature “Run Selected Test Steps Against Current Browser” for years.
Also, the upcoming Selenium 4 will support Chrome Debugging Protocol (CDP), a protocol to enable a debugger inside Chromium-based browsers. Cypress and Puppeteer are built based on CDP.
CDP is designed as a debugging protocol, it can change with no notice between versions. That’s why Puppeteer and Cypress are tied to specific versions of browsers, and that causes a dilemma for you as a test author.
Despite this, using the CDP opens up a host of possibilities, and that’s why we’ve added support for it in Selenium 4.
we’ve also gone further that just adding basic support.
-Simon Stewart, creator of Selenium WebDriver, 2020–11–16
In my view, there is only one Con for Selenium: supporting non-scripting languages such as Java and C#. Test scripts are better (and should) be in a scripting language such as Ruby and Python. Just in Brisbane (not a big city on a global scale) alone, I have seen many embarrassing test automation failures (cost businesses millions) caused by Java tech leads. Some tech leads shifted the blame to Selenium WebDriver with cheap made-up reasons, such as hard & unreliable, to cover their incompetence 😞. Surprisingly, they got away with it. A wise executive won’t be fooled by this: Facebook, Google and even Microsoft recommend Selenium WebDriver.
Disclaimer: I programmed Java & JavaScript since 1997, I still use Java sometimes for app development. But for test automation, I am much happier with Ruby.