Why do Test Managers/Architects Avoid Selenium WebDriver, the Best Web Test Automation Framework?

Human weakness.

If I ask you, “A car is well-designed as Poshcue, easy to maintain as Toyota, safe as Volvo, certified by the top government authority in every country, and accelerates as Ferrari. Also, it is super easy to drive. Its price is $0. Will you buy it?” The answer, of course, is YES. In doubt? I recommend reading the following articles (technical factors) after this one (human factors):

• Why Selenium WebDriver is So Easy to Learn?

• Correct a Common Misconception: “Setting up Selenium is Complex and Time-Consuming” with a benchmark test.

• Playwright vs Selenium Speed Comparison

• Cypress vs Selenium WebDriver Comparison by Example

• Page Object Model is universally applicable in web test automation

• Set up, Develop Automated UI tests and Run them in a CT server on your First day at work (by my daughter)

However, in the software testing industry, crazy things happen. The senior test engineer/management often choose a much inferior test automation framework/tool, with all sorts of problems and limitation, and also a high price tag.

Table of Contents:
· Web Test Automation Clarified
· Selenium WebDriver is the web test automation framework, period.
· The subtle reason why architects/managers did choose a bad vendor testing tool: “the ability to blame”

Web Test Automation Clarified

Sometimes, it puzzles me how software companies could choose other web test automation solutions, such as QTP, Ranorex, TestComplete and Cypress? Compared to the free Selenium WebDriver, these are much worse (Playwright is better than Cypress, still not good) for real test automation (a quick self-assessment, do you cover 90+% user stories with automated end-to-end vis UI tests? Do you run them daily? Are they still valid?).

Some might wonder, “much worse, can you quantify?”. Yes, I have been working in test automation since 2007. Each of every test automation attempt I witnessed with the above test automation framework failed, with no exceptions. To be specific, no single one could reach Level 2 of AgileWay Continuous Testing Grading, in fact, most are just Level 0 or 1. In case you ask, I often achieved Level 3 or 4, below is one Level 4 example: 559 raw Selenium + RSpec end-to-end (via UI, user story level) tests running daily (if changes are made) for my WhenWise app.

A recent run of the WhenWise regression suite, consisting of 559 raw Selenium WebDriver tests, running in a BuildWise CT Server. By the way, this process enables Daily Production Releases (if changes) for all my apps.

By the way, web testing has been the most common type of software testing over the last 25 years. There is no excuse for a senior engineer to fail, yet, >99% web test automation failed badly. Check out the Definition of End-to-End Test Automation Success.

“95% of the time, 95% of test engineers will write bad GUI automation just because it’s a very difficult thing to do correctly”.
- this interview from Microsoft Test Guru Alan Page (2015), author of “How We Test Software at Microsoft”

First, let’s start with the purpose of web test automation: using automated test scripts to test a website as a user would. Four keywords:

1. Website
   A series of webpages using web technologies, e.g. HTML and CSS, defined by W3C. Because of that, if one test automation engineer has mastered web test automation, he/she shall always be successful technically (politically, as you know, no one knows)

2. Automated
   using a scripting language to drive the target web app in the browser, namely, Chrome.
   Quite often, people choose Java/C# or JavaScript for scripting automated end-to-end tests. This is wrong, it makes things unnecessarily difficult. Check out,
   — Automated Test Scripts Shall be in the Syntax of a Scripting Language, Naturally!
   — Why JavaScript Is Not a Suitable Language for Real Web Test Automation?

3. Test
   Testing means objective assessment. We testers shall back up any saying or conclusion with the test results. Test Automation means no grey area: just 0 or 1. (don’t use low excuses such as “app changed” or “test data changed”, they always do)

4. User
   Real end-to-end testing. Write automated test scripts that a business analyst, manual tester and even a customer can understand and run the test scripts.
   Programmers often need mentoring to master this.

Now, with the above information (anyone can reason out if they want to), have you met a single real test automation engineer (Level 3+ of AgileWay Continuous Testing Grading)? Most likely, none.

If your answer is more than 0, check out the following articles:

• Four Screening Questions to Filter Out Fake Test Automation Engineers in Interviews

• One Simple Test Automation Scenario Interview Question that Most Candidates Failed

If you don’t bother reading the above, I can tell you, with a real test automation engineer, the software shall be technically deployable at least every week.

With the above established, this simply means people, who decide to purchase a software test automation tool/solution, do not understand test automation at all. Let me share an incredible story.

Once, I started a new role, as usual, I completed a few core tests on the first day, using raw Selenium WebDriver. A colleague was deeply impressed, he told me that test automation had been attempted not long ago, but then people quickly forgot about it. With his help, I managed to find a few automated tests on BitBucket. It was Cypress.

Of course, those Cypress tests failed when I tried running a few. This is not a surprise to me. What shocked me is that our target website used frames extensively, but the Cypress website clearly stated: “not supporting frames”. (By the way, that page was taken off, check out this article for more: Why Cypress Sucks for Real Test Automation? (Part 2: Limitations)). What the heck!

Selenium WebDriver is the web test automation framework, period.

Web technologies and Browsers play important roles in web test automation. There is only one test automation framework that is officially authorized by W3C (the governing body behind all web technologies) and all major browser vendors, e.g. Google, Apple, Microsoft and Apple. That framework is Selenium WebDriver.

So, for me, it is crazy not to use Selenium WebDriver for web test automation. Considering this:

1. If later W3C updates one web standard, e.g. HTML7 or CSS5, your existing test scripts probably won’t work. Selenium tests will be fine.

I have been writing automated tests exactly the same way since 2011. Yes, 12 years, only very minor changes when upgrading to v3 and v4. Why? Selneium WebDriver is based on W3C’s webdriver standard. W3C has a proven record for maintaining backward compatibility. (HTML is a good example)

2. Morden web apps are more complex and dynamic, i.e., depending on the browser.

Browser is a very complex software, even Microsoft gave it up to join Chromium. We know, web technologies are still evolving. Sooner or later, your test scripts based on a proprietary framework will fail on the new browser. When you report to Google’s Chrome team, they most likely will ignore your feedback. They would verify with Selenium, though. (by the way, WebDriver is largely developed or contributed by Google engineers)

I often see in a comment line in some test script: “This test script only works on Firefox vXX”. How sad!
My test scripts always run againsst the latest browser version. I use raw Selenium WebDriver.

The subtle reason why architects/managers did choose a bad vendor testing tool: “the ability to blame”