Web Automation Framework Trend ⇒ Selenium WebDriver
The only automation framework that is W3C Compliant
The above is a slide in my Web Test Automation with Selenium WebDriver training, created in 2018.
Table of Contents:
· Traditional Testing Tools
∘ Developer-focused Testing library/tools
∘ Free Open-Source Frameworks
· Show case: Facebook uses WebDriver for all its End-to-End Testing
· How about JavaScript test automation frameworks such as Cypress, TestCafe, and Puppeteer?Traditional Testing Tools
With the rise of Selenium WebDriver, there are generally three outcomes for those commercial testing tools (usually very expensive).
1. Change to support Selenium WebDriver
HP QTP → Micro Focus UFT → LeanFT (supports Selenium WebDriver)
HP was “the industry leader” of test automation before selling its testing tool business to Micro Focus in 2017. Micro Focus is now a Selenium Level Sponsor of Selenium WebDriver.
2. Try to link to Selenium WebDriver somehow
Ranorex, SmartBear, Applitools, .., etc. are the Selenium Level Sponsors of Selenium WebDriver.
While these vendors are still selling their proprietary testing tools, they are sponsoring Selenium WebDriver. The message shall be quite clear.
3. Phasing out the market
IBM Rational Functional Tester (RFT) was the chosen test automation tool by the Gartner Report 2013. However, I haven’t heard of RFT being mentioned in the last five years.
I used RFT before (~2013), and the tool was terrible and totally unnecessary. It did not run the test if your last opened IE window was on the second screen! Here is a related support ticket: “Rational Functional Tester support policy for dual screens” on IBM’s support site, still unresolved (last updated: 2018).
Frustrated, I created a Java project in RFT (effectively Eclipse with RFT plugin), added Selenium WebDriver Java libraries, got the test task completed on the same day. The test execution was far more stable and faster than RFT. Let’s face it: I was using a $11,000 per license tool as the free Eclipse Java IDE with Selenium WebDriver!
Developer-focused Testing library/tools
4. Deprecate its own and recommend Selenium WebDriver
A good example is Microsoft Coded UI Test. Microsoft deprecated Coded UI Test and recommended Selenium WebDriver in 2018. 👍
Free Open-Source Frameworks
5. Join Selenium WebDriver
Watir, Selenium, and Canoo WebTest are the three recommended test automation frameworks in the classic Agile Testing book (2009). Now, Canoo WebTest is gone (no updates since 2014). Watir becomes a syntax layer on top of the Selenium WebDriver.
Showcase: Facebook uses WebDriver for all its End-to-End Testing
Facebook names its UI Testing (the top tier of the Testing Pyramid) “WebDriver”.
The video clip from this brilliant presentation: “Continuous Integration at Facebook”.
And more proof, from a senior executive at Facebook.
“Facebook is released twice a day, and keeping up this pace is at the heart of our culture. With this release pace, automated testing with Selenium is crucial to making sure everything works before being released.” — DAMIEN SERENI, Engineering Director at Facebook, at Selenium 2013 conference.
How about JavaScript test automation frameworks such as Cypress, TestCafe, and Puppeteer?
While I created the slide in 2018, the hype of these JS frameworks was not around much. However, I witnessed that a fake test automation engineer ruined a software project by introducing Cypress at that time.
Some will defend these frameworks’ greatness. Sorry. In my work experience, every software project that has been using JavaScript-based test automation framework has failed badly and often ended with big embarrassment since 2018. I know some would say, “Cypress worked well in my last project?”. Well, after meeting and working with so many fake test automation engineers, I don’t believe it (not what they say, it is how they define good test automation, see below. Don’t take it personally, Microsoft, IBM and Garner all got it wrong. However, not many are as respectful as Microsoft to acknowledge the mistake and recommend Selenium WebDriver). I heard many .NET fans sing high for Microsoft Coded UI Test, and many test architects claimed ‘successful’ headless testing with PhantomJS. They all turned out to be plain lies. The vendors acknowledged that those products were not good/stable enough. They recommended WebDriver.
You don’t need others to acknowledge your success in test automation (and Continuous Testing), the best judges are the ones in your company. Test automation can be assessed objectively. A simple question is “Did You Push Software Updates into Production Yesterday?”. I do it for all my web apps if there are changes made on that day. Try to assess your project against the simple and objective AgileWay Continuous Testing Grading. My estimate is that 90% of software projects are on Level 0 or 1.
If you are interested in why Cypress (and other JS automation frameworks) always fail for real test automation, read my other two articles: