How Could New Automation Frameworks be Worse than Raw Selenium WebDriver?

Few people do real UI test automation, many fake it by toggling frameworks or chasing hypes.

This article title might upset some people. Oh well, we are testers and shall speak with results, objective results like this one. My Selenium E2E test suites have enabled me to do daily production releases for all my web apps, since 2012. Yes, E2E UI test automation is so useful that it makes my Micro-ISV at least 50X more productive than my competitors. The secret: I spend ~70% of SDLC Effort on Software Testing (Real E2E UI Test Automation). Once a software team masters E2E UI Test automation, as this LinkedIn report by Wired, everything is much easier. While it is not the ‘silver bullet’, I believe it comes remarkably close. I heard one elite engineer refer to “Continuous E2E UI Testing’ as the “Holy Grail of Software Development”, and I wholeheartedly agree.
It is a pity that most IT executives, managers, and software engineers do not understand the power of Continuous E2E UI Testing. The reason: they never met a real test automation engineer or witnessed real Agile enabled by daily Automated E2E UI regression testing. To give you an idea of how rare real senior test automation engineers are, even LinkedIn needed to lure one.

One attendant, who might have been involved in test automation before, asked this question (the article title) after attending my one-day web test automation training. He was convinced, by himself writing real tests within the one-day workshop, that raw Selenium WebDriver is so good, and without any constraints or limitations at all. Why are there still so many hypes of so-called new test automation frameworks? Most failed, by the way. Check out my article: Selenium vs X, Y, Z, …, all these years.

It is not easy to explain this insanity. The article, A Tale of a Deceptive End-to-End Test Automation Engineer, revolves around three points:

• Real Test Automation Engineers are extremely rare, even in the software giants like Google, LinkedIn and Microsoft.

• Ironically, most software engineers think “UI Test Automation is easy”, but have nothing to show for after a demonstration.

• Therefore, faking UI test automation is very common.

Selenium WebDriver ticks all the boxes for Web Test Automation, such as

• W3C standard
  (this is really important, W3C defines web technologies)

• Supported by all major browser vendors

• free and open-source

• well-supported and has a large user community

It is proven.

“Facebook is released twice a day, and keeping up this pace is at the heart of our culture. With this release pace, automated testing with Selenium is crucial to making sure everything works before being released.” — DAMIEN SERENI, Engineering Director at Facebook, at Selenium 2013 conference.

Some senior engineers would say (I heard it a few times), “That’s Facebook, the software giant. They can get Selenium WebDriver working, but that does not mean others can do too”. The view is wrong! Check out my showcase.