E2E Test Automation Anti-Pattern: Developer-Focused Approach. Part 2: Scripting Language, Frameworks and Tools

Choosing the wrong scripting language, framework, and tools is common.

This is an article in the E2E Test Automation Anti-Pattern series.

Part 1: It is Black-Box Testing!
Part 2: Scripting Language, Frameworks and Tools
Part 3: Most developers Lack Proficiency in Test Automation
Part 4: Hard to Steer Away even after it is Proved Wrong

Continuing from Part 1: It is Black-Box Testing! I will explain how a wrong Developer-Focused Approach often leads to bad technical choices, such as scripting language, framework and tools.

Scripting Language

One clear sign of developers’ influence on end-to-end test automation is the test scripts’ language. Typically, Using the Coding Language for End-to-End Test Automation, which is often decided for the wrong reasons.

For End-to-End Testing (black-box), the test script language is 100% independent from the code!

Between 2005–2012, Facebook used PHPUnit for unit testing and Watir (Web Application Testing in Ruby) for browser-based E2E testing. Two different languages for two different types of testing (one is white-box; the other is black-box).

https://www.adweek.com/performance-marketing/how-facebook-automatically-tests-code-phpunit-watir/, date: 2010–07–04

“For browser-based testing of our Web code, we use the Watir framework. We have Watir tests covering a range of the site’s functionality, particularly focused on privacy — there are tons of “user X posts item Y and it should/shouldn’t be visible to user Z” tests at the browser level.” — Steven Grimm, the testing engineering lead at Facebook (2005–2012), responded on Quora.

One main reason for Watir’s success (prior to Selenium WebDriver) is its choice of language: Ruby. The Watir creators liked Ruby so much that they included it in the framework’s name. But Watir’s dependency on Internet Explorer’s OLE (which was fine at that time when IT completely dominated the browser market) decided it would be short-lived.

In 2011, Selenium WebDriver was released, and very soon, it dominated web test automation for many good reasons:

• based on W3C WebDriver Standard

• supported by all major browser vendors

• its consistent syntax

• feature complete and good support (free)

However, there is one side effect many do not realize. Selenium supports five official language bindings: Ruby, Java, C#, JavaScript and Python.

As I said earlier, the principle of ‘End-to-end software testing 100% independent from developers’ has been breached. When automated test scripts are in a programming language, developers often end up with a dominant role. Therefore, test scripts were set to use the coding language for most test automation attempts. Back 20 years ago, this was unthinkable! Somehow, it became the natural choice 😱 .