Web Test Automation Clarified. Part B: Test
Testing implies objectivity and independence, with the goal of uncovering issues that might make some people uncomfortable. A good test automation strategy cannot be developer-oriented.
This Web Test Automation Clarified series:
In Web Test Automation, “Test” typically refers to two common types of black-box testing:
Functional Testing: Ensures the application performs as expected and meets its requirements.
Performance/Load Testing: Evaluates how well the application handles varying levels of user load.
Both types of testing aim to validate the developer’s work from the end-user’s (or customer’s) perspective. That’s why many so-called developer-oriented test automation frameworks and tools are fundamentally flawed — for example, the now-defunct Protractor.js and the dying Cypress.
I am not suggesting that developers should avoid test automation tools, quite the opposite. My father has successfully run a one-person business with several highly acclaimed apps. However, when it came to E2E test automation, he always put on the hat of an independent tester, never compromising by using developer-oriented methods.
While most software professionals understand the concept of “black-box” testing, many companies make mistakes in their test automation strategy. For example, senior developers (such as software architects or principal software engineers) often dictate the framework or language for end-to-end (E2E) test automation. This approach is inherently flawed.
When it comes to web functional testing (the most common form), JavaScript has been steadily rising to dominance because of its popularity in coding. However, it is not an ideal language for end-to-end (E2E) test automation due to issues such as:
Bloated
node_modulesComplex async/await handling
Confusing promises
…
All of the above make sense only to developers and NOT to QA engineers at all. However, test automation engineers often were mandated to use JavaScript, by devleopers.
For more, check out the article, “Why JavaScript Is Not a Suitable Language for Real Web Test Automation?”
Many hyped JavaScript Web Test Automation frameworks failed.
Real (visible) web test automation started with Watir (Web Application Testing in Ruby) around 2004. Facebook embraced it for web testing.
“For browser-based testing of our Web code, we use the Watir framework. We have Watir tests covering a range of the site’s functionality, particularly focused on privacy — there are tons of “user X posts item Y and it should/shouldn’t be visible to user Z” tests at the browser level.” — Steven Grimm, the testing engineering lead at Facebook (2005–2012), responded on Quora.
As we all know, the Facebook app was developed using PHP. There was a clear separation in the frameworks used for testing: PHPUnit for white-box testing in PHP and Watir for black-box testing in Ruby.
Related reading