Affordable End-to-End (UI) Test Automation
End-to-End Test Automation is far more affordable than most people think.
In various software testing surveys over the past two decades, a common response has been: “We don’t do end-to-end test automation because we can’t afford it.” In my opinion, this is a weak excuse.
Twenty years ago, cost was indeed a barrier to adopting E2E test automation. For example, in 2013, an IBM Rational Functional Tester (RFT) license cost $11,000, while HP QuickTest Pro (QTP), considered an industry leader at the time, was even more expensive.
A common strategy among commercial test automation vendors was to lock users into proprietary test scripts and tightly integrate frameworks with their tools. That’s why terms like “QuickTest Pro tests” became common. However, this is no longer the case.
The once-dominant QTP changed hands and was rebranded as MicroFocus UFT, which is rarely mentioned today. IBM RFT has faded even further into obscurity. In 2018, Microsoft also deprecated its commercial test automation tool: Coded UI Test in favour of Selenium WebDriver (recently changed to Playwright).
“Why we are deprecating Coded UI Test? Open source UI testing tools such as Selenium and Appium have gained momentum in recent years, have a strong community backing and are now pretty much industry standards. Our Coded UI’s cross-browser testing solution was itself based on Selenium. Additionally, both Selenium and Appium work cross-platform and support multiple programming languages.” — Changes to Coded UI Test in Visual Studio 2019
Even Microsoft, as a test automation tool vendor, recommends the following separations:
Automation Framework
Test Syntax Framework
Tools
Programming Language
Automation Framework
An Automation framework drives the app.
Taking web test automation, the most common form of E2E testing, as an example, all leading web test automation frameworks are free and open-source.
It must be completely free, open-source and well-supported.
Test Syntax Framework
Test syntax frameworks, examples like RSpec, Mocha, and Cucumber, provide test syntax structure and assertions.
To my knowledge, all test syntax frameworks are free.
Tools
Tools are about productivity (where a framework decides capability).
Dedicated Test Automation IDE, such as JetBrains Aqua and AgileWay TestWise, both of which have somewhat free editions, and the commercial licensed version is $30/month.
You don’t have to use a dedicated test automation IDE, you can also use programming editors (e.g. free Visual Studio Code, TextMate, Sublime, etc.) and run using command line or built-in extensions.
Programming Language
A scripting programming language provides the flexibility required for E2E testing.
All common programming languages that are used in E2E test automation, to my knowledge, are completely free to use.
Synergy
There’s also an implicit cost in re-learning different stacks for different types of applications.
Appium and Selenium share WebDriver as the protocol. This means you get synergy with Mobile, Desktop and Web Testing.
Summary
From the above, do you agree that E2E test automation, putting aside skill-learning, is super affordable?
Related reading: