Automated Test Scripts Shall be in the Syntax of a Scripting Language, Naturally!
Simple and yet powerful scripting is the way to succeed test automation.
Occasionally at talks, I got questions like the following:
You build a testing IDE (TestWise) from scratch, why not an Eclipse plugin?
Besides Ruby, what’s your view of test scripts in other languages, such as Java or C#?
The first question is easy to answer: automated testers don’t need the complexity of Eclipse, a programming IDE. They just need a simple tool that is easy to view/edit/run test scripts. (Debugging automated test scripts is completely different from debugging code, in terms of tool use. No step-over, step-into, breakpoints, …, etc, which are not efficient in the context of automated tests. I will write a separate article on this topic)
IBM’s Rational Functional Tester (RFT) was in fact an Eclipse plugin, at least was the case in 2013. I was able to create raw Selenium WebDriver tests in RFT (as a Java IDE) , easier than using RFT’s proprietary syntax. Now RFT, the former recommended testing automation tool by Gartner, is hardly seen.
Also, another important human factor, if testers are using the same tool as developers, it puts programmers in a dominant position. From my experience, it is no good. The reason: you simply shall not let an auditor verify A’s work, using a tool at A’s choice.
The same argument (sharing the same tool/language with programmers) goes for the second question (scripting language) as well. If the choice of test scripting language was made purely for the benefit of programmers, then I disagree.
At the TIST2011 conference, two test engineers from separate companies told me a similar story: despite a lot of efforts programmers spent providing various ways (such as page-objects, environment selection, data-driven,… etc) to help non-tech testers get engaged, the outcome was not ideal. That’s because the whole approach was programmer-centric.
The audience of automated E2E test scripts is the whole team. The fact is that testers, business analysts, and customers generally do not possess strong programming skills, and this changes the whole paradigm. It is not hard to imagine that an 85MB download of Eclipse (a Java IDE), configuration of library path, and JDK path will do to a non-tech-savvy tester as the first chapter of the automated testing orientation program. And you might hear a screaming minutes later: “What does ‘compiler error’ mean?”.
Tip: when introducing UI test automation to the team members, the tech lead (or manager) shall forget his/her favourite programming language, rather, choose the one that all team members find easy to understand.
I went through this process in 2007, and I have programmed Java for about 10 years (yes, I started with JDK 1.0.2). I found Java was too complicated for the team, so I switched to Ruby (Watir). The test automation was a big success. Business analysts, manual testers and the architect all utilized test automation on a daily basis.
Three-Minute Rule
Social media shortens our attention span, most will agree (and accept, 😞) now. To successfully engage customers/business analysts/manual testers/architects/managers with test automation (by the way, a core DevOps process), from my experience, three minutes is the maximum limit, after that, they will start losing interest. They can come up with many reasons to avoid it, the most often used one is: “I don’t have time for this”.
So, an experienced test automation coach needs to accomplish all the following tasks within 3 minutes:
Install the testing tool
(including runtime such as Ruby, and test libraries, e.g. Selenium)Get test scripts on a business analyst’s machine
(using Git)Open an automated test to help the business analyst understand to a certain degree
Run an automated, view test execution and the test result
(the execution time does not count for the 3-minute)Let the business analyst run one automated test
I could accomplish the mission in mostly around 3 minutes on various client projects, but definitely not with Java and C#.
Test scripts shall be in scripting languages such as Ruby and Python, not compiled ones such as Java or C#. Why, because it is called ‘test scripts’ for a reason.
A good functional test script language should have the following characteristics:
easy to learn,
no compilation is required,
non-proprietary,
extensible syntax to use domain-specific language,
and ideally, one operation per line
If you just want a simple answer, go for Ruby. On choosing a programming language for automated tests, I am probably more objective than most engineers, as My Selenium Recipes Book series covers these five languages: Ruby, Java, Python, JavaScript and C#.
Some might suggest JavaScript, No, avoid it. JavaScript has a poor history in test automation:
Selenium v1, thankfully, was replaced by Selenium WebDriver in 2011. It was a good initiative, though.
PhantomJS, its failure exposed many fake test automation engineers and tech leads who lied about practising test automation.
Check out my article: “Headless Browser Testing Clarified”.Protractor, a very popular (thanks to Angular.JS) test automation framework, has been deprecated.
Check out my article: “Protractor, another Automation Framework I rightly avoided, is being Deprecated”
For more, check out my article: Why JavaScript Is Not a Suitable Language for Real Web Test Automation?