Correct a Common Misconception: “Using the Coding Language for End-to-End Test Automation”
A common wrong choice. End-to-End Test Automation Scripts are nothing to do with code and shall be in a scripting language.
Recently, one comment was posted on my LinkedIn post.
This is a common and very wrong view. This article will explain why.
Table of Contents:
· End-to-End Test Automation for the whole Team, not just Developers
· End-to-End Testing is Blackbox Testing
· My view on programming languages is unbiased
· End-to-End Test Automation, technically speaking, is talking to an Interface
· “But surely there are some benefits … ”
End-to-End Test Automation for the whole Team, not just Developers
I think every software professional understands the meaning of End-to-End test automation: verify the app as an end-user would, but using automated test scripts.
According to World Quality Report 2018–19, “The first time ever that `end-user satisfaction” is the top objective of quality assurance and software testing strategy”.
Clearly, and in practice, end-to-end testing is mostly not done by developers. I know some might argue, “manual testing, of course, by manual testers or business analysts, here, automation, i.e. automated test scripts”.
Hold that thought for the moment, let us be crystal clear about the fundamentals first.
automation is an approach to End-to-End Testing.
manual testers or business analysts who understand the objectives and the test steps, regardless of the form.
For example, a manual tester might write the below step in his test design (in a Word/Excel document).
Enter email ‘wise@tester.com’ in the user name text box on the login page.
The equivalent automated test step might be
driver.find_element(:username, "wise@tester.com")a refactored version (based on Maintainable Automated Test Design):
login_page = LoginPage.new(driver)
login_page.enter_username("wise@tester.com")Astute readers will get my message, automation script is just another form of using (a more correct term: driving) the app. Of course, automation scripts can be used/run by every team member and even the customers. As a matter of fact, from my experience, a business analyst are often the first to discover the power of my automation scripts and then use them daily (directly or via me). Then, more business analysts, manual testers, and lastly some developers use automation scripts to make their daily work much easier. For more, check out my article series: Benefits of Test Automation and Continuous Testing series: Executives, Managers, Business Analysts, Developers, Testers and Customers.
This shows that end-to-end testing is far beyond the scope of software engineers (including SDET).
End-to-End Testing is Blackbox Testing
All software testers understand these two terminologies: White Box and Black Box Testing. I don’t I need to explain the differences here.
Is End-to-End Testing White Box or Black Box Testing? Black-Box Testing, of course.
Now, another question, “Is End-to-End Test Automation White Box or Black Box Testing?” The answer is still Black Box Testing, obviously. Automation does not change the type of testing.
Given that only white-box testing needs to concern with the code, automated test scripts have absolutely nothing to do with the code, in any form. This so-called “automation coach” saying “very strange” to use a scripting language, such as Ruby, to test an app coded in Java/C#, is incorrect, at least. Acknowledge it is “ACCEPTABLE (I will explain, it is usually much better, shortly)” from a technical perspective, like a man.
My view on programming languages is unbiased
Some readers who don’t know me might think, “you are saying that, because you are a tester, and don’t know much programming, especially in Java/C#”.
I do know coding well, in fact, very well. Besides being a real test automation coach, I am also an international award-winning programmer. By the way, I started coding in Java in 1997, until I switched my day work from Programming to Test Automation in 2010 (for more challenging, more fun and better careers). (I am still coding on a daily basis for my own apps, while 70+% of my development efforts are spent on end-to-end test automation)
Programmers tend to be fixated/biased on one language they know. I am probably more objective than most, as My Selenium Recipes Book series covers all these five languages.
Commonly, an end-to-end test automation approach is decided by the tech lead in the team, who probably only knows one language. So, the coding language such as Java, C# or JavaScript is an easy choice. But, is that right? It is at least debatable (as explained earlier, for black-box testing).
Compared to many programmers and test automation engineers, it is fair to say that my view of programming languages is unbiased.
End-to-End Test Automation, technically speaking, is talking to an Interface
In white-box testing, unit/integration tests, of course, are the same language as the code they are testing.
Please note: we don’t call unit tests: “unit test scripts”.
But for end-to-end testing, our test scripts, technically speaking, are driving the app via public APIs or standards. Let me illustrate it with web testing, the common form of end-to-end testing.
Technically speaking, we write automated scripts to drive the app in a browser based on W3C-defined web technologies such as HTML and CSS.
“Since 2006, I have implemented successful web test automation for many software apps, which were coded in Java, C#, JavaScript, Ruby, PHP and Python.” — Zhimin Zhan
FACT: As we all know, Facebook is developed in PHP. Facebook used Watir (Web application testing in Ruby) until 2011 (then moved to WebDriver). Check out this article, Facebook, Watir And Testing.
The same principle applies to end-to-end API testing as well. Automated test scripts (which can be written in any scripting language) invoke the API based on HTTP, SOAP, REST, GraphQL, …, etc.
“But surely there are some benefits … “
My answer is “A tiny little maybe benefit, but with significantly greater potential for harm.”
I learned this a hard way in 2006. As the tech lead, I asked pairs of manual tester and developer to work together on test automation. The result was no good, in a particular case, one manual tester wanted to quit because one developer complicated things too much (as a result, test scripts were much worse) and made this poor tester inferior. So, I made a call, “when working on testing, if there is any disagreement, alwasy listen to the manual tester. If developers unable to meet the tester’s demand, come to me”. This approach worked very well. I personally benefited a lot too, learned more thinking the test design from a non-programmer (i.e. user)’s perspective.
To be fair, I will refute each of the so-called ‘advantages’ listed in this article, Automation 101: Should The Programming Languages of Your Software and Autotests Match.
The so-called four “advantages of using the code language for test automation scripts” are all wrong.
Keep reading with a 7-day free trial
Subscribe to The Agile Way to keep reading this post and get 7 days of free access to the full post archives.