Protractor.js is dead, What did we learn?

Avoid hypes, do real hands-on end-to-end test automation daily. Raw Selenium WebDriver + RSpec is better.

My recent article “ ‘Cucumber is Dying’, What did we learn?” turned out to be a hit, in terms of reads within my 300+ articles. I have been warning people about “Gherkin” in test automation at software testing conferences, in my books, during my coaching, …, etc. There were disagreements and doubts about my judgment. This time, no single argument/disagreement, as my prediction (since 2013) about Cucumber is correct.

Today, I will write about another correct prediction on Protractor.js. Some of you might have read my articles on this topic:

• Protractor, another Automation Framework I rightly avoided, is being Deprecated

• Story: A Former Mentee Failed Test Automation with Protractor

A brief introduction to Protractor.js for testers who are not aware of it. Protractor.js was quite a big hype, not long ago. In fact, its deprecation was only announced in April 2021.

Protractor.js was created in 2013, i.e. it died at the age of eight.

On the paper, Protractor.js had all the reasons to be successful:

• “Protractor wraps selenium-webdriver”

• It was created by the Angular team (at Google), and Angular was the most popular web framework during 2013–2018.

• Using JavaScript, as it was (still is) the trendy thing (because of the huge popularity of Angular)

Yet, it failed.

I will list my reasons why Protractor.js failed.

Table of Contents:
· 1. So-called “enhanced syntax over Selenium” is actually making it much worse. 
· 2. Using the same language in automated test scripts as the coding one is a bad idea. 
· 3. JavaScript is not a good language for end-to-end test automation.
· 4. A test automation framework/tool from a high-profile company does not mean it will be successful.
· How did I predict its failure?

1. So-called “enhanced syntax over Selenium” is actually making it much worse.

A big selling point of Protractor.js is “simpler and enhanced test syntax over Selenium WebDriver”.

“Everything should be made as simple as possible, but no simpler.” — Albert Einstein

I disagree with my first sight of a sample Protractor script on its tutorial site.

// spec.js
describe('Protractor Demo App', function() {
  // setup of browser in config 
  it('should add one and two', function() {
    browser.get('http://juliemr.github.io/protractor-demo/');
    element(by.model('first')).sendKeys(1);
    element(by.model('second')).sendKeys(2);
    element(by.id('gobutton')).click();
    expect(element(by.binding('latest')) .getText()).
        toEqual('5'); // This is wrong!
  });
});

Let’s see an equivalent raw Selenium version (in RSpec):

describe('Protractor Demo App', function() {
  # set up browser in the helper
  it('should add one and two', function() {
    driver.get('http://juliemr.github.io/protractor-demo/');
    driver.find_element_by_xpath("//input[@ng-model='first']").send_keys(1)
    driver.find_element_by_xpath("//input[@ng-model='second']").send_keys(2)
    driver.find_element_by_id('gobutton').click;
    expect(driver_find_elements_by_class("ng-binding").last.text).to eq('3')
  });
});

Back in the “Protractor.js live” days, some people argued with me, saying the first version was an improvement. I explained “It is wrong for the following reasons”:

• Assuming the target web app is Angular, e.g. @ng-model
  Remember, end-to-end testing is for all web apps. How would Protractor testers think after it is announced ‘deprecated’? A Selenium Webriver tester can test any web app.

• Not clear, such as .binding('latest') 
  A tester might need to process on looking at this kind of test statement. As for the selenium version, testers who have a basic understanding of HTML, can get the idea unambiguously.

• Losing “Action” tone and rhythm.
  In the Selenium version, get , find , expect actions emulate end-users' operations, it is more logical and easy to follow. Protractor broke that rhythm, which will lead to problems in maintaining a large suite of tests.

Some might argue that “Protractor.js version is more concise, quicker typing ”. People are saying that lack the concept of “Snippets” in IDEs/Programming editors. For example, in TestWise, I type dfex followed by a Tab, which will be expanded to driver.find_element_by_xpath("...", "...") .

2. Using the same language in automated test scripts as the coding one is a bad idea.

A common mistake in test automation is using the coding language, typically Java, C#, and JavaScript, for end-to-end test automation scripts. There is no such rule, because end-to-end test automation is about the perspective of end-users, nothing to do with the code behind the app. The failure of Protractor.js ( coding with Angular.js) is proof.

For more, check out my article, Correct a Common Misconception: “Using the Coding Language for End-to-End Test Automation”.

3. JavaScript is not a good language for end-to-end test automation.

In recent years, JavaScript has become the mainstream coding language. However, in my opinion, JavaScript is bad for end-to-end test automation, for example, async does not much sense in the testing context.

I wouldn’t say any end-to-end test automation with JavaScript definitely fails, but every single test automation attempt (with JS) I witnessed failed. Protractor.js (in JavaScript) may serve as a side-proof, after all, it has been widely tried (and failed). Not long before Protractor, the big embarrassment in test automation is the failure of Phantom.js (another one in JavaScript).

For more, check out my articles:

• Why JavaScript Is Not a Suitable Language for Real Web Test Automation?

• Too Many Failed JavaScript Test Automation Frameworks!

4. A test automation framework/tool from a high-profile company does not mean it will be successful.

The Angular team (at Google) had the utmost reputation during its peak time. People have reasons (kind of real) to believe that its test automation framework will be successful, just like its core product, Angular.

In reality, big software companies had a very poor record on test automation products:

• HP QTP (being acquired by Micro Focus, now rarely heard)

• IBM Rational Function Tester (RFT)
  I haven’t heard of it since my review at a government job, in 2013.

• Microsoft Coded UI Test, deprecated in 2018.

First of all, real test automation engineers are extremely rare, even at Google and Microsoft.

“In my experience, great developers do not always make great testers, but great testers (who also have strong design skills) can make great developers. It’s a mindset and a passion. … They are gold”.
- Google VP Patrick Copeland, in an interview (2010)

“95% of the time, 95% of test engineers will write bad GUI automation just because it’s a very difficult thing to do correctly”.
- this interview from Microsoft Test Guru Alan Page (2015)

“Testing is harder than developing. If you want to have good testing you need to put your best people in testing.”
- Gerald Weinberg, in a podcast (2018)

Also, check out this story, “The Software Revolution Behind LinkedIn’s Gushing Profits”.

Yes, please notice, it is “Lure”, not “Hire”. LinkedIn should know how to hire people, right? 😊

Some might disagree, surely, there are at least a few real test automation engineers at Google. Yes, as Google Senior VP said, “they are gold”. What I said the above are facts, because I didn’t work at Google. The following is my assumption. Real test automation engineers would instantly know raw Selenium WebDriver is the best, wouldn’t bother joining this Angular/Protractor team. By the way, Simon Stewart, the creator of Selenium WebDriver, worked at Google during that period.

I often say a software engineer will be extremely lucky to meet on real test automation engineer in his/her career. I was very fortunate to meet one in 2005 and learned test automation and Continous Testing from him faithfully and hungrily. Check out My Test Automation Journey.

How did I predict its failure?