Why Raw Selenium Syntax is better than Cypress and Playwright? Part 2: The Audience Matters

Whether test script syntax is better or not depends on the audience, which many fake automated testers are not clear about.

To real test automation engineers, raw Selenium WebDriver syntax is clearly much better than Cypress, Playwright, and all others (including claimed ‘a better Selenium’ Protractor.js, which had been dead for a few years). However, there are so many misconceptions and lies about Selenium and marketing for so-called new frameworks, I have to write a series of articles to clear this up.

Update 2023–07–05: Part 3: Selenium is Accurate, Intuitive, Consistent, and Geniously Designed

Recap from Part 1:

• Selenium WebDriver is based on W3C WebDriver standard
  Many junior testers (not by age) underestimate this importance. Just think of anything in your life, train track, door size, tyres …, etc. W3C defined all the technologies used in websites: our testing target.

• The appalling record of using JavaScript in the history of test automation
  Developers, don’t get offended. I am talking about test automation here, not coding. Testers, it is in your code of conduct to accept the facts, no matter how uncomfortable.

• The successful examples of using Selenium WebDriver
  Selenium WebDriver really works well, very well. In fact, besides Watir (now just a syntax tier on top of Selenium), I only witnessed successful test automation in raw Selenium WebDriver since 2005. All others, including hyped Phantom.js, Protractor.js, TestCafe, and Cypress were all complete failures. The new favourite JS automation framework Playwright is indeed better than Cypress. Anyway, based on my observations, none of the attempts using Playwright were successful either. One reason: Playwright syntax is still not good, compared to raw Selenium WebDriver.

In this Part 2, I will get to the test script syntax. Let me start with a basic question.

“Who is the audience (i.e. users) of End-to-End Test Scripts?”

Once understand the users (for test scripts), we can start talking about which are good or bad for them, right?

Table of Contents:
· The audience of End-to-End Test Scripts is the whole team
· Test Script Tiers
∘ Both Tiers (DSL And Statements) Shall be in the same scripting language
∘ Using Cucumber, SpecFlow and other Gherkin syntax in DSL tier is wrong.
· The so-called ‘syntax enhancement’ of test statements for the top tier is very wrong
∘ 1. Ugliness of using JavaScript in the top tier.
∘ 2. Clearly, JavaScript is a bad language for end-to-end test automation. But why it is chosen?
· FAQ

The audience of End-to-End Test Scripts is the whole team

One major reason that people commonly make a bad choice of test automation framework or language: they take it from a developer’s perspective. This is clearly wrong, once pointed out, we testers are to verify their work.

“Our app is coded in JavaScript, so we choose JavaScript for end-to-end test scripts”, this is a common comment I heard when on a rescue mission. My next question is: “Does it have to be JavaScript though?”

No, of course, it is not. Putting “our code is JavaScript” argument aside, end-to-end test scripts (manual or automated), in one way, are the instructions to use (then verify) the app, having nothing to do with the coding language behind it.

These end-to-end test scripts speak in an active tone as an end-user. For example,

1. Open the ‘https://whenwise.com’ in a Chrome browser

2. Click the ‘SIGN IN’ button (technically it is a link, but looks like a button)

3. Enter email driving@biz.com

4. Enter password test01

5. Click the “SIGN IN” button

6. Verify the text “You have signed in successfully” present.

Let me show how I typically write this test script.

it "user login" do
  visit("https://whenwise.com")
  home_page = HomePage.new(driver)
  home_page.click_sign_in
  sign_in_page = SignInPage.new(driver)
  sign_in_page.enter_email("driving@biz.com")
  sign_in_page.enter_password("test01")
  sign_in_page.click_sign_in_button
  expect(driver.page_source).to include("You have signed in successfully")
end

When I explain the above to a business analyst, a manual tester or even a customer, I first said “Ignore HomePage.new(driver) part, do you understand this test script?”. A typical answer is “Yes, I understand, it is like the manual test script or even the user guide”. Please note, this is an executable script in a proper scripting language, Ruby.

For BA, manual testers or customers who showed interest in running this test script, I offered “I can help you run them on your machines”. At first, they thought it was going to be a huge effort. Then I told them, it would be just a couple of minutes. Then, I helped them

• Get and install TestWise

• Get our test scripts (using Git)

• Launch TestWise, find the test case and run it

All done, just a few minutes. Because the test script (the top tier as I just showed) is so easy, most BAs are comfortable making some changes and running automated tests for their own needs.

I know some readers will have questions, such as “Where are actual test scripts, i.e. clicking a specific element?”, “Are you suggesting BAs, manual testers, customers run or even run automated end-to-end test scripts?”, …, etc. I will answer these shortly. But here is a fact that we all test automation engineers should accept first:

Rule 1: The audience of Automated End-to-End Test Scripts (top tier) is the whole team, including business analysts, manual testers and even customers. They should be able to understand them confortably, and run them easily.

Once you accept this, the ‘fixation on a particular language such as JavaScript or Java is ridiculous, obviously”. I have written thousands of end-to-end test scripts against all kinds of apps (written in Java, PHP, C#, Python, Ruby, and JavaScript) using the same raw Selenium WebDriver + RSpec (ruby).

Test Script Tiers

The test script I showed earlier is the top tier, which I call it “Domain Specific Language (DSL) tier”. A well-written test script, for this tier, the non-technical team members should be able to understand them comfortably.

Your next question will be, “What is the statement tier look like?”. Below are the ones for the above DSL tier.

class HomePage < AbstractPage
  def initialize(driver)
    super(driver, "")
  end
  
  def click_sign_in
    driver.find_element(:link, "SIGN IN").click
  end
end

class SignInPage < AbstractPage
  def initialize(driver)
    super(driver, "")
  end
  
  def enter_email(email)
    driver.find_element(:id, "email").send_keys(email)
  end

  def enter_password(pass)
    driver.find_element(:id, "password").send_keys(pass)
  end

  def click_sign_in_button
    driver.find_element(:id, "login-btn").click
  end 
end

You can find raw Selenium statements here.

I know some don’t like the repetitive style of selenium driver.find_elementstatements, such as:

driver.find_element(:link, "SIGN IN").click
driver.find_element(:id, "email").send_keys("wise@tester.com")
driver.find_element(:id, "password").send_keys("useSelenium")
driver.find_element(:id, "login-btn").click

They try to “improve” it with something like enter_text("email", "xxx") or type('email’, ‘xxx’). By the way, I made this mistake myself 13 years ago, creating a syntax ‘framework’: rwebspec. It got a total of 6-digits downloads as well. That was before Selenium WebDriver. Soon, I deprecated it (acknowledging that was a bad or at least unnecessary approach, due to the tiered design) and have been using raw Selenium WebDriver directly, ever since.

Both Tiers (DSL And Statements) Shall be in the same scripting language