Playwright with Mocha against Chrome browser
Write Playwright tests in Mocha to take advantage of its wide support, such as test execution in CI/CT servers. By the way, Playwright sucks in every way compared to raw Selenium WebDriver + RSpec.
I don’t like using JavaScript for scripting automated tests. Every test automation attempt (I witnessed) that used JS frameworks, such as PhantomJS, Protractor, TestCafe, Puppeteer and Cypress, has failed (see the definition of E2E test automation success). These days, it seems that Playwright is the new favourite among JS automated testers. I tried Playwright, and I don’t like it.
My daughter started her intern job a week ago. On the first working day, she created two automated tests (using Selenium WebDriver + RSpec, the agileway test automation formula), and the team leader was very impressed. A few days later, she got the tests running in a BuildWise CT server (she had to wait for local admin access which took a while). Then, a team member asked her to talk to ‘the automation person’ in the company (a large one) who suggested she use Playwright. She was willing to learn Playwright, which I supported too (to realize how good the raw Selenium WebDriver + RSpec is).
Around noon yesterday, she messaged me, “Playwright is so hard, compared to raw Selenium WebDriver. Its syntax is not logical. I missed the ‘Run selected test against current browser’ feature in TestWise.” I replied, “Yeah, I understand. But hang in there and try really hard, be open-minded.” At the end of the day, she completed the core of the sample tests (did in Selenium in a fraction of the time), though some still need further refactoring. She said: “If using raw Selenium WebDriver, I would have finished a lot faster. But I doubt that Playwright tests are maintainable when a test suite reaches a certain size”. I answered: “It won’t. You saw 500+ End-to-End UI tests running daily in my test lab, which you now regard as normal. With a JavaScript framework, I don’t think you will ever see a real solution of 50 tests.”
She raised an issue that might prevent her from running her Playwright tests in the BuildWise CT server as Playwright got its own runner. She had trouble getting it working with Mocha. Therefore I took the task this morning.
Quick Start
I followed the instruction on the Playwright site.
npm i -D @playwright/test
# install supported browsers
npx playwright installWhen I saw the below, I shook my head.
Downloading Playwright build of chromium v939194–119.9 Mb [=========== ] 55% 16.4s
Downloading Playwright build of firefox v1304 - 71.7 Mb [============ ] 60% 5.3s
Downloading Playwright build of webkit v1578 - 54.2 Mb
[======= ] 36% 8.1sThis is wrong. Why not just use the browser (Chrome) on the user’s machine?
Run a hello-world test:
npx playwright testIt ran, but the browser was not shown. I realized it was the headless mode by default. (If you have read my article Headless Browser Testing Clarified, you will remember the history shows that fake test automation engineers like headless).
npx playwright test --headedI played a few test steps according to the Playwright's documentation. I could get the test steps working, but felt unnatural (as my daughter described). It was not intuitive as Selenium WebDriver, which has an easy-to-follow pattern “Find element and perform action”. Because I mentored and worked with numerous manual testers and business analysts, my understanding of “Good Test Script Syntax” is different from most JS programmers. By the way, I knew JavaScript well (authored one book on JS test automation: Selenium WebDriver Recipes in Node.js, Educative editors liked this book and converted it into an interactive course).
Use Chrome Browser
I don’t like to use the so-called “Playwright build of chromium” which might have issues. I want to use the same browser as the end-user: Chrome.
After some google searches, I found that it is actually not hard to do. Just supply executablePath .
const { chromium } = require('playwright');
// ...
browser = await chromium.launch({
headless: false,
executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
});
context = await browser.newContext();
driver = page = await context.newPage();
await driver.goto('https://travel.agileway.net');Run Playwright Mocha tests
Once I got a hold of the browser context (see above), I could put it in the Mocha syntax, the most widely used JS test syntax framework.
Here is a complete sample test script (with two test cases).
const { test, expect } = require('@playwright/test');
const { chromium } = require('playwright');
var assert = require('assert');
const timeOut = 15000;
let driver, page, browser, context
String.prototype.contains = function(it) {
return this.indexOf(it) != -1;
};
var helper = require('../test_helper');
describe('User Authentication', function() {
before(async function() {
browser = await chromium.launch({
headless: false,
executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
});
context = await browser.newContext();
driver = page = await context.newPage();
});
beforeEach(async function() {
await driver.goto('https://travel.agileway.net');
});
after(async function() {
if (!helper.is_debugging()) {
browser.close();
}
});
it('[1,2] Invalid user', async function() {
await driver.title().then(function(the_title) {
// console.log(the_title)
})
await driver.fill("#username", "agileway")
await driver.fill("#password", "playwright")
await driver.click("input:has-text('Sign in')")
await driver.textContent("body").then(function(body_text) {
//console.log(body_text)
assert(body_text.contains("Invalid email or password"))
})
});
it('User can login successfully', async function() {
await driver.fill("#username", "agileway")
await driver.fill("#password", "testwise")
await driver.click("input:has-text('Sign in')")
await driver.textContent("body").then(function(body_text) {
//console.log(body_text)
assert(body_text.contains("Signed in"))
})
await driver.click("a:has-text('Sign off')")
});
});I really don’t like typing
async,await,{ ... });in automated test scripts.
More tests can be found on this GitHub repository: https://github.com/testwisely/agiletravel-ui-tests/tree/master/playwright-mocha
Various locators (button, select dropdown, radio, link)
Text assertion
Maintainable Test Design (reusable helper function and page object model)
BuildWise CT server integration
Then, I ran a set of tests from the command line using mocha command.
This means I could run the Playwright tests easily in a CI or CT server such as BuildWise.
Chrome is not Playwright bundled browser, yet. Why not just use raw Selenium WebDriver + RSpec, much easier to learn, much higher efficiency, more intuitive, more stable, more readable and more fun. Above all, WebDriver is a W3C standard.
Run Playwright tests in TestWise
TestWise 6 supports executing Mocha tests too (but not as well as for RSpec). I used TestWise to develop the above 4 Playwright tests.
Further reading:
I strongly recommend switching to raw Selenium WebDriver + RSpec, which makes test automation engineers’ life better and more fun! Check out this article, Selenium WebDriver is the Easiest-to-Learn Web Test Automation Framework, Step by Step showing how to write raw Selenium WebDriver test scripts in minutes.
Why JavaScript Is Not a Suitable Language for Real Web Test Automation?
Evil Mudslingings against Selenium WebDriver
Selenium WebDriver is proven the best for testing web apps, and it ticks all the boxes.Why Do Most UI Test Automation Fail?: Wrong choice of automation framework