Case Study: User Login Tests in Selenium WebDriver

A super simple automated test case that some might still get wrong.

This is also included in the “How to in Selenium WebDriver” series.

Testing user login is usually the first testing task for an app. This article will go through typical user login test cases in Selenium WebDriver and highlight a common pitfall in automated testing.

Example site: Agile Travel (https://travel.agileway.net/login).

AgileTravel login page

Table of Contents: 
− Test Design
− Test Case 1: Login OK
− Test Case 2: Login Failed
− Oops! Failed when Running the Whole Test Script
− Analyse and Fix it
− Review by Zhimin

Test Design

For login tests, there are usually at least two test cases.

1. One happy 😊 case: Login successfully

2. One alternative 😞 case: Login failed from bad credentials

Test Case 1: Login OK

The test script (Selenium + RSpec) below is quite straightforward, it consists of four steps, just like how it is done manually.

it "[1] Can sign in OK" do
  driver.find_element(:id, "username").send_keys("agileway")
  driver.find_element(:id, "password").send_keys("testwise")
  driver.find_element(:id, "username").submit   
  expect(driver.page_source).to include("Signed in")
end

Run this test case.

It passes! Let’s move on to the alternative case now.

Zhimin: Courtney showed individual test execution in TestWise, which will leave the browser open after test execution completes. Check out this article: My Innovative Solution to Test Automation: Keep the Browser Open after Executing an Individual Test.

Test Case 2: Login Failed

The steps are the same, except with different input (wrong password) and assertions.

expect(driver.page_source).to include("Invalid email or password")

Run this test.

This test case passes as well!

Oops! Failed when Running the Whole Test Script

However, the job is not done yet. While both tests passed individually, we want to make sure the whole test script (including these two together) passes as well. To do that in TestWise (a testing IDE), click the “Run Test Script” button (big blue triangle) on the toolbar.

TestWise’s “Run all tests in the file” button

The result: only the first one (login OK) passes, and the second alternate case fails.

Why?

Analyse and Fix it

After the execution of the first test case, the user is logged in successfully in the browser. The second test tried to continue from there, but it was expecting the login page. That’s why it failed on the first line (line 29) of the second test case.

To fix this, we want a clean state before each test case. The easiest way to do this is logout at the end of the login OK case.

Click “Sign off” to log out after the login OK case

This is the easiest way to ensure the second test case starts on the login page.

This works, but if there are more alternate test cases in this login test script file, it might make more sense to extract it to the after(:each) block.

Full Test Script:

load File.dirname(__FILE__) + "/../test_helper.rb"
describe "User Authentication" do
  include TestHelper
  before(:all) do
    @driver =  Selenium::WebDriver.for(:chrome)
    driver.manage().window().resize_to(1280, 720)
    driver.get("https://travel.agileway.net")
  end
  after(:all) do
    driver.quit unless debugging?
  end

  it "Sign in ok" do
    driver.find_element(:id, "username").send_keys("agileway")
    driver.find_element(:id, "password").send_keys("testwise")
    driver.find_element(:xpath, "//input[@value = 'Sign in']").click
    expect(page_text).to include "Signed in!"
    driver.find_element(:link_text, "Sign off").click
  end

  it "Sign in failed" do
    driver.find_element(:id, "username").send_keys("agileway")
    driver.find_element(:id, "password").send_keys("test")
    driver.find_element(:xpath, "//input[@value = 'Sign in']").click
    expect(page_text).to include "Invalid email or password"
  end

Review by Zhimin

I have used this exercise for my one-day test automation training since 2006 and in the second chapter (‘First Automated Test’) of my book: “Practical Web Test Automation”.

The objectives of this exercise (for those who are new to automation, Courtney did this at the age of 12) are:

• Write automated test steps like manual testing
  Preferably, one step for one user operation.

• The importance of independence between test cases
  Also, understand the concepts of “test case” and “test script”.

The whole exercise is quick and fun because of that twist.

However, the test script is not done yet. It works but is not easy to maintain. We need to refactor it based on a Maintainable Automated Test Design.

load File.dirname(__FILE__) + '/../test_helper.rb'
describe "User Login" do
  include TestHelper
  before(:all) do
    @driver =  Selenium::WebDriver.for(:chrome)
    driver.manage().window().resize_to(1280, 720)
    driver.get("https://travel.agileway.net")
  end
  after(:all) do
    @driver.quit unless debugging?
  end

  it "Sign in ok" do
    sign_in("agileway", "testwise")
    expect(page_text).to include "Signed in!"
    sign_off
  end

  it "Sign in failed" do
    sign_in("agileway", "badpass")
    expect(page_text).to include "Invalid email or password"
  end
end

I extracted the three login steps to a reusable helper function sign_in . I did that quickly with the test refactoring support in TestWise. For a step-by-step guide, check out this article: Functional Test Refactoring: Extract Function.

---

Further reading:

• eBook: “Practical Web Test Automation”