07 Execute Test Case vs Test Script

Understand the two common test execution modes.

Learning Objectives

• Run test file vs Run test case

• Test Development Rythme

  • Run test case

  • Run test script file (all test cases)

• Test Execution side effect

• Review

  • Comment out scripts (exclude from execution)

Session #07

From the previous sessions, we have two test cases in one test script file: `login_spec.rb`.

Tasks

Test Execution Rythme
1. Run test case (one by one)
2. Until each execution of an individual test case pass
3. Run the test script (all test cases in it)

Task 1. Run two test cases individually, one by one

Right-click any line between the test case “User can sign in OK”, and run this test case (first option, as indicated below).

The test execution ✅

Now run the second test case “User can sign in failed”.

The test execution ✅ too.

Task 2. Run the test script (i.e. all test cases)

A test script file `xxx_spec.rb` may contain one or more test cases. Click the blue play button (as indicated below) to run the test script, i.e. all test cases in the test script file.

The second test case failed! But it passed a minute ago without any changes. (If you reran the second test, it would pass as well) The only logical explanation is something to do with the execution of the first test case.

Knowledgement Point: Execution Side Effect

We can treat the target application in one browser window as a shared resource; any test execution (manual or automated) can change its state, affecting the upcoming test executions. This is an important concept in Software Testing. It is a common concept but is made obvious in test automation.

In this exercise, the execution of the first test case affected the second. Let’s follow the state of the application in the browser window.

• Browser launched by Selenium (sign-in state: No)

• After execution of the first test case (sign-in state: Yes)

• Before the execution of the second test case (sign-in state: Yes)

• Try to sign in, Failed! Already signed in.

Knowledgement Point: Difference between two test execution modes.

Before I clarify the differences between the two execution modes (test case and test script), have a look at this `after(:all)’ block (in all of our test scripts).

  after(:all) do
    driver.quit unless debugging?
  end

1. Execute a single test case

   The debugging? flag is set to true (in TestWise).
   therefore, the browser will be left open.

2. Execute a test script file

   

   i.e., all the test cases in this test script file.
   The debugging? flag is set to false (in TestWise)

   The browser window will close, after test execution, in this mode.

Task 3. Analyse and Fixes

After the execution of the first test case, the user is logged in successfully in the browser. The second test tried to continue from there, expecting the login page. That’s why it failed on the first line (line 29) of the second test case.

To fix this, we want a clean state, i.e., not logged-in state, before each test case.

Approach 1: Swap the order of two test cases (not recommended)

This is acceptable but not optimal. Ideally, we want each test case independent of others, including the execution order.

Approach 2:

A more natural way is to sign off at the end of the login OK case. (you probably have done that in manual testing)

To do that, append a sign-off step in the first test case:

  driver.find_element(:link_text, "Sign off").click

Run the whole test script, pass!

Click “Sign off” to log out after the sign in Ok test

Complete Test Script

load File.dirname(__FILE__) + "/../test_helper.rb"

describe "User Authentication" do
  include TestHelper

  before(:all) do
    # browser_type, site_url defined in test_helper.rb
    @driver = Selenium::WebDriver.for(browser_type, browser_options)
    driver.manage().window().resize_to(1280, 800)
    driver.get("https://travel.agileway.net")
  end

  after(:all) do
    driver.quit unless debugging?
  end

  it "User can sign in OK" do
    driver.find_element(:id, "username").send_keys("agileway")
    driver.find_element(:name, "password").send_keys("testwise")
    driver.find_element(:name, "commit").click
    expect(page_text).to include("Signed in!")
    driver.find_element(:link_text, "Sign off").click
  end

  it "User can sign in failed" do
    driver.find_element(:name, "password").send_keys("testwise")
    driver.find_element(:name, "commit").click
    expect(page_text).to include("Invalid email or password")
  end
end

FAQ

1. If I put just one test case in one test script file, will that solve the dependency problem?

   No, One test case in one test script prevents dependencies between test cases. But there are also dependencies between test scripts. We will cover that topic in later sessions.

2. Why do you put more than one test case in one test script file?

   Grouping related test cases in one test script will help you manage them better and execute them faster. Furthermore, it provides opportunities for script reuse. Again, we will cover that in later sessions.