My Innovative Solution to Test Automation: Keep the Browser Open after Executing an Individual Test

Two Execution modes: an individual test → keep the browser open; all tests in one test script file → close the browser as normal

This is included in my “My Innovative Solution to Test Automation and Continuous Testing” series.

Table of Contents:
· Background
· The Problem
· First Solution
· Improvement
· Three Important Lessons I learned
· FAQ

Background

I started Automated UI Testing with JWebUnit in 2005 (learned from a world-renown programmer). I used IntellijIDEA, my favourite Java IDE, to develop and run tests. The automated functional tests were very similar to unit tests in many ways:

• Headless, unable to view actual test execution in the browser
  (it is actually simulation, not real browser rendering)

• In the JUnit syntax framework

As a programmer, I took it for granted. One day, I showed (with pride) the execution of one test to a manual tester; the poor tester was confused. He asked: “Where is the execution?” At that moment, I realized that headless testing was not good simply because it was not convincing to everyone.

Then, I found Watir, a true web test automation framework that can drive an IE browser.

Tip: The test scripting language does NOT need to be the same as the language your developers are using.

One problem, Watir is in the Ruby language. There was no IDE (that I was aware of) like IntellijIDEA. I decided to create my Testing Tool (iTest2, renamed to TestWise later) in my spare time. Frankly, I did not have much hope for it. Rather, it was a pet project to practise my Ruby programming. (I was so happy to discover Ruby, after programming Java for nine years. TestWise v1-v4 was coded in Ruby; TestWise 6 is coded in C++)

When I started using the initial version of my tool at work, frankly, I was a bit embarrassed by its poor and simplistic user interface. To my surprise, one business analyst liked it. He said: “It is simple”. By watching him using iTest2, I got some ideas for improvements.

A typical automated functional test script is like this:

before(:all) do
  open_browser
end

after(:all) do
  close_browser
end

it "test one" do
end

it "test two" do
end

The Problem

After trigging a run of a Watir test, an IE browser is launched to run the test steps. Once done, the browser is closed.

However, business analysts and manual testers want to keep the browser open after test execution, to inspect/verify the app. In fact, I, as an automated tester, wanted that even more! When a test failed (which happens a lot during test creation, debugging, and bug-fixing), the first two things a tester wants to see are:

1. Which test step failed (in the test script)?
   this one is quite easy to tell.

2. What is the app like (in the browser)?
   for this reason, we need to keep the browser open.

So I commented out close_browser .

after(:all) do
  # close_browser
end

However, this will lead to too many browser sessions (especially running in a CI/CT server), which is not good.

A programmer might suggest adding break points, and run tests in a debugging mode. I don’t recommend that, for its complexity. Back in 2006, I had a vision that test automation should be accessible to every team member. Because of this vision, I spent thousands of hours working on a dedicated testing tool from the ground up that is easy and free* to use. My daughter started writing a reasonable good-quality raw Selenium WebDriver tests at the age of 12, using TestWise.

“Free software” is a matter of liberty, not price — GNU

*free: TestWise comes with a free (price) mode and licensed mode. The only difference is that in free mode, there is 15 limit on test executions per launch. Once exceeded, just simply relaunch TestWise, which only takes a few seconds. There is no other restrictions (liberty).

First Solution

After a few attempts, I realized just changing test scripts is not enough, and need to be used with a combination of the tool.

I changed the test script:

after(:all) do 
  close_browser unless is_debugging_mode?
end

# this function is defined in the helper
def is_debugging_mode?
  $KEEP_BROWSER_OPEN == "true"
end

Basically, close_browser can be controlled by a global variable. This global variable is set in the iTest2 tool (later renamed TestWise).

A pin button on the toolbar in early TestWise versions: keep the browser open

Let’s see a demonstration. I am using the new TestWise 6 here, the pin button is gone (I’ll explain why later), just to let you get the idea of “Keep the browser open”.

Execution one test case: “Return Trip”

So, when the ‘PIN’ button is on, the browser will be kept on when the test is run in TestWise. When executing the test from the command line or in CI/CT server, the browser will close normally.

The business analysts in the team loved this feature. With their promotion, some team members (manual testers, programmers, the architect) started using TestWise.

Improvement

While the “Keep the browser open” feature is good, most people enable it by default. I notice few would change it. As a result, often left many browser sessions around.

This problem has hung around in my brain for a while. Frankly, I didn’t care that much because this may be classified as a user issue.

I got the idea one day while watching a business analyst run a few automated tests to generate test data. I noticed:

• When he ran an individual test case, he would like the browser kept open

• When he ran all test cases in one test script, he did not care about the browser.
  In this mode (executing ‘test one’ and ‘test two’ in the above test script example), he just wanted to complete a certain task.

Based on that, I classify test executions in TestWise into two modes:

1. Individual Test Execution
   execute a single test case. By the way, one automated test script may contain one or more test cases.

2. Test Script Execution
   execute all test cases in the script.

Once that is clear, I implemented two execution modes with different “Keep Browser Open” behaviours.

• Individual Test Execution => “Keep the Browser Open”

• Test Script Execution => “Close the Browser”

You have already seen a demo of “individual test execution”. Here is a “test script execution”.

Execute all tests in the script: “Return trip”, “Oneway trip”

The users get the desired behaviour by choosing the right execution mode, naturally. Later, I removed the unnecessary “PIN” button in TestWise.

Three Important Lessons I learned

During the process of implementing this feature, I learned some valuable lessons that help me come up with more innovative solutions (and implemented them).

• Dropping a programmer’s ego, then you will find out that you can learn a lot from business analysts and manual testers in test automation and CT.
  For example, commonly, almost all test automation attempts using C#, Java, and JavaScript failed, one common reason (IMO, it is silly) is that engineers insist on using the same language in test scripts as the coding language. Just learn Ruby, as suggested in the classic ‘Agile Testing’ book, and you will be surprised by Joy.

• Being proactive, you can make a difference.
  Instead of complaining (test automation failed everywhere), why not try to make a difference? After all, I am a software engineer.

• Automated End-to-End Testing is useful for all team members.
  This is quite logical and obvious when pointed out, as End-to-End testing is using/testing the app as a user would, which means all team members. Check for the “Benefits of Test Automation and Continuous Testing series”: Executives, Managers, Business Analysts, Developers, Testers, and Customers.
  I kept this in mind for test script design and my tools development (TestWise and BuildWise), that is, a non-tech business analyst or manual test can start using it in minutes.

This is the first successful implementation of my innovative idea.

Fifteen years passed, and I have been using this feature almost every working day. Many attendees to my training liked this instantly. I was quite proud of it and surprised that no one has copied this idea yet (I won’t charge IP for it).

FAQ

1. How do users invoke “individual test execution”?

Easy, right-click any line within the test case (for the example below, between lines 24 and 38), then select Run "THE SELECTED TEST CASE NAME"

2. How do users invoke “test script execution”?

Right-click any line outside a test case, and select Run test cases in 'TEST FILE NAME' .

Or, simply click the ‘Run’ button on the toolbar.

---

Related reading:

• My eBook: “Practical Web Test Automation with Selenium WebDriver”