Why Scripting Languages are Better for Writing Automated End-to-End Tests? Part 2

Comparing the Speed of Scripting and Non-Scripting Languages’ Test Execution

This is part 2 of “Why Scripting Languages are Better for Writing Automated End-to-End tests” (Part 1 here).

1. Faster

The raw code execution speed in compiled languages (e.g. C#) is certainly much faster than in scripting (a.k.a. dynamic) languages (e.g. Ruby). However, in the context of test automation execution, that advantage is minor, as most of the execution time is on the app.

When developing/debugging test scripts, we often need to run/debug individual test cases multiple times. In this mode, compiled languages are less efficient, and in fact, slower. This is due to the compilation step and VM (e.g. Java’s JVM) start-up time.

Timing/Execution of The Same Test in Ruby and C#

I timed the same test on a sample site written in Ruby (RSpec) and C# (MS VSTest).

Rspec — 8.5 seconds

Execution of Select Flight Test Script (2 test cases) in TestWise

C# — 11.8 seconds

Execution of Select Flight Test Script (2 test cases) in VS Code

The ruby version (on Gitbub) was about ~30% faster than C#’s (the test steps are exactly the same), in this case.

Why are scripting languages faster (generally)?

1. No Compile Time

2. Don’t need to load the VM

As we know, we run this mode (individual test execution for creating and debugging) very often. Code is designed to be run as a production server. But testing is different, more ad hoc & often run individually.

Maybe when running a big suite of tests at one go, compiled language will show its speed advantage (compile time is once-off, so is VM starting time).

Also, the speed gap between compiled languages and script languages is narrower. Scripting languages load lots of small files, which takes longer compared to loading a single large file. In previous decades, hard drive speed has increased significantly ~50x (HDD -> SSD, PCIe). As you can see now, even many large apps such as Github can run on scripting languages (Ruby on Rails).

2. More Readable

Generally speaking, test scripts written in a scripting language are more concise and easier to read.

Below are a few examples in Ruby, you may compare the Java/C# version in your head.

## COOL Date operations
sign_up_page.enter_dob( 17.years.ago.strftime('%Y-%m-%d' ) 
#...
report_page.enter_start_date(Date.today.beginning_of_week.to_s)
#...

## Remove uses of parentheses
driver.find_element("text").send_keys "ABC"

## Intuitive looping
3.times { driver.find_element("button").send_keys(:tab) }

3. More Powerful Text-Processing Features

Compiled languages such as Java and C# are designed for building business apps, not for testing.