Case Study: Mechanize Performance/Load Testing User Login with CSRF Token Protection
How to use Mechanize for a website’s performance and load testing
Mechanize is a library for automating interactions with websites, without a browser involved. Mechanize is implemented in many scripting languages, such as Perl (WWW::Mechanize), Python (mechanize) and Ruby. In this exercise, I will use it for performance/load testing in Ruby.
If you are new to Ruby, don’t need to worry about learning the Mechanize syntax, just run the sample scripts. The purpose is to get you a feel of scripting a performance/load test using Mechanize.
Table of Contents:
· Preparation
∘ Login
∘ Add assertion
∘ Debugging
∘ Add measurements
· Performance Testing
· Simple Load Testing
∘ Generate load
∘ Measure timings
∘ Compare Load Test Results
∘ Failure to load test AJAX
Preparation
Mechanize is a Ruby library, besides Ruby, you will need to install the Mechanize gem as well.
gem install mechanize
Login
Below is a Mechanize script that logs into a website.
require 'mechanize'
@browser = ::Mechanize.new
@browser.get("https://whenwise.agileway.net")
@browser.get("https://whenwise.agileway.net/sign-in")
login_form = @browser.page.forms.first
login_form.fields_with(:name => "session[email]").first.value = "driving@biz.com"
login_form.field_with(:name => "session[password]").value = "test01"
@browser.submit(login_form, login_form.button_with(:id => "sign-in-btn"))
When you run it, the program completes in about 1 second. Did it work?
Add assertion
We can add an assertion to verify that our Virtual User actually landed on the dashboard page.
# verify the specific text only shown to logged in user
raise "Not logged in" unless @browser.page.body.include?("Current Plan")
Debugging
Still not sure, we can save the current page to a file and open it in a browser to inspect.
File.open("/tmp/a.html", "w").write(@browser.page.body)
# then open /tmp/a.html in Chrome
It looks like this. It worked (i.e., logged in).
The page is rendered without CSS and JavaScript, we could modify this off-line HTML to use the site’s web resources (such as CSS, images and JavaScript). But it is not necessary, we only care about the actual page content in load testing.
Add measurements
The above automated script works functionally. To make it a performance test, we need to add some measurements.
In functional testing, the measurement of execution time is simple: end_time
minus start_time
. For performance/load testing, the measurement is on the time it took for the server to handle individual user operations. For a typical web request:
A. User initiates an action, e.g. click a link
B. The request arrives at the server
C. The server sends the response (HTML) back D. The response arrives at the client’s browser
E. The browser finishes the rendering of the page
The correcting timing for performance/load testing is B→C. However, the network transferring times (A→B and C→D) is negligible (and hard to measure consistently) nowadays. Therefore, we normally just take the timings of A→D.
Below is basic timing in Ruby.
start_time = Time.now
# operation here ..
duration = Time.now - start_time
puts "took #{duration} seconds"
As the above is going to be used often, I will extract them into a reusable function, log_time
:
def log_time(msg, &block)
start_time = Time.now
begin
yield
ensure
end_time = Time.now - start_time
puts("#{msg} | #{end_time}")
end
end
A sample usage:
log_time("Visit home page") {
@browser.get("https://whenwise.agileway.net")
}
Performance Testing
Record the timings of each operation, then we get a performance test script.
require 'mechanize'
site_url = "https://whenwise.agileway.net"
@browser = ::Mechanize.new
log_time("Visit home page") { @browser.get(site_url) }
log_time("Visit login page") { @browser.get(site_url + "/sign-in") }
login_form = @browser.page.forms.first
log_time("enter data") {
login_form.field_with(name:"session[email]").value = "driving@biz.com"
login_form.field_with(:name=> "session[password]").value = "test01"
}
log_time("Login") {
@browser.submit(login_form, login_form.button_with(:id=> "sign-in-btn"))
}
The log_time
function is predefined. Here is what an output looks like.
|Visit home page |1.088529|
|Visit login page |0.208664|
|enter data |0.000036|
|Login |1.043211|
These timings are true performance testing results (compared to when you do these operations in Chrome), as no browser rendering is required.
The timing of “enter data” in the above example is unnecessary, as there is no interactions with the server. Those two steps simply prepare the form data for submission.
Simple Load Testing
Next, we will generate load with a number of concurrent users.
One way is to run the command multiple times, such as:
ruby load_test.rb &; ruby load_test.rb &; ruby load_test.rb &
Of course, this way of generating load is no good, as it is hard to manage and unable to aggregate the timings. However, this gives engineers a quick taste of load testing.
Generate load
In protocol-based testing, concurrency is typically implemented using threads.
Threads, in software programming, are a way for a program to split itself into two or more simultaneously running tasks.
require 'mechanize'
@browser = ::Mechanize.new
virtual_user_count = 5 # may change concurrent users count here
threads = []
virtual_user_count.times do |idx|
threads[idx] = Thread.new do
Thread.current[:id] = idx + 1
@browser.get("https://whenwise.agileway.net")
@browser.get("https://whenwise.agileway.net/sign-in")
login_form = @browser.page.forms.first
login_form.field_with(:name => "session[email]").value = "driving@biz.com"
login_form.field_with(:name=> "session[password]").value = "test01"
@browser.submit(login_form, login_form.button_with(:id=> "sign-in-btn"))
raise "Not logged in " unless @browser.page.body.include?("Current Plan")
end
end
threads.each {|t| t.join; } # wait for all threads to complete
There will be no output from executing the above script (because there is no timing yet), however, if you monitor the log of your app, you will see a lot of activity there.
Measure timings
By combining the performance test script and load generation, we get a basic load testing script.
Keep reading with a 7-day free trial
Subscribe to AgileWay’s Test Automation & Continuous Testing Blog to keep reading this post and get 7 days of free access to the full post archives.