Testing RESTful Service in Ruby, Part 2: Run frequently in a Continuous Testing Server
Stabilizing the tests and setting them up in a Continuous Testing server
In Part 1, I created five RESTful service tests in Ruby. They were working, however, but the task is not completed yet.
Table of Contents:
· Stabilize Tests
∘ 1. LIST all records
∘ 2. READ one record
∘ 3. CREATE a new record
∘ 4. UPDATE a new record
∘ 5. DELETE a record
· Run all tests in TestWise a couple of times
· Set up build project in BuildWise to run all tests in a CT Server
· Review (by Zhimin)
∘ Effort on Test Creation is minor
∘ The Power of Scripting in Ruby
∘ Universally Applicable
∘ Freedom and Fun
∘ The SynergyI hardcoded an ID value (e.g. 66670) to use in some tests, which run fine once only. The ID might already be taken (duplicate error for CREATE), or may not exist (causing READ, UPDATE and DELETE operations to fail). Also, I didn’t clean up after using it.
To run tests in a Continuous Testing (CT) server, I should stabilize the tests firsts: can run them successfully multiple times. In this article, I will revise the tests written in Part 1 and set up running them in a CT server, as regression testing.
Stabilize Tests
1. LIST all records
No changes from Part 1.
it "List all records" do
require "httpclient"
require "rexml"
http = HTTPClient.new
resp = http.get("http://www.thomas-bayer.com/sqlrest/CUSTOMER")
xml_doc = REXML::Document.new(resp.body)
expect(xml_doc.root.elements.size).to be > 10
expect(xml_doc.root.elements.first.text).to eq("1")
end2. READ one record
No changes, maybe. It might be OK to assume that record #1 always exists, so long as there are no attempts to update or delete it. If this test starts to fail, we can always stabilize this by invoking CREATE service to use the freshly created one (see UPDATE or DELETE for more).
it "Read first record" do
http = HTTPClient.new
customer_id = 1
get_rest_url = "http://www.thomas-bayer.com/sqlrest/CUSTOMER/#{customer_id}"
resp = http.get(get_rest_url)
puts resp.body
expect(resp.body).to include("<LASTNAME>King</LASTNAME>")
end3. CREATE a new record
If the ID is not provided, the service might assign a unique ID for a newly created record. However, this is not the case for this service. If we supply a specific ID, the first test execution passes, but the second one would fail because of the “duplicate record ID error”. In fact, “500 Internal Server Error” was returned for this scenario.
How can we make sure our creation request passes every time? Supply a unique ID each time the tests are run. This is very easy to do with Ruby Scripting.
To prevent creating duplicate IDs, we need to make this ID dynamic (or random). So I generated a random number between 9001 and 9999 with the Faker gem. I chose this range because it’s can be easily differentiated from real records. There is a ~0.1% chance of duplicate IDs, but that’s a chance I’m willing to take for this exercise.
new_id = Faker::Number.between(from: 9001, to: 9999)Then use this new_id in the request data: <ID>#{new_id}</ID>.
After verifying a successful creation, delete this record via invoking DELETE service.
Full test case
it "Create a new record" do
new_id = Faker::Number.between(from: 9001, to: 9999) http = HTTPClient.new
create_rest_url = "http://www.thomas-bayer.com/sqlrest/CUSTOMER/"
new_record_xml = <<END_OF_MESSAGE
<CUSTOMER>
<ID>#{new_id}</ID>
<FIRSTNAME>P</FIRSTNAME>
<LASTNAME>Sherman</LASTNAME>
<STREET>42 Wallaby Way</STREET>
<CITY>Sydney</CITY>
</CUSTOMER>
END_OF_MESSAGE
resp = http.put(create_rest_url, new_record_xml)
expect(resp.body).not_to include("Internal Server Error") fail_safe { delete_record(new_id) }
endThe delete_record is a function I created (using TestWise’s Extract to Function refactoring). This is the beauty of scripting automated tests in a powerful scripting language, such as Ruby. We can use rich and well-implemented libraries, such as Faker (for generating test data), and some basic programming practices, such as Helper Functions and OO design.
4. UPDATE a new record
We cannot just update an existing record in an automated test, because the test won’t be valid after the first run. Check out this article, “One Test Automation Scenario Interview Question that Most Candidates Failed”.
Therefore, the test design will be:
Create a brand-new record
Update it
Verify the update
Delete it
it "Update a record" do
record_id = create_record
http = HTTPClient.new
update_rest_url = "http://www.thomas-bayer.com/sqlrest/CUSTOMER/#{record_id}/"
update_xml = <<END_OF_MESSAGE
<CUSTOMER>
<FIRSTNAME>Paul</FIRSTNAME>
</CUSTOMER>
END_OF_MESSAGE
resp = http.post(update_rest_url, update_xml)
expect(resp.code).to eq(200) # OK
expect(read_record(record_id)).to include("Paul")
delete_record(record_id)
endThere are two reusable helper functions I created: create_record and read_record.
5. DELETE a record
Like UPDATE, we cannot delete an existing record. Create a new one first.
it "Delete a record" do
record_id = create_record http = HTTPClient.new
delete_rest_url = "http://www.thomas-bayer.com/sqlrest/CUSTOMER/#{record_id}"
resp = http.delete(delete_rest_url)
expect(resp.body).to include("<deleted>#{record_id}</deleted>")
endRun all tests in TestWise a couple of times
Before I put the tests in a CT Server to run as regression, I will try to run them multiple times locally first. The below video shows the tests running twice in TestWise.
Good, these five tests seem reliable. We are ready to check in and run them on a CT server, such as BuildWise.
Set up a build project in BuildWise to run all tests in a CT Server
You can check out this article: “Set Up a Continuous Testing Server to Run Selenium Tests in Minutes” for setting up a new BuildWise CT server.
Once it is up, it is very easy to create a new API Testing project to run these tests. For any CI/CD/CT, we need to check in code/tests into a source repository, Git mostly. If you are not familiar with Git, check out the 10-Minute Guide to Git Version Control for Testers.
The CT build project setup is very easy, it is exactly the as the Selenium one in the guide above. The basic steps are:
1. Check the tests into a git repository
2. Update the Rakefile to run your tests
The project template created by TestWise IDE is BuildWise CT ready. I just need to update Rakefile to specify the test file to be included in a sequential build (running tests one by one on the server)
3. Set up a build project to run these REST tests in BuildWise.
Screenshot of the project set-up screen:
4. Trigger a few runs
Below is the report of the 2nd run (you can see two runs for this project, top-left).
Review (by Zhimin)
Many inexperienced automated testers (especially former programmers) weigh heavily on test creation. If an automated test pass once, considers it is done. This is very wrong. This is proved by ~100% failure of candidates on this simple scenario interview question.
Effort on Test Creation is minor
Test Creation, based on my guess from working in test automation for 17 years, only accounts for ~10% of the effort. I will write a separate article on that topic. In the meantime, I will just briefly touch on two other activities:
Test Stabilization and Refinement
Make the test run reliably when running them multiple times and with other tests at the same time. After that, refactor it to a better design, so that it is easier to read, faster, and more importantly, easier to maintain.Continuous Testing
Inevitably, the application will change, and hence, will affect your test execution. So, there will ongoing maintenance.
So when I am about to work on a new test, my end goal is clear, that is, this test will be running fine in automated regression testing (without breaking other tests).
The Power of Scripting in Ruby
Once you are clear about the above, you would understand that GUI automation tools such as Postman/SoapUI are not suitable for real test automation. You can use those tools as utilities to assist test creation, that’s all. You do need the power and flexibility of programming (in a scripting language) in test automation.
Universally Applicable
I have been using the same API testing approach, just shown by Courtney in these two articles, for over 17 years. It worked every single time and impressed others with super-high (to them, for me, it is normal) efficiency. Why? it works logically. API Testing → Scripting tests to invoke API based on protocol → use Ruby scripting to invoke RESTful services on HTTP protocol, or SOAP services on the SOAP protocol.
Freedom and Fun
A big problem (among many others) with proprietary script syntax from a vendor is the lack of freedom. Very likely, you will be frustrated with bugs and limitations. This is not fun.
On the other hand, using Ruby Scripting is creative and fun!
The Synergy
More often than not, API testing often needs to be used with other types of testing in conjunction. For examples:
retrieving test data from the database (SQL)
copy files from another remote machine (SSH)
creating a prerequisite scenario via Web (Selenium WebDriver)
Therefore, a tool for a specific protocol is not enough. The solution is simple: write your tests in a mature scripting language such as Ruby or Python, and all these problems are resolved.
 | A guest post by
|