Test SOAP Web Services Effectively, Easily and Freely
Use raw ruby scripts, not SoapUI or PostMan.
In the article “Failed Tech Hype: SOAP Web Services”, I shared my experience and opinion as a programmer. As I switched my daytime job to a test automation engineer in 2010, here I would like to show you how I conduct automated testing SOAP web services, an approach I have successfully used in many projects.
Table of Contents:
· Why do most SOAP testing with SoapUI fail?
· Understanding the fundamentals of SOAP
· Construct SOAP Request
∘ Generate SOAP requests using SoapUI
· Invoke a SOAP Web Service in Ruby
· Assert SOAP response
· SOAP Request with Dynamic Data
∘ Construct request messages with a template
∘ Generate message from a template with defined data
· More on Dynamic
· Continuous Testing
· A story of a junior test engineer learning this approachWhy does most SOAP testing with SoapUI fail?
Firstly, I do not use GUI tools such as SoapUI, Ready API or PostMan. Every project that I know attempted with the above has failed. (Check out my Definition of End-to-End Test Automation Success)
Yes, anyone can quickly create a handful of automated tests for a SOAP service with a GUI tool. However, there are some serious shortcomings:
Not Flexible
For example, specify a random email for sign up test (if using a fixed email, the test would fail next time)Usually not run often
In theory, we can run a suite of tests in Soap UI. However, very few people would do it as it would take some time for a reasonable suite. Moreover, there will be a lack of feedback.Hard to maintain
Experienced IT professionals would know that test maintenance requires more effort than test creation. In GUI testing, those so-called industry-leading test automation tools such as QTP and Coded UI Test all lost the battle to free, open-source and programming-based automation frameworks, such as Selenium WebDriver. Why? automated test scripts in a good scripting language are much easier to maintain.Not easily version-controlled or used by all team members
From my memory, in every failed SOAP testing attempt, there was only one person who worked on it, which was clearly wrong. While version controlling SoapUI tests is possible, plain-text test scripts are better in terms of team use.Lack of / hard-to-access test execution history
Compared to previous execution results is important in test automation.Vendor-locking
Lacking freedom.Difficult to integrate
For example, a test needs to perform an operation on the web to get the app in a specific state, and then invoke a SOAP service.Very low productivity
Many testers do not realize this until they see a real automation engineer working with plain-text-based test scripts. Check out a story of a junior test engineer learning this approach.Low script reuse
A lot of copy-n-paste.Not fun!
The above are objective; This one is my subjective opinion. Working on automated tests in Ruby is creative and fun!
Understanding the fundamentals of SOAP
SOAP consists of one request and one response, both in XML format. Fundamentally, it is XML over HTTP.
By understanding this, testing SOAP web services will be much easier. To test a SOAP web service, there are three steps:
1. Construct SOAP Request from WSDL
2. Post request to invoke the Web Service
3. Parse returned response XML to perform assertion
Construct SOAP Request
SOAP Web service is identified by its EndPoint, a URL. Here is an example: W3CSchools’ temperature conversion service.
http://www.w3schools.com/xml/tempconvert.asmxThis service includes two service functions.
But the one we really care about is Web Service Definition Language (WSDL). As its name suggests, WSDL defines web services. More specifically, it defines the XML request that it accepts and returns. Typically, the URL of WSDL is: EndPoint + ?WSDL , such as http://www.w3schools.com/xml/tempconvert.asmx?WSDL
Generate SOAP requests using SoapUI
While not using SoapUI to execute tests, I can use SoapUI (or another tool) to generate a sample SOAP request (in XML).The standard SoapUI (non-Pro) edition is free.
1. Create a new SoapUI Project with WSDL
2. Select target operation and select ‘Request 1’ under it
Normally, the sample request generated from SoapUI is fine. In this particular case, the sample request for this w3school’s web service is not right (the previous version was OK). The namespace prefix shall be x instead of x\. The actual string for the namespace prefix does not matter, The issue is with the \ character. To fix it, just replace x\ with x.
Copy the request XML to the clipboard.
3. Test Web Service (optional)
Feel free to change the request data (replacing ?) and invoke the web service to see what response the web service will return.
WSDL is a standard. There are many other tools that generate sample request XML. I simply use SoapUI (free version) as an example here.
Invoke a SOAP Web Service in Ruby
Now let’s write an automated test in Ruby for this SOAP service. (Previous Ruby experience is not required)
# part 1
request_xml = <<END_OF_MESSAGE
<?xml version="1.0" encoding="utf-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:x="http://www.w3schools.com/xml/">
<soapenv:Header/>
<soapenv:Body>
<x:CelsiusToFahrenheit>
<x:Celsius>10</x:Celsius>
</x:CelsiusToFahrenheit>
</soapenv:Body>
</soapenv:Envelope>
END_OF_MESSAGE# part 2
http = Net::HTTP.new('www.w3schools.com', 80)
resp, data = http.post("/xml/tempconvert.asmx", request_xml,
{
"SOAPAction" => "http://www.w3schools.com/xml/CelsiusToFahrenheit",
"Content-Type" => "text/xml",
"Host" => "www.w3schools.com",
}
)The first part is just the XML I copied from a sample request generated in SoapUI. Part 2 is a simple invocation of HTTP Post of that XML, which effectively invokes a SOAP service.
As you can see from the above, the key data we need are:
Request XML, filling values in the template generated from SoapUI
SOAPAction, can be found in SoapUI, set in request headers
Webservice endpoint.
After successfully invoking a SOAP Web service call, we get a response.
<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:\ xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XML\ Schema-instance">
<soap:Body>
<CelsiusToFahrenheitResponse
xmlns="http://www.w3schools.com/webservices/">
<CelsiusToFahrenheitResult>86</CelsiusToFahrenheitResult>
</CelsiusToFahrenheitResponse>
</soap:Body>
</soap:Envelope>Assert SOAP response
Here are two simple assertions.
# ...
expect(resp.code).to eq("200") # OK
expect(resp.body).to include("<CelsiusToFahrenheitResult>50</CelsiusToFahrenheitResult>")This test script is OK but only works for one scenario. We need it to make it more dynamic to take different inputs (i.e. parameters).
For API testing, it is usually a good idea to dump the response,
puts resp.bodywrites to output orFile.open('dump.xml', ‘w').write(resp.body)saves it to a file.
For more specific checking (XML), you will be in luck with Ruby, as the Nokogiri gem (for XML parsing) is a pleasure to use.
xml_doc = Nokogiri.parse(resp.body)
node = "//CelsiusToFahrenheitResponse/CelsiusToFahrenheitResult"
expect(xml_doc.xpath(node).text).to eq("86")SOAP Request with Dynamic Data
In the above SOAP test, the requested data is hardcoded: <x:Celsius>10</x:Celsius>. If we test with multiple sets of temperatures, we need to copy the request XML and make minor changes. This is not good as duplication is evil (hard to maintain). One approach is to use templates:
Construct a template with the structure of request XML, with placeholders for dynamic data
Generate a message from the template with defined data
Construct request messages with a template
ERB (Embedded Ruby) is a built-in feature of Ruby for generating dynamic text from templates. A template is a text document that contains values marked with <%= %> that can be substituted. Here is a sample ERB template saved in a file c_to_f.xml.erb.
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:x="http://www.w3schools.com/xml/">
<soapenv:Header/>
<soapenv:Body>
<x:CelsiusToFahrenheit>
<x:Celsius><%= @degree %></x:Celsius>
</x:CelsiusToFahrenheit>
</soapenv:Body>
</soapenv:Envelope>As you can see, the content of this template is very much like the request XML in the previous recipe, except that <%= @degree %> replaces the hard-coded template value.
Generate a message from a template with defined data
With the template defined, the code below uses the template to inject dynamic data.
template_erb_file = File.expand_path("testdata/c_to_f.xml.erb", __FILE__)
template_erb_str = File.read(template_erb_file)
@degree = 30 # changeable in your test script
erb = ERB.new(template_erb_str)
request_xml = erb.result(binding)Line 1 and 2 are quite straightforward: read the content of a template file to a string. Line 3 defines an instance variable @degree with our test value.
The key statements here are erb = ERB.new(template_erb_str); erb.result(binding), which creates an ERB template from a string and substitute
<x:Celsius><%= @degree %></x:Celsius>with
<x:Celsius>30</x:Celsius>Complete Script
require 'erb'
template_erb_file = File.expand_path("../../testdata/c_to_f.xml.erb", __FILE__)
template_erb_str = File.read(template_erb_file)
@degree = 30 # changeable in your test script
request_xml = ERB.new(template_erb_str).result(binding)
http = Net::HTTP.new('www.w3schools.com', 80)
resp, data = http.post("/xml/tempconvert.asmx", request_xml,
{
"SOAPAction" => "http://www.w3schools.com/xml/CelsiusToFahrenheit",
"Content-Type" => "text/xml",
"Host" => "www.w3schools.com",
})
expect(resp.code).to eq("200") # OK
expect(resp.body).to include("<CelsiusToFahrenheitResult>86</Cels")A big advantage of test scripts in Ruby: Synergy! You can mix this API test with other types of tests, such as E2E tests in Selenium WebDriver.
More on Dynamic
The concept of the template is very powerful. Here I will show some examples.
Random data
<UserSignUpRequest>
<Email><%= @email %></Email>
<Gender><%= @gender %></Gender>
<BirthDate><%= @birth_date %></BirthDate>
</UserSignUpRequst>With the above template, I can write the test script below to supply dynamic data:
a random new email
@email = Faker::Internet.emaila random gender from two fixed values
@gender = ["Male", "Female"].samplea dynamic but precise date, e.g. a person who is exactly 18 years old
@birth_date = 18.years.ago.strftime("%Y-%m-%d)2. Conditional Logic
We can add programming logic to the template as well. Here is an example.
<% if @vip %>
<discount>10%</discount>
<% end %>3. Loop Logic
<Vehicles>
<% @vehicles.each do |u| %>
<Vehicle>
<Make><%= u.make %></Make>
<Model><%= u.model %></Model>
</Vehicle>
<% end %>
</Vehicles>Continuous Testing
So far, I have addressed most of the shortcomings of using SoapUI except for managing test executions. This is where the raw Ruby scripting shines! You can run all the tests easily in a Continuous Testing Server, such as BuildWise.
With a click of a button, you can trigger a run of all tests. You can either monitor the test execution (the test result shows on BuildWise immediately after its completion) or do other tasks and come back to check the results later.
You can view the test failure directly on BuildWise.
You can also view test execution history which is very useful (to find out the last time it passed).
A story of how a junior test engineer learned this approach
A few years ago, a junior test automation engineer joined our team temporarily. She told me that she had developed a suite of inherited SOAP tests in ReadyAPI (an expensive version of SoapUI Pro). Then she showed me a couple of tests she had developed.
I asked: “How long did it take you?”
She answered: “About 2 days.”
I said: “I will show you a way to implement a much better version in 30 minutes. ”
She showed a sign of disbelief.
I added: “Why don’t we do it now? Moreover, for this exercise, I won’t touch the keyboard/mouse. You just follow my direction.”
Then she completed the tests and made them run in TestWise (any testing IDE or programming editor, such as Visual Studio Code would also have worked) within 30 minutes. She couldn’t believe that all the frameworks/tools were free to use, too.
I told her that running the test in a tool, such as TestWise, was not enough. A real test automation engineer shall run all tests in a Continuous Testing sever. I showed her setting up a Git Repository, adding the tests in, and creating a project in BuildWise to run them in a CT server.
This article is an excerpt from my eBook: API Testing Recipes in Ruby.