Make the world a better place by adding ‘End-to-End (via UI) Test Automaton’ in the software contract
It should have always been the case. The word “Engineering” has meanings in other industries, but sadly, not in the software industry.
In a previous article, I listed some news headlines about a big software failure (run by IBM) in my state. The state government dared not to sue IBM for compensation.
Quite the opposite, IBM grabbed a lot more money by delivering a bad system. Clearly, Law-Suit seems not viable, at least in this case. I can understand that state government lawyers are not a match for the big guns that software giants engage in. The escaping causes might have been written skillfully in the contract, before the project started. Basically, they have “Jail-Free Cards”.
Table of Contents:
· How can governments and organizations prevent software disasters?
∘ A Simple and Practical Advice
· How Does it Work in Practice?
· FAQ
∘ 1. Why End-to-End (via UI) Test Automation?
∘ 2. Why Regular Check (every sprint)?
∘ 3. Will this incur more costs?
∘ 4. I think few software vendors can provide daily automated execution report
∘ 5. What will happen if the vendor can’t accomplish this after Sprint 2?
∘ 6. What will happen if more and more customers require these clauses in the software contracts?
How can governments and organizations prevent software disasters?
There is one simple and logical thing that government can do to protect from the shenanigans. The answer is “End-to-End (via UI) Test Automation”.
The root problem is software testing, as stated in the auditor-general report for QLD Health Payroll failure. That’s the thing the customers can control, and the vendors find it hard to debate or fuss.
A Simple and Practical Advice
My Advice to the Software Customers (mainly the government, because it uses my money too, as a taxpayer), add this simple cause to the contract:
All User Stories (with user as the actor) must be covered at least one automated end-to-end (via UI) test cases.
A Proof of End-to-End (via UI) Test Automation Report is required, for each sprint, to claim progress payments.
Basically, customers associate “Ongoing payments with Ongoing Quality Checks”. The software vendors care only about the money, right? That’s the only button we can push to make them change.
Software vendors may find excuses but won’t disagree. Why? It is pretty much in the definition of Agile.
How Does it Work in Practice?
The contracting part is easy, ask your lawyers to reword my two sentences nicely, adding them as two clauses.
In term of how to perform checking automated UI tests, it is quite easy and cost a little (resources: time and money).
A typical sprint (2 weeks) looks like the one below.
Before the last day, most work should have been completed. A customer asks the vendor to supply:
A testing server environment
A daily execution report of the whole end-to-end test suite
Test scripts
Then you assign two or three staff (who will be assigned as UAT testers, this role is called “On-Site Customer” in AgileWay) to execute automated tests one by one. This is much easier and quicker than most thought.
Firstly, I will show you an execution report of my app WhenWise’s end-to-end regression suite, which consists of 560 raw Selenium Ruby tests.
The whole test execution time of 560 end-to-ends takes 3 hours and 10 minutes (executed sequentially on one machine)
Please note a test case name is a business scenario in plain English that on-site customers do understand.
Keep reading with a 7-day free trial
Subscribe to The Agile Way to keep reading this post and get 7 days of free access to the full post archives.