“Done, Done” in Agile
A user story is done after it is implemented by developers and verified by testers/business analysts. However, it is not “Done, Done”.
This article is one of the “IT Terminology Clarified” series.
The definition of “Done, “Done” varies across ‘Agile’ software teams.
Done Done
“We’re done when we’re production-ready” — The Art of Agile Development book
I have seen many ‘fake’ agile coaches or ‘fake’ scrum masters talk about different versions of “Done Done”, such as
“Unit tested”
“manual tester verified against the acceptance criteria”
“passed business analyst/customer review”
“the user story is marked ‘done’ in JIRA”
“the feature is deployed in production”
In my opinion, none of the above is really “Done, Done”. When I told my definition (see below) to many colleagues/friends, I don’t remember a single disagreement.
“Done, Done” for a user story in Agile
After the end-to-end automated tests passed and are verified by the business analyst or customers, and the new automated tests are included in the automated regression suite, which are run frequently (multiple times a day) in the Continuous Testing server.
— Zhimin Zhan (practised this since 2006)
The logic is quite simple. The nature of software is that many features can be easily broken by another check-in. Don’t we see it too often? If the broken features are in production, surely it is not “Done, Done”, right?
The below is an excerpt of a newspaper article on a government project, its initial budget was A$50 million, but end up costing the taxpayers over a billion. (“QLD Health payroll bungle to cost $1.25 billion”)
“Our wonderful CorpTech has done another ‘system fix’ and that has created all these other problems!!!!”, a QLD Health payroll client service officer told the reporter.
- The Courier Mail, 2011–07–23.
Why? there was no regression testing.
How to prevent unexpected broken features in production? Automated Regression Testing.
“The only practical way to manage regression testing is to automate it.” — Steve McConnell (one of the three most influential people in the software industry along with Bill Gates and Linus Torvalds) in his classic Code Complete book.
Automated End-to-End tests pass
Despite that many people agree (after all, we are doing ‘agile’), few have actually experienced a rhythm of verifying a user story by automated end-to-end tests. Their common excuses are:
“tested by automated unit tests”
“manual testers verified the user story in UI” (end-to-end)
“We have automated tests verifying the end-to-end services/API”
However, the combination of the above does not mean “automated end-to-end tests”, not at all.
unit tests are the concern of programmers. That’s why it is also called “programmer tests”. It has nothing to do with end-to-end tests.
verification by manual testers is necessary. Since it is not automated, it is time-consuming and often with human mistakes
testing that the services (SOAP/REST) and APIs are not performing end-to-end tests. Modern web apps are dynamic. An extra comma in one JavaScript file can cause the web app totally unusable.
Some readers might have figured out that all these are excuses for the team’s incompetence in test automation. As I have often addressed in my articles, there is no shame in admitting this. It is the failure of IT education and training. Only for those who pretend or fake it (not willing to admit their lacking of test automation knowledge), it is shameful.
Putting all confusing excuses aside, the correct approach is very simple. After implementing a user story and creating one or more automated end-to-end tests, run these tests while a business analyst or the customer watches the execution (in browser for web apps) and verifies it. If the result is not satisfactory, provide feedback to the developer and/or the automated tester to correct the code and/or automated test scripts.
Yes, do this for every user story.
2. Verified by the customer
Common sense. A user story needs to be verified by a business user. Normally, if an on-site customer is not feasible, a business analyst fits in the role.
3. As a part of Automated Regression Testing
Many software professionals don’t get this, not because they don’t understand the concept of automated testing or regression testing, but because they have never experienced it. Here I mean, running automated end-to-end (UI) frequently on new builds as regression testing. In other words, Continuous Testing.
Why only “Done, Done = verified automated tests in regression” makes sense?
The reason is simple: this is the nature of software development:
One small change may affect unknown parts elsewhere
changes are constant
one business feature often requires many rounds of correction and refinement
Therefore, regression testing is vitally important. For example,
user story #1 “users can log in the app after authenticating via their Google and Facebook accounts” is done (moved under the ‘Done’ column in JIRA).
Later, user story #2 “users can log in the app after authenticating via their TikTok” is also implemented and verified, i.e. Done.
However, while working on user story #2, the developer introduced a defect that caused users not able to log in to their Google account in the new build. In other words, user story #1 is no longer in a “Done” state, isn’t it?
How come my ‘Agile Coaches’ do not know this?
First of all, it is possible that your agile coaches are fake ones, quite common these days. Real agile coaches are extremely rare, and hard to find even for big companies such as LinkedIn (LinkedIn lured Scott from Google to implement the Test Automation and Continuous Testing process, see this Wired magazine’s article).
Because these fake agile coaches lack knowledge of real Agile, they tend to make up excuses and come up with fake stories to confuse poor IT executives and managers. Because so many things can go wrong in an agile project, it is quite easy for a fake agile coach/scrum master to find excuses to neglect test automation and regression testing, which could easily reveal their true ability.
On the contrary, real agile coaches take automation and continuous regression activities as the highest priority. They care little about Jira (by the way, Agile existed well before Jira), story points, velocity charts, …, etc. If you think about it, it makes perfect sense for the IT executives who care about the implemented features and quality, specifically, not breaking the existing features (i.e. no regression issues). Does the business really care about subjective velocity based on the estimated story points? one developer gave 2 points while another could give 13 (to make him more productive). The answer is no. Pathetically, most IT executives don’t have the knowledge. Therefore, they are often fooled by fake agile coaches/consultants and fall into traps.
I have seen these low acts from fake agile coaches a lot, which reminded me of this classic Seinfeld episode (Season 2, Episode 2). A simple and quick way to verify whether an ‘agile coach’ is real or not: “Show me automated end-to-end tests, show me Continuous Testing”. Some might wonder if that seems not an Agile Coach’s responsibility. Wrong, you have been brainwashed. Do you really think Jira (or alike) will deliver you software faster with high quality?
In 2005, two real agile coaches were introduced to our project. I remember right after the introduction and the first stand-up meeting, one agile coach picked up a story card (physical card) and started to work with me (pair programming) by writing automated tests. Yes, a real agile coach’s main responsibility is to help the team to implement Test Automation and Continuous Testing.
Here is a made-up story for a laugh (some parts are true though).
Tim: “You are an agile coach, right? My name is Tim, a new engineer. I have a question, I heard that test automation is the foundation of Agile, is that true?”
“Yes, absolutely. No test automation, no Agile.” The agile coach replied with confidence. “Not only that, DevOps is dependent on Test Automation. Test automation can greatly enhance a software team’s productivity.”
Tim: “Great, I have always wanted to learn test automation. Can you show me how to automate this key scenario: business receives a client’s booking?”
‘Agile Coach’: “Eh.., I have a meeting soon, I will come back to help you.”
(‘agile coach’ is researching in the office …)
“Hi Tim, it is good that you take the initiative. I recommend you use Cypress for automation. I have used it quite extensively in my past projects, very good. I sent you an email with a few links, including a couple of good video tutorials”.
Tim: “I got your email, thanks. I watched the video, it does not look hard. Are you sure Cypress is capable of testing the scenario (which you know), two browser windows side by side, simulating two different users?”
‘Agile Coach’: “Sure, I have done it many times, and run tests every night in CI/CD pipeline.”
Tim: “But I read this article: Why Cypress Sucks for Real Test Automation? (Part 2: Limitations), which has a link to Cypress's official documentation, which says:
“You cannot use Cypress to drive two browsers at the same time.”
— the offical Cypress Doc
‘Agile Coach’: “Interesting, Eh…, Can we discuss this at another time? I have another meeting coming up.”
(a few days later)
‘Agile Coach’: ‘Hi Tim, sorry I was quite busy recently. As you may know, we are under the pump. The velocity chart is not looking good. We have a large backlog, testing is well behind schedule. We don’t have time for test automation right now Also, the Agile Transformation team is considering another testing tool, anyway.”
Tim: “But you said, no test automation, no Agile.”.
‘Agile Coach’: “I said many things. The reality is that we are in the head-down bum-up mode, trying to get the release out. When this is over, I promise I will help you with test automation. It could be done.”