~70% of SDLC Effort is on Software Testing and Related Activities
Wise IT Executives or Software Engineers focus on improving the testing process first.
In 2013, one architect (of a government department) told me that 70% of the budget was spent on software testing for the aged mainframe system. I was surprised. Since then, I started observing the team’s activities on various projects, and my conclusion: ~70% of all software development effort is on software testing or related activities. A clarification on the meaning of ‘related’ here: this task can be significantly shortened (and made much easier) if E2E test automation is implemented well.
New readers might think, “Zhimin, you said so because you are a test automation coach”. I worked as a full-time programmer between 1997–2010, and since 2007 I have started coding my own apps (in my spare time). The fact is that I have always been actively involved in software development (including coding).
Yes, I spent most of my time in software testing (see below for more).
Table of Contents:
· Supporting Figures (kind of)
· Testing Efforts breakdown for all team members
∘ 1. Programmers / Developers / Software Engineers
∘ 2. Functional Software Testers (manual & automated)
∘ 3. Business Analysts / Customers
∘ 4. Project Manager
∘ 5. Performance/Load Testers and Security Testers
∘ 6. UAT Testers
∘ 7. Support Programmers
· Defending effort in a Football Match
· My Personal Journey on Discovering this
Supporting Figures (kind of)
70%! Many readers, especially programmers, would disagree.
Over a year ago, I wrote a short article analysing the European Software Testing Benchmark Report 2022.
“38% think 51+% of SDLC is spent on Testing.” — European Software Testing Benchmark Report 2022
From the chart above, 12.3% think the testing effort is > 70%. There you go; about 1 in 8 agree with me.
Then readers would say, “Zhimin, even relaxing to 51% (not 70%), there are still 62% disagree with you”.
70% is a general indication, not exact, of course.
Like the famous “80/20” rule, few will question the exact 80% of all outcomes derived from 20% of causes.
For my own development projects, the testing-related activity is more like 80%. Why? Because I do daily end-to-end (via UI) regression testing.
A broader definition of testing-related tasks
I might have a slightly broad definition of testing-related activities. For example, when a programmer starts fixing a bug, typically three steps: replicate, fix, and verify. I regard Steps 1 and 3 as testing activities. Also, when a business analyst creates a data scenario, it is a testing-related activity, too.
All these become obvious when a good test automation process is implemented.I exclude useless overhead activities, e.g. meetings.
Meetings are mostly unnecessary overhead, which I exclude here. For more, check out Jason Fried’s famous TED talk: Why work doesn’t happen at work? (video), or “Meetings are Toxic” (video and transcript).
Also, when a good Continuous Testing process is implemented, the unnecessary of most meetings become apparent.I see testing efforts from multiple perspectives.
I worked in many roles: developer, architect, business analyst, tester, tech lead, project manager, support engineer, trainer and product owner. For my own projects, I have performed all activities. My view might be more complete than most people, and in a long perspective.“Release Early; Release Often” means a lot more testing effort.
The foundation of Agile is Testing, Automated Testing. DevOps means Continuous Testing.
One defect means at least two testing activities are required, one by a developer and the other by the tester (retesting), and quite often, a business analyst needs to do as well. — Zhimin Zhan
Not convinced? Let me break down how a typical software team member works daily.
Testing Efforts breakdown for all team members
1. Programmers / Developers / Software Engineers
50% — 80%
Generally speaking, programmers are considered the stars of a software team. The software they created ultimately is why all teams get paid.
I started working as a software engineer in 1997 and worked on numerous projects (Perl, C++, Java, C# and Ruby), mostly as a contractor. I would say that a large percentage of my time (when working as a programmer) was on testing.
To set a tone, let’s look at the two joke images.
There is some realness in the jokes.
In this article, Benefits of Continuous Testing (Part 4: to Developers), I listed all the typical activities a programmer does daily. Over 50% is end-to-end software testing. For more, check it out. Here, I will just point out the simple fact: defects and debugging are all associated with testing.
If we include unit testing and integration testing, the figure is even bigger.
2. Functional Software Testers (manual & automated)
100%
Needless to say, except for team meetings, all their remaining time is spent on software testing.
3. Business Analysts / Customers
40% — 70%
Like manual testers, business analysts use the app (via UI) a lot for the following purposes:
verify the implementation (actively testing)
create certain data scenarios (passive testing)
navigate to a particular state in the app (passive testing)
Once a good E2E test automation is implemented, the BA’s involvement with testing becomes obvious. Check out this article.
4. Project Manager
20% — 40%
While a project manager does not typically allocate 30% of his time to software testing, his involvement with testing is spontaneous, besides usual testing work allocation.
How often have you heard a project manager say, “There is Severity 1 defect; everyone stops the current work to focus on that”? Quite often, right? This is testing-related.
Have you heard the term “Defect triage”, a process where each bug is prioritized based on severity, frequency, and risk? Obviously, the project manager needs to be actively involved. One senior project manager told a figure, “15% of his time was spent on Defect triage”.
5. Performance/Load Testers and Security Testers
100%, but usually engaged with limited time.
For apps that achieving high performance is the top priority, non-functional testing effort can be quite big.
6. UAT Testers
100%, but engaged with limited time.
Often, during the UAT testing, a fair percentage of team members (business analysts, developers and testers) will be assigned to assist during that period.
7. Support Programmers
80–90%
Once the app is released (to the customer), then for any activity onwards, e.g. bug fixes, change requests and new feature implementation, the testing effort absolutes dominates.
Please note, for a well-implemented software app, the supporting phase is far longer than the development phase.
Overall, 70% is about right. Do you agree?
Some may agree with me after reading here. I heard people have used the terms “attacking” for coding and “defending” for testing, which I agree with. In software development, Good “Testing” prevents many regression errors.
“The good fighters of old first put themselves beyond the possibility of defeat, and then waited for an opportunity of defeating the enemy.” — Sun Tzu, Art of War.
Defending effort in a Football Match
Football (soccer) is the most popular sport in the world, and a lot of research has been invested in it. While the gameplay is dynamic, every player is assigned a primary responsibility: Attack or Defend.
One question is, “What is the overall defending effort in a football match?”
Below is an illustration of the most commonly used (and known as balanced) 4–4–2 formation:
2 Forwards (also called strikers) in the front.
4 Midfielders
4 Defenders in the back.
Please note that we are talking about the primary effort here (I know, sometimes, strikers do defend; defenders do attack). Midfielders can also be classified into “attacking midfielders” and “defensive midfielders”.
So, six players (two of four midfielders + 4 defenders) are defending. Wait, there is another one, the goalkeeper, 100% on defending.
So, the overall defending effort: 7/11 = 64% .
When I explained this to some colleagues and friends (IT workers), most were surprised by the final figure, but that’s an unquestionable fact.
People tend to remember famous Soccer stars, such as Lionel Messi and Cristiano Ronaldo, who are usually strikers. They got the most attention, just like programmers in a software team. Defenders usually get neglected, just like the software testers. However, sports professionals do value defending highly.
My Personal Journey on Discovering this
In 2005, I was extremely fortunate to work with a world-class programmer (as a mentor) for six weeks. That experience completely transformed my view of software development and programming. I did not have many realizations, such as this one, back then. I just knew E2E Test automation is very useful from many perspectives, so I practised every night and weekend. About one month later, my colleagues viewed me as a different programmer, with ~5X productivity (the PM had measurements of User Story Points).
A few years later, with boosted confidence, I started to develop my own apps, two of which reached the world stage. The secret: I invested my effort wisely, ~80% in the testing, mostly E2E UI functional testing (using Selenium WebDriver + RSpec).
So, if you are a wise CIO or an ambitious software engineer who wants to build your own apps (or even Micro-ISV), start with learning E2E Test Automation.
In the next article, I will explore whether this (70%) applies to manual testing or test automation differently.
Related reading: