Unit Testing Clarified
Only programmers do unit testing, which is no concern to software testers (QA team).
This article is one of the “IT Terminology Clarified” series.
In software testing, there are many terminologies that are used wrongly by IT professionals. In this article, I will talk about ‘Unit Testing’, which is misused a lot in software projects.
Unit testing is a type of white box testing performed by programmers at the source code level. Software testers (QA team) should have no concerns about it. A more correct term for ‘Unit Testing’ would be “Programmer Test” (I didn’t invent this term, I read it from a book. Pitifully, I forgot the name of that book).
Last week, an IT manager showed me their release document, which has “QA unit testing” (by manual testers). I pointed out the error. This manager was quite open-minded and corrected it immediately: “QA functional testing”. I have seen this mistake many times; however, not everyone was willing to admit that. Though I don’t remember anyone arguing on this (the mistake was so obvious), a common response from the tech leads: “put aside the differences of the terminology, blah blah blah, …”. Frankly, I have lost interest in listening to a tech lead who does not even understand this basic stuff. It is pointless to discuss Test Automation, CI/CD, or Continuous Testing with him.
What is a Unit Test?
A software product that passes comprehensive programmer tests (i.e. unit tests) can still fail on many functional tests. That’s because unit tests are from a programmer’s perspective, and functional tests are from a user’s perspective. A unit test is a type of automated test. You must have heard of the xUnit framework. JUnit, created by Kent Beck and Erich Gamma, kickstarted the unit testing revolution in 1998.
“Never in the field of software development have so many owed so much to so few lines of code” — Martin Fowler, on “xUnit Test Patterns” book.
However, many programmers do unit tests poorly. Michael Feathers, the author of ‘Working Effectively with Legacy Code”, wrote a famous post “A Set of Unit Testing Rules” in 2005:
A test is not a unit test if:
It talks to the database
It communicates across the network
It touches the file system
It can’t run at the same time as any of your other unit tests
You have to do special things to your environment (such as editing config files) to run it.
Some programmers would think: “How could that be possible?” That was how I thought when I first saw this post. Luckily, I had an opportunity to pair-program with a world-class programmer later. On the first day, he pointed out that my unit tests were not unit tests at all, “the tests you wrote restarted EJB bean. They are not unit tests”. Then he wrote some real unit tests that totally blew me away. Michael Feather’s unit testing rules are correct. For example, for unit testing DAO (data access object) classes, you can use an in-memory database.
Differences between a unit test and a functional test
I know the audience of my blog is mostly interested in functional testing. There are three clear factors that distinguish unit tests from functional tests:
Who does it?
Programmers, and only programmers, write unit tests. That’s why unit tests are also known as ‘programmer tests’.Scope
Individual function/method in source code.How long does it take to run?
To help you understand the difference, I come up with a 30X factor rule (not based on science, but rather a guide for myself when working on these three kinds of automated tests).
A programmer shall always run and pass all the unit tests before he checks in the code. It is possible as the execution of good unit tests is fast. I remember that in 2006, my team created ~1800 JUnit tests (mostly done by me) which took about 6 minutes to run. There was room for improvement ( Computers at that time were quite slow).
Q & A
1. Why do the tech leads/managers/programmers use the ‘Unit Test’ term wrongly?
Managers are excusable, but not for ‘tech leads’ and programmers. The short answer is that these tech leads have never written or executed real unit tests. This is very concerning. We know that “Good programmers write unit tests”, and the T in TDD (Test-Driven-Development) is Unit Testing. If we have these fake/incompetent engineers as the tech leads, the project will be doomed.
Let me share an interesting story. I worked as a contract programmer (C#) on a large government project many years ago. I started writing unit tests (a practice I took from years of coding in Java) using NUnit. Soon, I used NMock for mocking in unit tests. One day, the tech lead came to my desk and told me to “Delete the unit tests!”. Yes, you heard it right. His reason was, “You are using technology (NMock) that our future support programmers are unable to maintain”.
I was shocked. I emailed him with a quote from MSDN: “NMock can greatly enhance the design” and asked him alternate ways for mocking in unit tests. The reply was simple: “Delete the unit tests. Here is check-in task ID.” (at that time, every check-in required a task to be associated with). I wished I took a screenshot of that email, but I did show it to some people. They thought it was crazy too.
Clearly, this was a human issue, but it is fundamentally about the team programmers who do not understand unit testing and have never done it properly. Now that Test Automation, CI/CD/CT, and DevOps are hot topics in IT, these incompetent engineers want to get involved ….
2. Do you do unit tests?
I retired from programming as a daytime job in 2011. I switched to working on Test Automation and CT during the day (for others) and developing my own software in my spare time (for myself). Before that, I followed TDD (writing unit tests first) since I mastered effective unit testing in 2006 (learned from a master) and asked my team members to do it as well.
However, as I was getting more efficient with automated E2E testing (using TestWise: next-gen functional testing IDE and BuildWise CT Server/Agents, disclaimer: I created these tools), my focus switched to “Daily release to production”. I implemented most (>95%) of the customer requirements the next day (I will write an article on this soon).
This means I need to spend my time wisely. I focused my testing effort mostly on automated E2E (functional) testing. Still, I would write unit tests first for a feature with complex logic, such as workflow.
3. How do you learn to write better unit tests?
Many skills in software engineering, such as automated testing (unit/functional/load) and CI/CD/CT, are rarely taught at university. As you have probably found out already, very few did well despite almost all software projects claiming to have been conducting these practices.
My solution: find a real professional (colleague/consultant/coach), pair-program with him/her.
4. I found writing some unit tests simple but hard for other functions.
This mostly means that your software design is not optimal. Writing unit tests is not simply verifying a function. More importantly, in my opinion, it is to verify your software design. Generally speaking, it is easier to write unit tests for well-designed code. For example, back in my Java days, writing unit tests for the Spring framework was a lot easier than EJB.
Why should software professionals do automated testing (unit and functional)?
Many developers write automated unit tests unwillingly. Because “Good programmers write unit tests” is well accepted in the software industry, programmers who do not have the skills often try to write some unit tests and soon give up. Besides technical aspects, I think software engineers and IT managers should first upgrade their understanding of test automation's value.
In my opinion, the reasons for doing automated testing (unit & functional) are:
to provide benefits to the customer
customers like frequent and high-quality releases. With a happy customer who trusts the team’s capability, everything is a lot easier.to provide quick feedback to the team
If we can find out regression issues earlier, it will be much easier and cheaper to fix. The feedback from unit tests is quicker than E2E functional tests. Given automated E2E regression testing is in place, effective unit testing is always welcome as it provides much quicker feedback. To achieve it, you need a capable team to be able to design and maintain a good unit test suite.
With a green build (passing all automated unit and E2E functional tests) every day, programmers can go home with pride. This is priceless.