Developer and Tester Collaboration with Automated Testing
Automated Testing will help programmers fix defects early and quickly while getting along well with testers.
This article is an excerpt from my first book Practical Web Test Automation (with minor modification); Also, it is a part of my first presentation at software testing conferences (ANZTB 2010)
“It works on my machine” is one of the most popular excuses used by programmers.
It is not hard to understand why some programmers can get defensive on receiving defect reports. A programmer’s profession is to produce working code, and a tester’s reporting a defect (against a programmer) is a bit like “Your work sucks”.
Table of Contents:
· Defect Reporting: Confrontation between a manual tester and a programmer
· Defect Reporting: Collaboration with Automated Testing
· Benefits (with Automated Testing)
∘ 1. Avoid misunderstandings
∘ 2. No disruption
∘ 3. Efficient
∘ 4. Make defect-tracking unnecessary from some issues
∘ 5. No confrontationsDefect Reporting: Confrontation between a manual tester and a programmer
Imagine you are a programmer, and a tester comes to your cubicle and tells you that your work is defective in front of other fellow programmers, as a sample scenario below:
Tom: “Excuse me, Paul. User story 123: Admin user can log in failed on this morning’s build.”
Paul: “Really?” (examining the report…)
Tom: “I double-checked, just then.”
Paul: “It worked on my machine, I can show you.”
(Paul demonstrating on his development server..)
Paul: “See, it is working. Are you sure you typed the password properly?”
Tom replied unhappily: “Yes, Why don’t you try it on the test server?”
Paul: “I am in the middle of another task.”
Tom insisted: “I think it is a regression error, I don’t want to go through a defect-raising procedure. How about we can both have a look at it after tomorrow’s stand-up meeting, it won’t take long. ”
Paul: “All right.” (unwillingly)
(Tomorrow, Tom came to Paul’s desk after the stand-up. Paul tries it on the test server…)
Paul: “Hmmm …, It did fail, oh well, I will add it to the backlog.”
Tom: (Sigh)
The reason why Paul did not want to look at and fix it right away: he needed quite some time to read the defect report (which could be a feature), check the user story requirement, and replicate and confirm the issue. While this had to be done, but not directly linked to the velocity reports which (fake) agile coaches care.
(The bug-fixing itself is usually simple and easy)
Defect Reporting: Collaboration with Automated Testing
To make collaboration work, we need to avoid confrontations like the above. Automated testing can help with this. Let’s see that scenario again:
Tom: “Paul, user story 123 Admin user can log in failed on this morning’s build.”
Paul: “Thanks, I will check in a few minutes.”
(Minutes later…, Paul starts to investigate this defect)
1. Perform source update (from Git) to get the latest test scripts
2. Open the test tool TestWise, navigate to the test case for story 123
3. Run the test locally first, it is OK
4. Change the target environment to the test server
5. The same test failed on the test server
An important note here, there is no changing of test scripts for these two runs. In other words, programmers would not question the test scripts, it could be only app or infrastructure issues.
Also, when running an individual test case in TestWise, the browser will be left open. This feature is very handy for investigating the issues.
Paul: “Tom, I found the problem, and replicated the error in my local environment. It will be fixed in the next build.”
After finishing his work at hand, Paul started to work on fixing this defect. The automated test would be of great help.
- (a) run the test to replicate the issue on his local machine.
- (b) change code or configuration
- (c) deploy to the local server
- (d) rerun the test to verify
- (e) if failed, go to step (b)
- (f) run a few related automated tests to ensure no regression errors (quick and easy)
- (g) ready to push the new build to the Dev and Test server.
(moments later, Tom noticed the new build and ran the automated tests for #12)
Tom: “Paul, just let you know, the issue with user story 123 is now fixed, thanks.”
Benefits (with Automated Testing via UI)
If your project is still conducting software End-to-End testing manually, in other words, Fake Agile/DevOps, forget about silly story-point based velocity or Jira, do things actually matters. Check out How to Implement Real Automated Regression Testing? If doing properly, you get results/benefits quickly. My daughter accomplished this on her first intern job: Set up, Develop Automated UI tests and Run them in a CT server on your First day at work.
1. Avoid misunderstandings
As both programmers and testers run the same test script and see the execution in a browser, there is little room for misunderstandings.
2. No disruption
As long as the programmer is aware of which test script failed (via email or slack), the tester can continue performing other tests. Programmers can calmly finish their work at hand before addressing the defect.
3. Efficient
Programmers don’t need to read written defect reports, rather, run failed automated tests to replicate the issue in minutes or even seconds. They are much more willing to resolve the defects quickly.
4. Make defect-tracking unnecessary from some issues
Defect tracking is expensive and not effective. It is common to see a defect report (in Jira or Quality Center) with a long list of comments. With automated End-to-End tests, a large percentage of internal defect tracking is not needed.
Check out my article: Why Don’t I Use Defect Tracking? No Need, I do real Continuous Testing.
5. No confrontations
No embarrassment for both programmers (introducing defects) and testers (raising false alarms).
Related reading: