An IT Graduate’s Frustration with a Fake ‘Senior Test Automation Engineer’
Fake Test Automation Engineer is worse than useless.
This article is one of the “Be aware of Fake Test Automation/DevOps Engineers” series.
This is a true story. To protect the identity of the fake senior test automation engineer (who claimed promoting Playwright), I refer to him as B.
‘Fake’ here means a perspective for other team members, judged by the objective results. Commonly, an automated tester bragged his work or process; However, the team relied heavily on manual End-to-End testing. That’s wrong, isn’t it? Real senior test automation engineers are extremely rare (see the quotes, at the bottom. from renowned experts). A quick question: “Does a software team really trust the success of automated test execution? such as deploying the new build to production daily with high confidence.” Check out Fake End-to-End Test Automation Clarified for clarification.
A young uni graduate C, who is close to me (long-time readers of my blog probably already guessed out), joined a large tech company as an intern, C’s first full-time job. When the team leader heard C has test automation knowledge, assign C some testing tasks. The test server was not available, C found a demo site of the product and implemented two automated tests in Selenium WebDriver + RSpec using TestWise, and sent a video of execution to the team leader, on the first day.
The team leader was deeply impressed and shared the video with the team. One developer said in the group chat: “B is our department’s best test automation guy, C, you’d better have a chat with him”. C scheduled a video meeting next week with B, who seemed quite busy on Outlook Calendar.
In the meeting, B said “I promote using Playwright, and it is very good” and suggested C develop automated tests in Playwright. C asked me for suggestions. I said: “Every test automation engineer claiming using a JavaScript framework I met was a fake. Still, you shall be open-minded. Do raw Selenium in TestWise first, as it will be much quicker and more fun. After working out the test steps and stabilizing the RSpec test scripts, convert to Playwright. It will be easier and more efficient this way. Most importantly, set up running both suites (Selenium & Playwright) in BuildWise CT Server ”.
C followed my advice and implemented a total of 29 End-to-End (user story level) tests in raw Selenium WebDriver, and then converted them to Playwright (its own test runner, later with Mocha). C sent B the test script repository in the first week.
After starting working on Playwright tests, C scheduled another video meeting to show B the test scripts for opinions. In that meeting, C asked B for examples of how Playwright is currently being used. B opened a repository (for a different project) on BitBucket, which contains 19 test scripts. The last modified date of those test scripts was 4 months ago. Upon C’s request, B tried to run a few tests, and every single of them failed. B made up with some excuses.
This is remarkably similar to the story told by Michael Feathers in 2009:
It happens over and over again. I visit a team and I ask about their testing situation. We talk about unit tests, exploratory testing, the works. Then, I ask about automated end-to-end testing and they point at a machine in the corner. That poor machine has an installation of some highly-priced per seat testing tool (or an open source one, it doesn’t matter), and the chair in front of it is empty. We walk over, sweep the dust away from the keyboard, and load up the tool. Then, we glance through their set of test scripts and try to run them. The system falls over a couple of times and then they give me that sheepish grin and say “we tried.” I say, “don’t worry, everyone does.”
Later that day, C texted me: “You are right, this ‘senior test automation’ guy was a fake”.
I replied: “Ignore him, just focus on developing/refining/maintaining automated tests, in both Selenium and Playwright”.
C has set up a BuildWise CT server (local machine) to run Selenium tests in the first week and later runs all Playwright tests daily. C could provide quick feedback to the team: detected many regression errors quickly.
One day, the team leader suggested C present the work in a department-wide meeting (video conference). Besides live demonstrations, three key points:
Both Selenium WebDriver and Playwright are in the company’s proven tech stack.
Raw Selenium WebDriver with RSpec is better than Playwright in every way, and much easier to work with.
Playwright’s so-called features, such as its “automated-waiting” and “auto-retry” actually are not good or wrong. JavaScript is not a suitable language for scripting automated tests.
The director was impressed and praised C’s work. During the Q&A (and afterwards), B, ‘the automation guy’ did not say anything.
However, the story did not end here. The CT server only runs on C’s laptop, not accessible to the team. C suggested setting up a cloud-based (such as AWS) machine. I told C that you wouldn’t be able to get it. C was doubtful, a cloud VM only costs ~$40/month.
Somehow, B got involved. B wanted C to run the Playwright tests in Bamboo, the company’s default CI server.
I told C: “This is a trap. Bamboo, a CI server, is not built for running automated UI tests. Every attempt to run UI tests in Jenkins/Bamboo (I knew of) failed”.
C asked: “How should I respond?”
I replied: “Just say you have no experience with Bamboo, and you heard it is not suitable for executing automated UI tests. Given you are running Playwright tests as B suggested and he suggested Bamboo, there surely is an existing process running the tests.” (of course, they did not, as they faked test automation and CI/CD)
After C’s reply, no response.
A few weeks later, C is nearly finishing the intern engagement. As a part of the hand-over conversation, the team leader knew no progress was made on the CT server installation. The team leader contacted B, to C’s surprise, B acknowledged Bamboo’s issues with running Playwright tests. (otherwise, B would have to set up running the Playwright tests in Bamboo. The team is currently getting used to quick feedback from C’s BuildWise CT Server). The outcome of their meeting is another meeting: B, C, and another programmer in the team.
This meeting made C cry afterwards. B denied the request for a ~$40/month cloud server by saying “the Test Excellence Centre won’t allow it”. Furthermore, he expressed that the Playwright tests C wrote were not the End-to-End tests, what the department wanted were actually API tests. C was totally shocked by the big fat lie.
“Playwright enables reliable end-to-end testing for modern web apps.” — Playwright’s home page
After the intern engagement finished, C started working at one of FAANG companies (the youngest ever in the branch. A tip for ambitious young IT graduates or software engineers: learn real Automated End-to-End UI Testing, today).
On reviewing the internship, C asked me: “How could you predict all of these?”
I answered: “First of all, B might not be a bad person, just a fake Test Automation Engineer in fear. I have met many fake automated testers. B’s liking of Playwright exposed him. His busy meeting schedule confirmed that. Real test automation engineers have to do hands-on work every day, little time for meetings.”
I continued: “Test Automation is objective and quick, it tests the software as well as human beings. What is a fake’s fear? His/her incompetence and lies. You accomplished what B talked about for years, in just a matter of days/weeks. Moreover, you were using all standard and proven technologies in the company, such as Selenium WebDriver. You could, but he couldn’t. Therefore, B’s logical reaction is to sabotage and wished you to leave. Don’t you feel strange that B, had your test scripts (both Selenium and Playwright), never discussed the test scripts with you? Not any comments on Selenium or Playwright at all? That’s because he saw real test automation for the first time in his life. He didn’t want to expose his fakeness all these years. ”
I paused for a few seconds to let this poor youngster consume, then continued: “ B could not stop you from developing real automated tests (Selenium and Playwright, which happened to be the two proven automation frameworks in this company; Selenium is much much better though). If you just did one version, B might ask to do the other. But you did both, which left him no choices. He avoided talking about the actual automated test scripts or execution, even for Playwright, the not-good framework he claimed he was promoting, because he saw your real stuff the first time. However, he could prohibit the real CT available for others to see. If more people saw the BuildWise CT server, which actually runs automated test suites daily. Many questions would be raised, such as what did previous automated testers do? or Why Bamboo or Jenkins couldn’t do that? B and some others wanted to prevent that. Remember “Emperor’s New Clothes”?
That’s why I said: you wouldn’t be able to get a $40/month server to run a real CT server. I experienced this a number of times.”
For the benefits of real Continuous Testing (actual running automated UI test suites multiple times a day as regression testing) to all team members, please read this Benefits of Continuous Testing series. Then you will know why fake test automation engineers fear of that.
C was in silence. This might be too much for a young IT graduate who is about to start an IT career.
I tried to comfort C: “You did nothing wrong, as a real engineer. Engineers with real Test Automation and CT knowledge are extremely rare, you happened to learn these under the correct guidance. (By the way, C is at Level 2 of AgileWay Continuous Testing Grading , still a lot to learn). It is a pity that B did not grasp this rare opportunity to learn real test automation and CT from you. But if B did, he would be your new friend, and you could learn some from him too. Frankly speaking, it is hard for the ‘best automation guy’ in the department to learn from an intern.”
“In my experience, great developers do not always make great testers, but great testers (who also have strong design skills) can make great developers. It’s a mindset and a passion. … They are gold”.
- Google VP Patrick Copeland, in an interview (2010)“95% of the time, 95% of test engineers will write bad GUI automation just because it’s a very difficult thing to do correctly”.
- this interview from Microsoft Test Guru Alan Page (2015)“Testing is harder than developing. If you want to have good testing you need to put your best people in testing.”
- Gerald Weinberg, in a podcast (2018)