Four Screening Questions to Filter Out Fake Test Automation Engineers in Interviews

Hiring a fake automation tester can be costly. These four simple screening questions may help prevent that.

A repost of my past article on Medium (2022)

🥇 1st most popular article of the 127th issue of Software Testing Weekly.

A job interview aims to determine whether the candidate is suitable for the role. The interview time is short, therefore, choosing high-quality screening questions is important. Here are the famous 3-questions in “The Walking Dead”, a popular post-apocalyptic TV show.

Image credit: https://www.pinterest.com.au/pin/549861435731348085/

Rick [and later on other members of the group] started asking potential newcomers three questions as a kind of screening process:

1. How many walkers have you killed?
2. How many people have you killed?
3. Why?

Some fans even made a T-shirt with them and sell on eBay.

Table of Contents:
· Screening Questions shall be Simple
· My four simple Screening Questions for Test Automation Engineers
∘ Question 1. How many automated UI tests did your last project have?
∘ Question 2. How long does it take to run these tests?
∘ Question 3. How often does your team run them?
∘ Question 4. How do you run them?
· What are the typical outcomes?
· What are the correct answers then?
· FAQ

Screening Questions shall be Simple

Here is another famous simple question (during the 2008 US presidential election), this time, real (you can hear, clicking the link below):

“How many houses do you and Mrs. McCain have?”

“I think — I’ll have my staff get to you,” John McCain, 2008 US presidential candidate in the interview with The Politico, August 20, 2008

Good screening questions are simple, easy to understand, and difficult to talk around.

In one “Yes Minister” episode (UK Comedy show), the minister complained that “kids asked difficult questions”. He really meant simple but hard-to-answer questions.

I have interviewed many automated testers, and frankly, most of them were not qualified. That’s not their fault, really. Where can software engineers/testers learn test automation (not taught at Uni)? Few software professionals witnessed one successful Test Automation. In 2005, I was extremely fortunate to pair program with a world-renowned Java programmer, from whom I learned real test automation and CI (see My Test Automation Journey).

My four simple Screening Questions for Test Automation Engineers

1. How many automated UI tests did your last project have?

2. How long does it take to run these tests?

3. How often does your team run them?

4. How do you run them?

Obviously, these questions are not applicable to graduates. For graduates, focus on personal attributes (which I will cover later).

Question 1. How many automated UI tests did your last project have?

The candidate is unable to avoid this simple question. Therefore, I often heard extreme answers.

• Five or fewer tests
  I think the candidates were caught unprepared and told the truth. However, many of them quickly realized the answer was no good and started to patch it up with all kinds of excuses.

• A big number, such as 7000 (the highest I heard: 20,000) automated UI tests
  Lies, clearly. The next question is waiting …

Most would answer between 100–500. However, these were lies, or the candidate did not understand real test automation. Anyway, it will be revealed shortly.

One common excuse in recent years: “I do API testing, not UI Testing”. It is wrong.

1. First of all, “Automated End-to-End UI Testing, preferably Selenium WebDriver” was clearly stated.

2. API Testing, compared to Automated UI Testing, is much less challenging (also much fewer benefits to the team). In my eyes, the candidate who answered ‘API Testing’ is like mentioning his/her high-school certificate when applying for a Professorship.

3. Functional Testing, in its most intuitive and practical form, is testing from the user’s perspective, i.e., via UI. (A recent year’s World Quality Report stated that)
   It is fair to require any automated tester (as a profession) to master a basic level of UI Test Automation, such as Level 2 of AgileWay Continuous Testing Grading.

Question 2. How long does it take to run these tests?