Why do CIOs fear “Test Automation” as much as they want it?

They wanted it but had too many bad memories.

Table of Contents:
· 1. Desire:  for “Release Early, Release Often”
· 2. Reality: real test automation engineers are extremely rare
· 3. Politics: it is hard to acknowledge being fooled
· 4. Solution: easier than you think

A couple of years back, I was working at a large local financial company as a test automation consultant. My first week's work brought some attention, and a test lead of the main division invited me to a meeting along with my manager. I heard in that meeting that the last thing the CIO wanted to hear was test automation. He said: “The last test automation attempt cost $2 million (mostly the human cost, plus some software license fees), with absolutely nothing to show for”.

I heard a similar comment about a departing CIO by an architect. “He wanted test automation implemented, and he trusted IBM’s Rational Functional Tester based on a Gartner report”.

It is not CIOs who don’t want “test automation”. They really want it. A CIO would dream about test automation success like the one in LinkedIn:

• “spiking profits” in a short time; “stock has more than tripled in less than two years”

• might be featured in Wired Magazine or CIO.com

However, they have been hurt too many times. Consequently, test automation attempts are often associated with “failure”, “expensive”, “fake”, and “embarrassment”.

By the way, this 2000 years+ old idiom story explained this behaviour, in a direct and interesting way.

To understand why, there are three factors:

• Desire: for “Release Early, Release Often”

• Reality: real test automation engineers are extremely rare

• Politics: it is hard to acknowledge being fooled

1. Desire: for “Release Early, Release Often”

CIO, even in a small company, means authority in technical direction. Most CIOs like to paint a big and better future on stage. “Release Early, Release Often” is certainly the most important part of a CIO’s perfect world.

• “Facebook Release Twice a Day” (source: CNET and Selenium Conference)

• LinkedIn “pushes out updates multiple times per day” (source: Wired’s article, The Software Revolution Behind LinkedIn’s Gushing Profits)

• and many articles like this on CIO.com

Even a mediocre CIO would know that the testing process costs the most in the whole SDLC (and further maintenance phase). Test Automation is surely the solution.

As we all know, once a boss has a desire, some people will try to fulfil that desire for benefits. Unfortunately, in the context of test automation, the actions are common among due to incompetent/bad people. (don’t worry, I will show you a simple yet effective solution towards the end of this article).

2. Reality: real test automation engineers are extremely rare

Here are some quotes from renowned experts:

“In my experience, great developers do not always make great testers, but great testers (who also have strong design skills) can make great developers. It’s a mindset and a passion. … They are gold”.
- Google VP Patrick Copeland, in an interview (2010)

“95% of the time, 95% of test engineers will write bad GUI automation just because it’s a very difficult thing to do correctly”.
- this interview from Microsoft Test Guru Alan Page (2015)

“Testing is harder than developing. If you want to have good testing you need to put your best people in testing.”
- Gerald Weinberg, in a podcast (2018)

The message is clear, real test automation engineers are extremely rare, even for tech giants such as Google, Microsoft and LinkedIn. According to The Software Revolution Behind LinkedIn’s Gushing Profits, LinkedIn’s success is traced back to successfully luring Kevin Scott from Google. Yes, “Lure” (the exact word used in Wired’s article).

I have visited Silicon Valley, seeing many young people wearing Google and LinkedIn T-shirts there. If you watched the movie “The Internship”, you will know that many of these employees are top graduates from prestigious universities such as Stanford and UC Berkeley. According to Alan Page (the main author of How We Test Software at Microsoft), 0.25% of them (we are talking about this in the context of Microsoft engineers here) can write good GUI test automation.

So, what is the chance you can work with a real senior test automation engineer in your life? Nearly 0%.

3. Politics: it is hard to acknowledge being fooled