"Testing in Production" is bad, Why is it rather common?

Mainly, lack of capability.

“Testing in Production” is rather common in software projects. Let’s first see some memes.

Source: https://x.com/_workchronicles/status/1904571940506910806

Source: https://www.ministryoftesting.com/memes/39

"Testing in production" is undoubtedly a bad software development practice, leading to defects, low morale, high expenses, data corruption, project failures, and more. Yet, it is rather common. I was personally involved in an early stage of my career as a senior Java software contractor developer before mastering end-to-end test automation.

By the way, "Testing in Production" is almost exclusively done manually.

Before diving into the reasons and solutions, I have a question for readers to consider: "How would your senior or principal software engineers, or solution architects, react if someone said, 'We are testing in production'?" Ashamed? Embarrassed? Feeling incompetent? Or a combination of all three? That seems like the expected reaction, right? However, in reality, I’ve met several principal-level software engineers whose response was simply: “It is not optimal, but acceptable.”

Of course, it is NOT OK, as depicted in the above three Memes.

The job titles are not equal to capability. Those “principal” software engineers could even get a junior role in my company, simply they are no good at all. Why? It violates the basics of software engineering, so, not qualified to do software engineering work.

However, to be fair, it's not entirely the individual's fault. The education system shares some of the blame as well—very few universities offer courses on test automation. And software companies rarely seek external instructor-led training to grow the staff.

Why "Testing in Production" is So Common?

In theory, even a junior software engineer knows “testing in production” is wrong. So, it's not a matter of misconception but simply a lack of capability. The app was never designed to allow verification of most of its core features in test environments. Therefore, ‘software engineers’ deploy it to production and perform testing there.

Here, I will point out, that lacking the core infrastructure is usually not the problem nowadays. Why? It is quite easy (and quite affordable) to do a cloud infrastructure setup.