Why I Prefer an On-Premise Continuous Testing Lab Over the Cloud? – Part G: Security Concerns
An often neglected and regrettable consideration.
In this article series:
Part A: Introduction
Part B: Significant Cost Savings
Part C: Superior Performance
Part D: Higher Reliability *
Part E: Greater Flexibility
Part G: Security Concerns
Part H: Advice and FAQ (upcoming)
In this article, I highlight a key advantage of using an on-premise Continuous Testing (CT) lab: significantly reduced security concerns. These concerns typically fall into two main categories:
The risk of exposing business-sensitive data embedded in end-to-end (E2E) test scripts.
Inability to execute certain tests due to network access restrictions.
Unfortunately, many software architects overlooked this when opting for a cloud-based solution.
Security Concerns of E2E Test Scripts outside the business
As we all know, end-to-end (E2E) test scripts can also function as a form of documentation. However, these scripts may include two elements that could raise security concerns:
Business Flows
While customer-facing business flows are usually not an issue, backend processes might involve sensitive workflows that some businesses prefer to keep confidential.Test Data
It’s not uncommon for careless testers to leave behind test data—such as authentication credentials—that remain valid for production servers, posing a potential security risk.
Let me clarify that I am not questioning the integrity of Cloud Testing Platforms—not at all. About 11 years ago, I worked on a project where the State Education Department transitioned school email systems to Microsoft Office 365. This involved a lengthy and costly process to determine whether student data could be hosted outside of Australia.
Frankly, if you are a middle manager/director and stuffed up on this issue, you are screwed.
Access to the Main Test Server Instance(s)
Suppose you have chosen a cloud testing platform to execute your automated E2E tests. Which server(s) do those tests execute against? A dedicated public-accessible server?
In reality, many software teams are hesitant to make their testing server instances publicly accessible. This means extra infrastructure/security work is required to run your E2E tests, which are hosted on the cloud (a third party), against an internal server.
Access to Supporting Services
Often, an E2E test script relies on supporting services, which were configured for internal access only. For example, it might involve placing a batch file into a specific network folder and waiting for an asynchronous queuing service to process it. When test execution occurs outside the company network, these dependencies can become problematic.
Related reading:
