Why Auto-Retry of Test Execution in a Test Framework is Wrong?

Auto-Retry shall be done at CT server, not messing up with the test scripts.

Recently, I noticed the ‘Auto-Retry’ feature in some ‘new test frameworks’, such as Jest and Cypress. It is wrong! I pioneered (and implemented) the ‘auto-retry’ back in 2007 and have been using it ever since. The difference is that my ‘Auto-Retry’ is in the Continuous Testing Server, not the automation or syntax framework. BuildWise, the CT server I created with the Auto-Retry feature, received Ruby Award from Matz (the creator of Ruby, a true software legend).

“Infrastructure failures get retried” — from an excellent CI presentation “CI at Facebook”

Let’s see a sample ‘auto-retry’ syntax in the Jest test framework:

jest.retryTimes(3) //set maximum retries number

describe('Test', () => {
    test('Test_Test', async () => {
        //test code
        ...
    })
})

First of all, I am sure most will agree that adding retryTimesto a test framework does not sound ‘right’. Still, why is it bad?

1. It does not address the root problem.