Change failure rate

Change failure rate is the ratio of failed deployments to all deployments.

Summary

Change failure rate is one of the DORA metrics and a key deployment health metric in Swarmia. The exact definition of a change failure is up to you. As a rule of thumb, it should be an incident that must be remedied immediately instead of waiting until the next regular deployment. If a deployment introduces a bug that doesn't need an immediate reaction, it probably shouldn't be defined as a change failure. Failed deployments (ie, when a deployment doesn't reach production) are not change failures either, at least as long as they don't cause a production incident.

Swarmia uses deployments as the basis for change failures. Deploys that fix other deploys (eg, revert, rollback, hotfix) mark the original deployment as a failure.

Change failure rate = change failures / total number of deployments

Example

If you completed 20 deploys in a week, but 5 of those led to a change failure, you would have a change failure rate of 25%.

Why it matters

A high change failure rate can indicate either an issue in your quality or deployment systems that needs to be improved.

How to use it

If you see an increase in your change failure rate, you might want to dig into the root cause with your team to help alleviate any issues in delivery.

Where to find it

You can find the change failure rate (as well as the other DORA metrics) under Metrics / DORA or Infrastructure / Deployments.