Track DORA Metrics
      Back to home
      1. Help Center
      2. Using Swarmia
      3. Track DORA Metrics

      Automatic change failure detection

      Automatically detect DORA change failures from application version rollbacks and pull request reverts.

      About automatic change failure detection

      When enabled, Swarmia automatically looks into every created deployment to see if it includes fixes to a previous change. When a fix is detected, Swarmia automatically records a change failure for the deployment that shipped the failing change to production.

      Currently, automatic change failure detection supports identifying version rollbacks and pull request reverts.

      Automatic change failure detection works with all available deployment sources.

      Enabling automatic change failure detection

      Automatic change failure detection is enabled by default for all new applications.

      You can enable or disable automatic change failure detection by navigating to your application in Settings / Deployments and toggling the "Auto-detect change failures" checkbox.

      Rollback detection

      Rollback detection checks if the exact same version has been previously deployed to the same environment (e.g. production). 

      If Swarmia detects a rollback, a change failure is recorded for the latest deployment before the rollback.

      Pull request revert detection

      To detect reverts, Swarmia checks if any of the pull requests included in your deployment were reverting a previously deployed pull request.

      Swarmia first looks into all merge commit messages in the base branch to find a merge commit for a pull request that was reverted through the GitHub UI.

      If none is found, we check pull request branch names (format: `revert-PULL_REQUEST_NUMBER-`) and descriptions (format: `Reverts #PULL_REQUEST_NUMBER`) for reverted pull request numbers.

      If we find a reverted pull request, we record a change failure for the deployment where the pull request was deployed. 

      Note: For applications using the Deployment API, the payload you send must include commit and repository information. This data is automatically included if you have been using the examples from our latest documentation. 

      Using issue tracker data to identify failed changes, and more advanced customization options

      Swarmia supports two additional customization options, enabled on a customer-by-customer basis. These options are not available in the UI. Reach out to us at hello@swarmia.com with the rules you're looking to enable in your organization.

      • You can identify change failures based on deployments linked to specific PRs using criteria like pull request title or branch name. Matching PR deployments will be considered fixes, marking the preceding deployment as a failure.

        See the available filtering options below:

        Screenshot 2024-07-12 at 11.47.59
      • Similarly, it's possible to detect change failures in deployments linked to specific issues in your issue tracker using criteria such as issue title, label, or custom label. Matching deployments will be considered fixes, marking the preceding deployment as a failure.

      Limitations

      • Automatic change failure detection only looks into the first 1000 commits per deployment. If your deployment includes more commits, those are not checked.
      • We record a single change failure per deployment. If your deployment includes fixes for multiple change failures, only one of them will be marked as failed.