LogoLogo
Book a demoLog in
  • Swarmia documentation
  • Getting started
    • Get started in 15 minutes
    • Integrations
      • GitHub
        • GitHub Enterprise Server
        • Multiple GitHub organizations
        • Forked repositories
        • Troubleshooting
          • Reinstalling the Swarmia GitHub app
          • Updating app permissions
          • Installing Swarmia outside of GitHub Marketplace
      • Jira
        • Jira Server and Jira Data Center
        • Multiple Jira organizations
      • Linear
        • Private Linear teams
        • Disconnect Linear
      • Slack
        • Private Slack channels
      • Authentication
        • Google Single Sign-On
          • Frequently asked questions
        • Okta Single Sign-On
      • HR systems
      • Data export
        • Data cloud
        • Export data as a CSV file
      • Other integrations
        • Other issue tracker integrations
        • Other source code hosting integrations
  • Configuration and data quality
    • Teams & members
      • Creating & managing teams
        • Teams API
      • Contributors
      • Roles and permissions
      • Inviting team members
    • Issue tracker configuration
      • Jira configuration
      • Jira best practices
      • Linear configuration
    • Pull request exclusions
    • Linking pull requests to issues
    • Investment categories
    • Deployments
      • Generate deployments from merged pull requests
      • Generate deployments from GitHub deployments
      • Generate deployments from GitHub checks
      • Generate deployments via the API
        • Generate deployments for monorepos via the API
    • Sprint configuration
  • Use cases
    • Improve pull request flow
      • Pull request insights
      • Reducing pull request cycle time
      • Review code faster
      • Managing pull requests in progress with the Pull Request view
      • Diagnosing low pull request throughput
      • Analyzing pull request batch size
  • Improve your team's focus
    • Optimizing issue cycle time
    • Analyzing activity patterns on Work Log
    • Grouping activity on the Work Log view
    • Focus summary
  • Balance engineering investments
    • Activity and effort-based models
    • Categorizing work
    • Common problems with balancing engineering investment
  • Deliver strategic initiatives
    • Forecasting initiatives
  • Capitalize software development costs
  • Run developer experience surveys
    • Creating a survey
    • Managing surveys
    • Viewing and sharing survey results
    • How we show your survey responses
    • Survey communication guide and templates
  • Track DORA metrics
    • Automatic change failure detection
    • How Swarmia links PRs to deployments
  • Coach software developers
  • Get visibility into your CI pipeline
  • Continuous improvement
    • Working agreements
  • Notifications
    • Team notifications
    • Personal notifications
  • Retrospectives with Swarmia
  • Metrics & definitions
    • Pull request cycle time
      • What's the difference between "Change lead time" and "Pull request cycle time" metrics in Swarmia?
    • Issue cycle time
      • Defining issue lifecycle and cycle time
    • Developer effort (FTEs)
  • DORA metrics
    • Change lead time
    • Deployment frequency
    • Mean time to recovery
    • Change failure rate
  • Throughput
  • Time to deploy
  • Batch size
  • Flow efficiency
  • Scope creep
  • Sprints
  • Frequently asked questions
    • How do you treat weekends in metrics?
    • Tracking squashed commits
    • How do merge queues affect my metrics?
    • Why is my commit not visible in Swarmia?
    • How do I account for people leaving my organization?
  • Resources
    • Security & data retention
      • Data security
      • Data access
      • Swarmia IP Addresses
      • Single Sign-On (SSO) / SAML
      • Can I get a copy of the SOC 2 Type II audit report?
      • Deleting your organization
  • Pricing & plans
    • Compare plans
    • Free plan
    • Do I need a credit card to start a free trial?
    • What are the differences between the individual modules and the standard plan?
    • How do you determine the number of developers for billing?
    • What happens to customers with the Lite plan after the December 2024 pricing and plan change?
  • Changelog
On this page
  • Getting started
  • What you can measure
  • Deployment Frequency
  • Change Lead Time
  • Time to Deploy
  • Change Failure Rate
  • Mean Time To Recovery
  • How deployments are attributed to teams
  • About deployments and change failures
  • Automating change failures
  • Manually indicating change failures

Was this helpful?

Track DORA metrics

Analyze deployment frequency and quality across your entire organization in Swarmia.

PreviousSurvey communication guide and templatesNextAutomatic change failure detection

Last updated 27 days ago

Was this helpful?

How often do you deploy code to production? How many of these production deploys cause unintended defects, as experienced by end-users? How long does it take to fix such defects?

Swarmia's DORA and deployment insights help you answer these questions and analyze their trends across your organization (learn more from ).

Tracking DORA metrics and their trends per team helps you assess both the frequency and quality of deployments. It provides teams with a tool for lowering their batch size and understanding the impact of their deployments.

Getting started

Start by setting up applications and configuring the way Swarmia tracks production deployments.

What you can measure

Deployment insights let you both measure your deployments and visualize their trends. Besides looking at the volume of deployments, you can track which deployments had defects. This helps you understand the quality aspect of deployments.

Deployment Frequency

Deployment frequency is used to indicate the performance of a software development team. Ultimately, deploys to production are what make any work visible all the way to the user – or what makes it possible for your work to deliver a business impact.

According to the authors of Accelerate, “elite” teams are able to deploy new code on-demand or multiple times per day, whereas the release frequency of high-performing teams is between once per day and once per week.

A low deployment frequency can be an indication of working with large batches, or signal other problems such as poor deployment infrastructure or lack of reliable automated tests.

Change Lead Time

Measures the time it takes for pull requests to go from the first commit to a production deployment. Helps you identify wait times and bottlenecks in your development process.

According to Accelerate, “elite” teams can go from a code committed to production in less than an hour, while for high-performing teams, it takes between one day and one week.

High change lead time can indicate too large batch sizes, slow code review/QA, or long CI/CD wait times.

Time to Deploy

Measures the time it takes from pull request merge to deployment. Part of change lead time.

Useful for understanding, how much delay your current deployment process is causing in your deployment process.

Change Failure Rate

The exact definition of a change failure is up to you. As a rule of thumb, it should be an incident that must be remedied immediately instead of waiting until the next regular deployment. If a deployment introduces a bug that doesn't need an immediate reaction, it probably shouldn't be defined as a change failure.

Swarmia uses deployments as the basis for change failures. Deploys that fix other deploys (eg. a patch, hotfix, rollback, forward fix) mark the original deploy as failure.

We look at the number of such failed deployments and calculate the Change Failure Rate by comparing this to the number of total deployments.

Mean Time To Recovery

Mean Time To Recovery (MTTR) is the average time it takes to address change failures. It helps teams understand how quickly they're able to resolve issues.

Time To Recovery (TTR) can be determined for each failure as the time between the original deploy, and the fix for the problem. TTR can be used to understand the impact of each change failure (how long did the problem last, and what was its impact to the customer).

How deployments are attributed to teams

Note that one deployment can be linked to multiple teams if it has multiple pull requests or an author who belongs to multiple teams.

About deployments and change failures

We define a deployment as any change that was applied to your production code. In other words, you should only include deployments that were processed successfully, resulting into some sort of change in your production application. This can also mean deploying a version that was live previously.

Change failures are deployments that have defects that somehow impact the user of your production application. Some typical examples are regressions, downgraded performance, or other types of bugs that impact the user.

There's no one definition for failures, but we offer some guidelines. The purpose of measuring failures is to understand when a team is moving too fast, to act as a feedback signal to slow down. You want to proxy the impact to users, so focus on production changes that are affect them.

To capture change failures, Swarmia forms links between deployment events to understand when a deployment is seeking to fix a previous production deployment. These targeted deployments will be considered failures.

Automating change failures

Manually indicating change failures

You can use the to mark a deployment as a fix for an earlier deployment. In addition to this, you can navigate to and mark a deployment as fix manually.

The Authors column in the table lists the authors of the. The Team and Author filters on the page are based on the authors. These are then aggregated to calculate DORA metrics per team in . This lets you see DORA metrics for every team in your organization.

Swarmia automatically generates deployments for some of your repositories. These applications and production environments can be configured further in .

Swarmia can from rollbacks or reverted pull requests. This is enabled by default for new applications and is the easiest way to get started.

In addition, you can use the to mark a deployment as a fix for an earlier deployment. By integrating it into e.g. your forward-fixing process, you can maximize the quality of change failure data and get reliable insights on the quality of your engineering process.

Alternatively, you can mark a deployment as a fix manually in . We fetch previous deploys for quick access, or you can find deployments from the past 90 days by searching with the version.

Configuring deployments in Swarmia
Deployment frequency
Change lead time
Time to deploy
Deployments API
Deployments
Change failure rate
Mean time to recovery
Deployments
pull requests in each deployment
Metrics / DORA
Settings / Deployments
automatically detect change failures
Deployments API
Deployments
this blog post