AI coding agent metrics

Gain visibility into the pull requests assisted by different AI coding agents and how they perform across metrics like throughput, merge percentage, and batch size.

What is an AI coding agent?

AI coding agents are AI tools responsible for creating pull requests end-to-end — planning work, editing code, running tests, and responding to reviews — not just suggesting code in your editor.

Most often, AI coding agents are individually assigned tasks from issue trackers or prompted by users to create a single pull request. AI coding agents can also be autonomous, in which case they will have a backlog or objective and continuously pick work items on their own.

Examples of coding agents

• GitHub Copilot coding agent can create pull requests from various sources, including IDEs, GitHub.com, GitHub issues, GitHub Mobile, CLI, and MCP.

• Cursor Cloud Agents can create pull requests from cursor.com/agents, or you can trigger them via integrations, such as Slack and Linear.

• Claude Code GitHub Actions can create pull requests when you mention @claude in PRs or issues.

• Claude Code on the web is available in research preview to selected pricing plans and can create pull requests from claude.ai/code.

Understanding your coding agent performance

To help you understand how your AI coding agents are performing, Swarmia automatically detects pull requests created by GitHub Copilot, Cursor, or Claude Code AI coding agents. Swarmia provides a dedicated Coding agents view for analyzing them.

Agent PR throughput

The Agent PR throughput chart shows how much work AI coding agents help you ship.

Agent PRs, Merged, and Closed show the raw number of AI coding agent pull requests.

Merge percentage tells you about your conding agent performance and how effectively people are able to use agents. Ideally, the number should grow close to 100%, but never quite reach it. Even though every closed pull request is technically waste (i.e., work done that was never shipped), a small percentage of closed pull requests indicates that people are experimenting with AI coding agents and pushing their limits.

Agent commit percentage

The Agent commit percentage chart breaks down your merged pull requests based on how much human intervention was needed for the work to be shipped.

Ideally, all your agent pull requests would have 100% agent commits, which means that the person was able to complete the whole task through the agent interface (e.g., GitHub Copilot's pull request integration). If a pull request ends up in the other buckets, it means that someone needed to pull up their coding environment and commit code to finish the work.

Batch size

The Batch size chart helps you compare the distribution of AI coding agent pull requests with your overall Batch size metrics. It also gives you an indication of the complexity of work AI coding agents can tackle.

Depending on how you start using AI coding agents, you might see that most agent pull requests initially fall in the smallest 1–5 or largest 500+ buckets. As your AI coding agent usage develops, you should see the distribution move toward the buckets in the middle. This means your agents can tackle tasks of all sizes and ship code in manageable increments.

Teams breakdown

The "Teams" tab provides a breakdown of the core metrics shown in the top charts, grouped by team.

Pull requests tab

You can use the "Pull requests" tab in combination with the pull request filters at the top to inspect the individual agent pull requests.

Identifying PRs created with an AI coding agent

The AI coding agent used to create a pull request is shown on top of the pull request author’s avatar. You can hover over the avatar to see more details.

In Swarmia, pull requests authored by AI tools are attributed to the people who initiated them. For example, if GitHub Copilot creates a PR on your behalf, Swarmia shows you as its author.

AI agent filter

You can filter pull requests with the AI agent filter in views like Code metrics > Cycle time and AI tools > AI impact.

Frequently asked questions

Which AI coding agents are supported?

We currently support GitHub Copilot, Cursor, and Claude Code. (See examples above.) If you're using a different coding agent or have built your own custom AI coding agents, reach out to us at [email protected] with the agent details (the agent's GitHub account or Git commit email, plus an example pull request) you'd like to track.

Why is my PR not attributed to an AI coding agent?

Swarmia attributes a PR to a coding agent if the PR or its first commit has been authored by an AI agent.

Example 1: With GitHub Copilot coding agent, GitHub shows Copilot as the PR author:

Example 2: When using Cursor Cloud Agents, GitHub shows cursoragent as the author of the first commit, and the user who requested the agent as the PR author:

Both are considered coding agent PRs in Swarmia.

Note: Regardless of what GitHub displays, Swarmia always shows the author as the person who initiated the AI coding agent PR. Identifying PRs created with an AI coding agent above for more details

If I ask an AI tool to create a PR with my local changes, is it considered an AI coding agent PR?

No, we don't regard that as an AI coding agent PR since you review and accept the suggestions (and perhaps manually write some of the code) before creating the PR.

The AI uses your local Git identity, so you are marked as the PR author and the commit author. (The tool might mark itself as the co-author, however.)

Swarmia shows these PRs as AI-assisted.