Measuring the impact of AI coding tools

The most reliable signal is commit metadata. Many AI tools sign their work — a “Generated with” trailer in the commit message, or a recognizable git author identity for autonomous agents like Devin, Cursor’s background agent, or Copilot’s coding agent. DevPerform reads these signatures from your git history and classifies pull requests as human, AI-assisted, or agent-authored.

This is a lower bound, and saying so is part of the metric. Inline autocomplete, chat copy-paste, and any tool with attribution turned off leave no trace, so real AI usage is always at least as high as the measured share — never lower. Signature-based measurement tells you what you can prove, not everything that happened.

The adoption context is stark. Stack Overflow’s 2025 Developer Survey found 84% of developers using or planning to use AI tools, and Google’s 2025 DORA report found 90% of software professionals using AI, with a median of 2 hours a day. The question is no longer whether your team uses AI — it’s whether it’s helping.

Be skeptical of any single productivity multiplier. The published studies span an enormous range depending on who was studied and on what work:

• −19% — tasks took 19% longer with AI. METR RCT (2025): 16 experienced open-source maintainers on mature repos they knew deeply.
• +26% — completed tasks. Field experiments, Cui et al. (2024–26): 4,867 developers at Microsoft, Accenture, and a Fortune 100 firm, Copilot-style completion.
• +55.8% — faster. Copilot RCT, Peng et al. (2023): a single scoped greenfield task (JavaScript HTTP server).

The same technology measured −19% for experienced maintainers on mature codebases they knew deeply and +55.8% on a single scoped greenfield task. That spread is the whole point: there is no universal number, which is exactly why you should measure your own team rather than import someone else’s headline.

• Untagged assistance. Autocomplete and chat copy-paste don’t show up in commit signatures, so a big share of real AI help is invisible to any telemetry.
• Causation. High-adoption teams may differ in a dozen other ways; correlating adoption with delivery is suggestive, not proof.
• Individual productivity. AI impact is a team-level question. DevPerform deliberately keeps the smallest unit of any AI-impact view at the team, the same anti-surveillance stance as its no-individual-rankings rule — which is also what keeps the data candid.

Because the perception and the telemetry each have blind spots, the trustworthy approach is to show them side by side rather than multiply them into a single number. DevPerform’s AI ROI report fuses self-reported time savings from developer surveys (labeled as perception) with AI-authorship telemetry from your git history (labeled as a lower bound), with the methodology written on the page, not hidden in a tooltip. Explore the demo to see how adoption connects to delivery outcomes at the team level.