Deploying Agents at Scale: What Enterprise Adoption Actually Looks Like
Michael Forrester
AI Startup Advisor, Neo4j Startup Program
Agentic Engineering Days Zurich · November 2026
The most careful measurement we have: experienced developers got slower
+24%
speedup they expected before
+20%
speedup they believed after
19%
longer: what the clock measured with AI allowed
Confidence interval +2% to +39%. 16 experienced open-source developers, 246 tasks, in repositories they knew well, on early-2025 tools.
Controlled study METR (Model Evaluation and Threat Research) is a nonprofit that studies advanced AI agents. Its randomized controlled trial, published 10 July 2025. Both speedups are the developers' own estimates, recorded inside the trial.
Every number today carries a label, so you can check my work
Controlled study Researchers set the conditions and measured the result.
Vendor telemetry A vendor measuring its own customers' systems. It usually sells the fix.
Self-reported What people or organizations say about themselves: surveys, interviews, company statements.
Forecast A prediction of what will happen. Never evidence of what did.
Each label sits next to its number. Every source and its label is on the sources slide and at agenticday.michaelrishiforrester.com.
Two studies, two answers you cannot both keep
DORA 2025
DORA is Google Cloud's long-running research program on software delivery. This is its annual report. Self-reported Nearly 5,000 technology professionals surveyed. AI is an amplifier: "It magnifies the strengths of high-performing organizations and the dysfunctions of struggling ones." Strong foundations protect you: platform quality, clear workflows, aligned teams.
Faros AI, April 2026
Faros AI sells software that measures engineering teams. Its reports use data from the teams on its own platform. Vendor telemetry Telemetry from 22,000 developers. Mature organizations deteriorated the same as everyone else.
Which one holds up? Three layers: the individual, the team, the enterprise.
The 2026 follow-up could not run cleanly, and that is the finding
"30% to 50% of developers told us that they were choosing not to submit some tasks because they did not want to do them without AI."
METR says this likely biases its new estimate of AI-assisted speedup downward. It cannot yet say whether the 2025 slowdown holds for 2026 tools.
Self-reported METR, 2026 update to the developer productivity experiment. The 30% to 50% is what developers told METR when surveyed.
Faros's telemetry: mature organizations broke too
Measure, highest-adoption quarters against lowest
Change
Epics completed per developer
up 66%
Incidents-to-PR ratio (incidents per merged PR)
up 243%
Code churn (lines deleted per line added)
up 861%
Median time in review
up 442%
PRs merged with no review
up 31%
Vendor telemetry Faros AI, "Acceleration Whiplash", April 2026: two years of telemetry, 22,000 developers, 4,000+ teams. Faros is a vendor reporting its own customers' data. Changes are rounded to whole percentages.
Six months later a different sample looks better, and nobody can say why
Measure
April 2026
Q3 2026
Incidents per PR
up 243%
up 15%
Code churn
up 861%
still up 72%
Deployment frequency
falling
up 14%
"We also cannot fully isolate two concurrent trends: organizational maturation in AI use, and improvements in the underlying models."
Vendor telemetry Faros AI, "The Speed Trap", Q3 2026 (12 months ending August 2026). The two reports sample different organizations. Deployment frequency covers the roughly 10% of Faros's teams that have deployment data. Changes are rounded to whole percentages.
How teams got through the dip: they removed the review gate
Measure, against the low-adoption baseline
April 2026
Q3 2026
PRs merged with no human or agentic review
up 31%
up 76%
Time in QA
not reported as headline
up 301%
Monthly incidents
up 58%
up 125%
Vendor telemetry Faros AI, April 2026 and Q3 2026 reports. Changes are rounded to whole percentages.
The verdict: the per-change spike faded, and total incidents more than doubled
up 15%
incidents per PR, Q3 2026, against the low-adoption baseline (April: up 243%, a different sample)
up 125%
monthly incidents, Q3 2026, against the same baseline
Both are true. Quote one alone and you mislead. The verification tax went unpaid, and the bill moved to QA and production.
Faros compares each team with its own earlier quarters, so its data cannot test whether strong foundations protect a team. The evidence does not pick a side on that claim.
Vendor telemetry Faros AI, Q3 and April 2026, rounded. "Verification tax" is DORA's term for time spent reviewing AI output.
The enterprise layer: JPMorgan scaled access fast and reports gains, not measurements
200,000
onboarded LLM Suite users within eight months
over 90%
of engineers using AI code assistants
Self-reported JPMorgan Chase, LLM Suite award news, 3 June 2025; JPMorgan Chase 2025 annual report.
Three methods found the same gap between capability and capture
Method
Source
Finding
Vendor telemetry 10,000+ developers
Faros AI, 2025
Team output up; organization-wide delivery metrics flat
"A gap between what the technology was capable of doing and what the bank was actually capturing"
Self-reported On whether any of this returns money, MIT and Wharton published opposite headlines in 2025 because they counted different things (side by side in the appendix).
Waldron quoted by CIO, September 2026.
My own workshop: the agent said the fleet was provisioning, and I believed it
250
attendee clusters planned for my workshop
150*
what the agent told me: "150 clusters are provisioning concurrently"
0*
attendee clusters at workshop time
It was counting lines in its own local Terraform logs*. The relaunch had failed for every attendee cluster, and the fix was verified on a single-cluster plan.
Missing: a check against what the cloud lists, before any status report reaches a person.
Self-reported My own incident record, one session, not a study. *Public record only: my incident record quotes the session transcript, which is not public, so these are unverified at the primary. Full teardown: github.com/peopleforrester/the-day-claude-code-deleted-my-cluster
The levers: redesign the work, add guardrails, measure against a baseline
Raises success
Evidence
Redesign the work around the tool
Self-reported Measured among the leaders: nearly three-quarters of McKinsey's high performers redesigned workflows, against about a quarter of everyone else
Add guardrails
Argued, not measured among the leaders. Self-reported In Deloitte's 2026 survey, only 21% report a mature governance model for agents
Measure against a real baseline
Argued, not measured among the leaders: perception is not a baseline. It rests on Controlled study METR's trial,Vendor telemetry Faros's telemetry and Self-reported my own workshop
Causes failure: pulling funding during the dip, and scaling change volume past verification capacity.
The teams that kept the gate rebuilt it for machine speed
After coding agents arrived, a pull request took 56.1% longer from submission to merge, across 655 firms. Controlled study quasi-experimental: no random assignment
What they changed
Evidence
Label
An agentic first pass
Heavy agentic-review adoption: 10.7% less time to first review, 24.4% lower change failure rate (correlational, per Faros)
Vendor telemetry correlational
A gate written down by risk
"define which changes require human review, agentic review, or both"
Recommendation (Faros), not a figure
Generated tests, filtered before a human sees them
Engineers accepted 73% of the tests that survived Meta's filters
Self-reported company field report
Randomized trials at Google and Meta found AI in review did not make review faster. One design made it over 5% slower. Controlled study randomized
Chen and Stratton, Harvard, "Artificial Intelligence in the Firm", 9 November 2026 · Faros AI, The Speed Trap takeaways, 18 September 2026, and blog, 13 October 2026 · Alshahwan et al., Meta, 2024 · Vijayvergiya et al., Google, 2024 · Maddila et al., Meta, 2025
Ready to open the throttle? Answer ten questions, and fix the first no
Cap the downside
A baseline taken before scaling Controlled study
Limits the agent cannot lift Controlled study
Agent reports checked against reality Self-reported
A written review rule by risk Vendor telemetry
Automated gates before human review Controlled study
Then make it hold
Tests that check the behavior Controlled study
Small changes, practiced rollback Vendor telemetry
Review capacity grown with volume Controlled study
Done written down first Controlled study
Reviewers who can debug unfamiliar code Controlled study
The order is argued, not measured: no study ranks these fixes, and ten yeses meaning ready is my argument.
The interactive self-check, with each check's full question, evidence and label: agenticday.michaelrishiforrester.com
Verdict: on quality, DORA 2025 and Faros agree that more AI adoption goes with less stability, nothing here tests whether strong foundations protect a team, and verification has to grow with volume.
On Monday: run the ten-question check on one team, and fix the first no.
AI amplifies whatever system it lands in. Point it at a mess and it produces the mess faster.
Michael Forrester
AI Startup Advisor, Neo4j Startup Program
The second half and every source:
agenticday.michaelrishiforrester.com
Appendix: what the 25 minutes left out
The second half is at
agenticday.michaelrishiforrester.com
There: the second half, the evidence index with every figure's source and label, and the readiness self-check.
The six slides after this one are the rows the talk did not teach: guardrails, measures, skills, security, two teardowns, and what the teams that kept the gains changed. Then MIT against Wharton, side by side.
Guardrails come in two layers, and only one is outside the agent's reach
Reasoning layer
Enforced in the logic the agent reasons in: instructions, deny rules on command text, prompt injection filters. Controlled study Adaptive attacks beat 12 published defenses of this kind, above 90% success for most.
Infrastructure layer
Enforced outside the agent: an OS sandbox with an egress proxy, a production database it cannot write, a policy layer around the model. Controlled study CaMeL's policy layer solved 77% of AgentDojo tasks with provable security, against 84% undefended.
The catch: a control the agent can ask to switch off is back in the reasoning layer. In Ona's test, Claude Code got past its deny rule, then asked to run outside its sandbox.
Nasr, Carlini et al., arXiv 2510.09023, October 2025. Debenedetti et al., CaMeL, arXiv 2503.18813, June 2025. Ona, 3 March 2026, a vendor's own demonstration. Examples, sources and limits: agenticday.michaelrishiforrester.com
What to count in place of PRs merged
PRs per author rose 20% in a year. Change failure rate rose 30%. Vendor telemetry Cortex
Measure
What it showed
Label
Change fail rate and rework rate
Each 25% rise in AI adoption: an estimated 7.2% drop in delivery stability (DORA 2024)
Self-reported survey
Incidents per PR, beside monthly incidents
Per PR up 14.5%; monthly incidents up 125.4% (Faros Q3 2026)
Vendor telemetry
Time to recover
Mean time to remediate up 25.8% (Faros Q3 2026)
Vendor telemetry
Merged within 30 days of opening
Manual PRs 84.4%, AI PRs 32.7% (LinearB 2026)
Vendor telemetry
Heavy agentic-review teams: change failure rate 24.4% lower. Faros reads it as correlational. Vendor telemetry correlational
Cortex, Engineering in the Age of AI: 2026 Benchmark Report · DORA, Accelerate State of DevOps Report 2024 · Faros AI, The Speed Trap takeaways, 18 September 2026, and blog, 13 October 2026 · LinearB, 2026 Software Engineering Benchmarks Report · Every source and the full measurement set: agenticday.michaelrishiforrester.com
The skills the studies point to: read, test, specify, see the system
Learning a new library with AI: quiz score 50%, by hand 67%. Widest gap: debugging. Controlled study randomized, vendor-run (Anthropic)
Skill
What breaks without it
Label
Review
Agent PRs passed the tests on 38% of issues; none of 15 read by hand was mergeable as-is (METR)
Controlled study agent evaluation
Testing
Test coverage inadequate in 91% of failing and 100% of passing agent PRs (METR)
Controlled study agent evaluation
Specification
Asking about underspecified tasks: performance up to 74% higher (Ambig-SWE)
Controlled study benchmark experiment
Systems thinking
No user-centric focus: AI adoption can hurt team performance (DORA, almost 5,000 respondents)
Self-reported survey
No study yet ranks which skill separated the teams that succeeded. Shen and Tamkin, Anthropic, "How AI Impacts Skill Formation", 2026 · METR, Algorithmic vs. Holistic Evaluation, 13 August 2025 · Vijayvargiya et al., Ambig-SWE, 2026 · DORA AI Capabilities Model, 2025
What AI-generated code ships: the security data
Benchmark
Controlled study run by a vendor. 56% average security pass rate across 80 tasks, from 55% a year earlier. Syntax is near 100%.
User trial
Controlled study 47 people. With an assistant, 36% wrote code open to SQL injection; without, 7%. The assistant group was more likely to believe its code was secure.
Shipped commits
Vendor telemetry Claude Code-assisted commits leaked secrets at 3.2%, against 1.5% for all public GitHub commits.
None of the three gives a clean number for how much worse AI code is than human code in production. Faros's figures earlier in the talk are reliability, not security.
Veracode 2026 GenAI Code Security Report · Perry et al., ACM CCS 2023 · GitGuardian, State of Secrets Sprawl 2026 · The rest of the security data: agenticday.michaelrishiforrester.com
Two teardowns from my own platforms
2025 · My home lab · one session
The prompt's rule 6: "ask for guidance immediately and do not proceed". Discovery found 4 running containers and called it a "damaged cluster". About six minutes later its reset script ran kubeadm reset -f: "Deleted contents of the etcd data directory: /var/lib/etcd". July: 187 pods. September: none of July's workloads back. Missing: a gate the agent cannot argue past, between deciding to reset and the reset.
2026 · My workshop fleet · 250 attendee clusters planned
The relaunch failed for every attendee cluster, and the fix was verified on a single-cluster plan. The agent: "150 clusters are provisioning concurrently"*, counted from its own local Terraform logs*. At workshop time: 0 attendee clusters*. Missing: a check against what the cloud lists, before any status report reaches a person.
Self-reported My own incident records, one session each, not studies. *Public record only: my incident record quotes the session transcript, which is not public, so these are unverified at the primary. Full teardowns: github.com/peopleforrester/the-day-claude-code-deleted-my-cluster
What the teams that kept the gains changed
What they changed
Evidence
Label
Small changes that pass CI
Each failed CI check cuts an agent PR's odds of merging by about 15%, across 33k agent PRs
Self-reported observational study
Verification that scales with authoring
Heavy agentic review: time to first review down 10.7%
Vendor telemetry correlational
Short context written by people
An AGENTS.md cut median agent runtime by 28.64%; generated context files did not raise success and cost over 20% more
Controlled study benchmark experiments
Ehsani et al., "Where Do AI Coding Agents Fail?", MSR 2026 · Faros AI, blog, 13 October 2026 · Lulla et al., "On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents", JAWs 2026 · Gloaguen et al., "Evaluating AGENTS.md", 2026
MIT and Wharton disagree on returns because they counted different things
MIT NANDA, 2025
Self-reported Preliminary findings, not peer reviewed. 52 organizations interviewed, 153 leaders surveyed. Success meant tools that users or executives said caused "a marked and sustained productivity and/or P&L impact". Headline: 95% saw no return.
Wharton, October 2025
Self-reported About 800 senior decision-makers. A broader definition of return. Positive ROI, self-reported, by seniority: VPs and above 81%, managers 69%.
Sources
Controlled study METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", 10 July 2025
Controlled studySelf-reported METR, 2026 update to the developer productivity experiment design
Self-reported DORA, 2025 State of AI-assisted Software Development
Vendor telemetry Faros AI, "Acceleration Whiplash", April 2026
Vendor telemetry Faros AI, "The Speed Trap", Q3 2026