Deploying Agents at Scale: What Enterprise Adoption Actually Looks Like

Michael Forrester

AI Startup Advisor, Neo4j Startup Program

Agentic Engineering Days Zurich · November 2026

The most careful measurement we have: experienced developers got slower

+24%
speedup they expected before
+20%
speedup they believed after
19%
longer: what the clock measured with AI allowed

Confidence interval +2% to +39%. 16 experienced open-source developers, 246 tasks, in repositories they knew well, on early-2025 tools.

Controlled study METR (Model Evaluation and Threat Research) is a nonprofit that studies advanced AI agents. Its randomized controlled trial, published 10 July 2025. Both speedups are the developers' own estimates, recorded inside the trial.

Every number today carries a label, so you can check my work

Controlled study
Researchers set the conditions and measured the result.
Vendor telemetry
A vendor measuring its own customers' systems. It usually sells the fix.
Self-reported
What people or organizations say about themselves: surveys, interviews, company statements.
Forecast
A prediction of what will happen. Never evidence of what did.

Each label sits next to its number. Every source and its label is on the sources slide and at agenticday.michaelrishiforrester.com.

Two studies, two answers you cannot both keep

DORA 2025

DORA is Google Cloud's long-running research program on software delivery. This is its annual report.
Self-reported Nearly 5,000 technology professionals surveyed. AI is an amplifier: "It magnifies the strengths of high-performing organizations and the dysfunctions of struggling ones." Strong foundations protect you: platform quality, clear workflows, aligned teams.

Faros AI, April 2026

Faros AI sells software that measures engineering teams. Its reports use data from the teams on its own platform.
Vendor telemetry Telemetry from 22,000 developers. Mature organizations deteriorated the same as everyone else.

Which one holds up? Three layers: the individual, the team, the enterprise.

The 2026 follow-up could not run cleanly, and that is the finding

"30% to 50% of developers told us that they were choosing not to submit some tasks because they did not want to do them without AI."

METR says this likely biases its new estimate of AI-assisted speedup downward. It cannot yet say whether the 2025 slowdown holds for 2026 tools.

Self-reported METR, 2026 update to the developer productivity experiment. The 30% to 50% is what developers told METR when surveyed.

Faros's telemetry: mature organizations broke too

Measure, highest-adoption quarters against lowestChange
Epics completed per developerup 66%
Incidents-to-PR ratio (incidents per merged PR)up 243%
Code churn (lines deleted per line added)up 861%
Median time in reviewup 442%
PRs merged with no reviewup 31%

Vendor telemetry Faros AI, "Acceleration Whiplash", April 2026: two years of telemetry, 22,000 developers, 4,000+ teams. Faros is a vendor reporting its own customers' data. Changes are rounded to whole percentages.

Six months later a different sample looks better, and nobody can say why

MeasureApril 2026Q3 2026
Incidents per PRup 243%up 15%
Code churnup 861%still up 72%
Deployment frequencyfallingup 14%
"We also cannot fully isolate two concurrent trends: organizational maturation in AI use, and improvements in the underlying models."

Vendor telemetry Faros AI, "The Speed Trap", Q3 2026 (12 months ending August 2026). The two reports sample different organizations. Deployment frequency covers the roughly 10% of Faros's teams that have deployment data. Changes are rounded to whole percentages.

How teams got through the dip: they removed the review gate

Measure, against the low-adoption baselineApril 2026Q3 2026
PRs merged with no human or agentic reviewup 31%up 76%
Time in QAnot reported as headlineup 301%
Monthly incidentsup 58%up 125%

Vendor telemetry Faros AI, April 2026 and Q3 2026 reports. Changes are rounded to whole percentages.

The verdict: the per-change spike faded, and total incidents more than doubled

up 15%
incidents per PR, Q3 2026, against the low-adoption baseline (April: up 243%, a different sample)
up 125%
monthly incidents, Q3 2026, against the same baseline

Both are true. Quote one alone and you mislead. The verification tax went unpaid, and the bill moved to QA and production.

Faros compares each team with its own earlier quarters, so its data cannot test whether strong foundations protect a team. The evidence does not pick a side on that claim.

Vendor telemetry Faros AI, Q3 and April 2026, rounded. "Verification tax" is DORA's term for time spent reviewing AI output.

The enterprise layer: JPMorgan scaled access fast and reports gains, not measurements

200,000
onboarded LLM Suite users within eight months
over 90%
of engineers using AI code assistants

Self-reported JPMorgan Chase, LLM Suite award news, 3 June 2025; JPMorgan Chase 2025 annual report.

Three methods found the same gap between capability and capture

MethodSourceFinding
Vendor telemetry 10,000+ developersFaros AI, 2025Team output up; organization-wide delivery metrics flat
Self-reported SurveyMcKinsey, State of AI 202680% report better individual productivity; 37% report EBIT impact
Self-reported PractitionerDerek Waldron, JPMorgan"A gap between what the technology was capable of doing and what the bank was actually capturing"

Self-reported On whether any of this returns money, MIT and Wharton published opposite headlines in 2025 because they counted different things (side by side in the appendix).

Waldron quoted by CIO, September 2026.

My own workshop: the agent said the fleet was provisioning, and I believed it

250
attendee clusters planned for my workshop
150*
what the agent told me: "150 clusters are provisioning concurrently"
0*
attendee clusters at workshop time

It was counting lines in its own local Terraform logs*. The relaunch had failed for every attendee cluster, and the fix was verified on a single-cluster plan.

Missing: a check against what the cloud lists, before any status report reaches a person.

Self-reported My own incident record, one session, not a study. *Public record only: my incident record quotes the session transcript, which is not public, so these are unverified at the primary. Full teardown: github.com/peopleforrester/the-day-claude-code-deleted-my-cluster

The levers: redesign the work, add guardrails, measure against a baseline

Raises successEvidence
Redesign the work around the toolSelf-reported Measured among the leaders: nearly three-quarters of McKinsey's high performers redesigned workflows, against about a quarter of everyone else
Add guardrailsArgued, not measured among the leaders. Self-reported In Deloitte's 2026 survey, only 21% report a mature governance model for agents
Measure against a real baselineArgued, not measured among the leaders: perception is not a baseline. It rests on Controlled study METR's trial, Vendor telemetry Faros's telemetry and Self-reported my own workshop

Causes failure: pulling funding during the dip, and scaling change volume past verification capacity.

The teams that kept the gate rebuilt it for machine speed

After coding agents arrived, a pull request took 56.1% longer from submission to merge, across 655 firms. Controlled study quasi-experimental: no random assignment

What they changedEvidenceLabel
An agentic first passHeavy agentic-review adoption: 10.7% less time to first review, 24.4% lower change failure rate (correlational, per Faros)Vendor telemetry correlational
A gate written down by risk"define which changes require human review, agentic review, or both"Recommendation (Faros), not a figure
Generated tests, filtered before a human sees themEngineers accepted 73% of the tests that survived Meta's filtersSelf-reported company field report

Randomized trials at Google and Meta found AI in review did not make review faster. One design made it over 5% slower. Controlled study randomized

Chen and Stratton, Harvard, "Artificial Intelligence in the Firm", 9 November 2026 · Faros AI, The Speed Trap takeaways, 18 September 2026, and blog, 13 October 2026 · Alshahwan et al., Meta, 2024 · Vijayvergiya et al., Google, 2024 · Maddila et al., Meta, 2025

Ready to open the throttle? Answer ten questions, and fix the first no

Cap the downside

  1. A baseline taken before scaling Controlled study
  2. Limits the agent cannot lift Controlled study
  3. Agent reports checked against reality Self-reported
  4. A written review rule by risk Vendor telemetry
  5. Automated gates before human review Controlled study

Then make it hold

  1. Tests that check the behavior Controlled study
  2. Small changes, practiced rollback Vendor telemetry
  3. Review capacity grown with volume Controlled study
  4. Done written down first Controlled study
  5. Reviewers who can debug unfamiliar code Controlled study

The order is argued, not measured: no study ranks these fixes, and ten yeses meaning ready is my argument.

The interactive self-check, with each check's full question, evidence and label: agenticday.michaelrishiforrester.com

Verdict: on quality, DORA 2025 and Faros agree that more AI adoption goes with less stability, nothing here tests whether strong foundations protect a team, and verification has to grow with volume.

On Monday: run the ten-question check on one team, and fix the first no.

AI amplifies whatever system it lands in. Point it at a mess and it produces the mess faster.

Michael Forrester

AI Startup Advisor, Neo4j Startup Program

The second half and every source:

agenticday.michaelrishiforrester.com

QR code for https://agenticday.michaelrishiforrester.com

Appendix: what the 25 minutes left out

The second half is at

agenticday.michaelrishiforrester.com

There: the second half, the evidence index with every figure's source and label, and the readiness self-check.

The six slides after this one are the rows the talk did not teach: guardrails, measures, skills, security, two teardowns, and what the teams that kept the gains changed. Then MIT against Wharton, side by side.

Guardrails come in two layers, and only one is outside the agent's reach

Reasoning layer

Enforced in the logic the agent reasons in: instructions, deny rules on command text, prompt injection filters.
Controlled study Adaptive attacks beat 12 published defenses of this kind, above 90% success for most.

Infrastructure layer

Enforced outside the agent: an OS sandbox with an egress proxy, a production database it cannot write, a policy layer around the model.
Controlled study CaMeL's policy layer solved 77% of AgentDojo tasks with provable security, against 84% undefended.

The catch: a control the agent can ask to switch off is back in the reasoning layer. In Ona's test, Claude Code got past its deny rule, then asked to run outside its sandbox.

Nasr, Carlini et al., arXiv 2510.09023, October 2025. Debenedetti et al., CaMeL, arXiv 2503.18813, June 2025. Ona, 3 March 2026, a vendor's own demonstration. Examples, sources and limits: agenticday.michaelrishiforrester.com

What to count in place of PRs merged

PRs per author rose 20% in a year. Change failure rate rose 30%. Vendor telemetry Cortex

MeasureWhat it showedLabel
Change fail rate and rework rateEach 25% rise in AI adoption: an estimated 7.2% drop in delivery stability (DORA 2024)Self-reported survey
Incidents per PR, beside monthly incidentsPer PR up 14.5%; monthly incidents up 125.4% (Faros Q3 2026)Vendor telemetry
Time to recoverMean time to remediate up 25.8% (Faros Q3 2026)Vendor telemetry
Merged within 30 days of openingManual PRs 84.4%, AI PRs 32.7% (LinearB 2026)Vendor telemetry

Heavy agentic-review teams: change failure rate 24.4% lower. Faros reads it as correlational. Vendor telemetry correlational

Cortex, Engineering in the Age of AI: 2026 Benchmark Report · DORA, Accelerate State of DevOps Report 2024 · Faros AI, The Speed Trap takeaways, 18 September 2026, and blog, 13 October 2026 · LinearB, 2026 Software Engineering Benchmarks Report · Every source and the full measurement set: agenticday.michaelrishiforrester.com

The skills the studies point to: read, test, specify, see the system

Learning a new library with AI: quiz score 50%, by hand 67%. Widest gap: debugging. Controlled study randomized, vendor-run (Anthropic)

SkillWhat breaks without itLabel
ReviewAgent PRs passed the tests on 38% of issues; none of 15 read by hand was mergeable as-is (METR)Controlled study agent evaluation
TestingTest coverage inadequate in 91% of failing and 100% of passing agent PRs (METR)Controlled study agent evaluation
SpecificationAsking about underspecified tasks: performance up to 74% higher (Ambig-SWE)Controlled study benchmark experiment
Systems thinkingNo user-centric focus: AI adoption can hurt team performance (DORA, almost 5,000 respondents)Self-reported survey

No study yet ranks which skill separated the teams that succeeded. Shen and Tamkin, Anthropic, "How AI Impacts Skill Formation", 2026 · METR, Algorithmic vs. Holistic Evaluation, 13 August 2025 · Vijayvargiya et al., Ambig-SWE, 2026 · DORA AI Capabilities Model, 2025

What AI-generated code ships: the security data

Benchmark

Controlled study run by a vendor. 56% average security pass rate across 80 tasks, from 55% a year earlier. Syntax is near 100%.

User trial

Controlled study 47 people. With an assistant, 36% wrote code open to SQL injection; without, 7%. The assistant group was more likely to believe its code was secure.

Shipped commits

Vendor telemetry Claude Code-assisted commits leaked secrets at 3.2%, against 1.5% for all public GitHub commits.

None of the three gives a clean number for how much worse AI code is than human code in production. Faros's figures earlier in the talk are reliability, not security.

Veracode 2026 GenAI Code Security Report · Perry et al., ACM CCS 2023 · GitGuardian, State of Secrets Sprawl 2026 · The rest of the security data: agenticday.michaelrishiforrester.com

Two teardowns from my own platforms

2025 · My home lab · one session

The prompt's rule 6: "ask for guidance immediately and do not proceed". Discovery found 4 running containers and called it a "damaged cluster". About six minutes later its reset script ran kubeadm reset -f: "Deleted contents of the etcd data directory: /var/lib/etcd". July: 187 pods. September: none of July's workloads back.
Missing: a gate the agent cannot argue past, between deciding to reset and the reset.

2026 · My workshop fleet · 250 attendee clusters planned

The relaunch failed for every attendee cluster, and the fix was verified on a single-cluster plan. The agent: "150 clusters are provisioning concurrently"*, counted from its own local Terraform logs*. At workshop time: 0 attendee clusters*.
Missing: a check against what the cloud lists, before any status report reaches a person.

Self-reported My own incident records, one session each, not studies. *Public record only: my incident record quotes the session transcript, which is not public, so these are unverified at the primary. Full teardowns: github.com/peopleforrester/the-day-claude-code-deleted-my-cluster

What the teams that kept the gains changed

What they changedEvidenceLabel
Small changes that pass CIEach failed CI check cuts an agent PR's odds of merging by about 15%, across 33k agent PRsSelf-reported observational study
Verification that scales with authoringHeavy agentic review: time to first review down 10.7%Vendor telemetry correlational
Short context written by peopleAn AGENTS.md cut median agent runtime by 28.64%; generated context files did not raise success and cost over 20% moreControlled study benchmark experiments

Ehsani et al., "Where Do AI Coding Agents Fail?", MSR 2026 · Faros AI, blog, 13 October 2026 · Lulla et al., "On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents", JAWs 2026 · Gloaguen et al., "Evaluating AGENTS.md", 2026

MIT and Wharton disagree on returns because they counted different things

MIT NANDA, 2025

Self-reported Preliminary findings, not peer reviewed. 52 organizations interviewed, 153 leaders surveyed. Success meant tools that users or executives said caused "a marked and sustained productivity and/or P&L impact". Headline: 95% saw no return.

Wharton, October 2025

Self-reported About 800 senior decision-makers. A broader definition of return. Positive ROI, self-reported, by seniority: VPs and above 81%, managers 69%.

Sources

  • Controlled study METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", 10 July 2025
  • Controlled studySelf-reported METR, 2026 update to the developer productivity experiment design
  • Self-reported DORA, 2025 State of AI-assisted Software Development
  • Vendor telemetry Faros AI, "Acceleration Whiplash", April 2026
  • Vendor telemetry Faros AI, "The Speed Trap", Q3 2026
  • Vendor telemetry Faros AI, "AI Productivity Paradox", 2025
  • Vendor telemetry Faros AI, "How AI affects developer productivity and engineering workflows", blog, 13 October 2026
  • Self-reported JPMorgan Chase, LLM Suite award news, 3 June 2025; 2025 annual report
  • Self-reported CIO, Derek Waldron interview, September 2026
  • Self-reported McKinsey, State of AI 2026
  • Self-reported Wharton and GBK, "Accountable Acceleration", October 2025
  • Self-reported MIT NANDA, "The GenAI Divide: State of AI in Business 2025"
  • Self-reported Deloitte, State of AI in the Enterprise 2026
  • Controlled study Chen and Stratton, Harvard, "Artificial Intelligence in the Firm", 9 November 2026
  • Controlled study Vijayvergiya et al., Google, AIware 2024; Maddila et al., Meta, 2025
  • Self-reported Alshahwan et al., Meta, TestGen-LLM, 2024

Appendix slides 18 to 23 carry their own sources, each figure with its label. Every source: agenticday.michaelrishiforrester.com