Is your environment ready to open the throttle?
A self-check from Deploying Agents at Scale: What Enterprise Adoption Actually Looks Like, Agentic Engineering Days Zurich, 20 November 2026
How to answer
Is your environment ready to open the throttle on coding agents? Answer these ten questions yes or no, for one team or one system, as things stand today. Each question is about your own environment and needs no tool to answer. They are in fix-first order: if you answer no to any, the first no is the first thing to fix. Each comes with the evidence behind it and the kind of evidence it is, so you can check the work.
Answer yes only if it is true today, not planned. This page sends nothing and stores nothing: your answers stay on your screen and are gone when you close it.
Key to the labels
Every piece of evidence carries one of these labels in words. Where a label goes on after a comma, the rest says which kind it is, such as a benchmark, a randomized trial or a survey.
- Controlled study
- Researchers set the conditions and measured the result: a trial, a benchmark or a comparison before and after with a control.
- Vendor telemetry
- A company measuring its own systems or its customers' systems, usually a company that sells the fix.
- Self-reported
- What people or organizations say about themselves: a survey, a company's own field report, a vendor's own demonstration, or Michael's own record of an incident in one session he ran.
- Observational study
- Counts taken from public activity, such as pull requests on GitHub, with no control group.
- Recommendation, not a figure
- A source's advice, quoted. It carries no number.
- Unverified
- Not checked against the primary source. It is kept and marked, and no check rests on it alone.
- Measured
- A reason for the order that rests on a finding in the evidence, which it links to.
- Argued
- A reason for the order that is the talk's own reasoning, not a finding.
"Verified, read" gives the date the source was read live and found to say what the evidence quotes.
"Evidence-index rows" under each question names the rows of the talk's evidence index that its figures come from, each a link to its row. A row marked unverified was not read in a live primary source. Evidence that states no figure, such as a quote or a recommendation, has no row and links its source instead.
The ten checks, in fix-first order
-
1. A baseline taken before scaling
Do you have delivery measures recorded from before agent use grew, such as lead time, change fail rate, rework rate, recovery time and total monthly incidents, and do you still collect them the same way?
Evidence-index rows: metr-expected-24-believed-20, metr-19-percent-longer, s10-dora-2024-each-25-percent-increase, faros-q3-code-churn-71-6 (unverified)
The evidence for check 1
- Experienced developers expected a 24% speedup and believed afterward they had been 20% faster Controlled study Source: METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", 10 July 2025 Verified, read 2026-10-05
- The same developers' tasks took 19% longer with AI allowed Controlled study Source: METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", 10 July 2025 Verified, read 2026-10-05
- DORA's five delivery metrics: change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate Self-reported, survey Source: DORA, "DORA's software delivery performance metrics", last updated 5 January 2026 Verified, read 2026-11-13
- Each 25% increase in AI adoption went with an estimated 7.2% drop in delivery stability, a drop only a before-and-after measure shows Self-reported, survey Source: DORA, "Accelerate State of DevOps Report 2024", pages 39 and 40 Verified, read 2026-11-13
- Code churn still up 71.6% in Q3 2026, from the form-gated report Vendor telemetry Source: Faros AI, the Speed Trap report, a form-gated PDF that was not read; the link is Faros's public takeaways page Unverified at the primary: the report was not read, and the figure is kept as an earlier pass recorded it.
-
2. Limits the agent cannot lift
Are the things an agent must never do, such as deleting data, resetting infrastructure or using production credentials, blocked by permissions it does not hold, and not only by instructions in its prompt or by an approval a person clicks through?
Evidence-index rows: s9-adaptive-attacks-bypass-12-recent-defenses, s7-prompt-of-481-lines-and-8, s7-reset-script-targets-9-nodes, s9-sandboxing-cut-permission-prompts-by-84
The evidence for check 2
- Adaptive attacks beat 12 recent published defenses that work inside the model's reasoning, with attack success above 90% for most Controlled study, adaptive attack evaluation Source: Nasr, Carlini and others, "The Attacker Moves Second", arXiv 2510.09023, 10 October 2025 Verified, read 2026-11-13
- A 481-line prompt with eight rules, rule 6 "ASK WHEN STUCK", did not stop a reset script aimed at all nine nodes Self-reported, own incident record Source: Michael's own incident record: the rebuild prompt, August 2025, in his public cluster repo Verified, read 2026-11-13
- The reset log records the etcd data directory deleted Self-reported, own incident record Source: Michael's own incident record: the reset log of 23 August 2025, in his public Ignite repo Verified, read 2026-11-13
- Claude Code got past its deny rule by path, then asked for approval to run outside its sandbox: "Approval fatigue turns a security boundary into a rubber stamp." Self-reported, a vendor's own demonstration Source: Leonardo Di Donato, Ona, "How Claude Code escapes its own denylist and sandbox", 3 March 2026 Verified, read 2026-11-13
- Sandboxing cut permission prompts by 84% in Anthropic's internal usage, so fewer approvals reach a person Vendor telemetry, the vendor's own internal usage Source: Anthropic, "Making Claude Code more secure and autonomous with sandboxing", 20 October 2025 Verified, read 2026-11-13
-
3. Agent reports checked against reality
Is every claim an agent makes about what it built, changed or deployed checked against the real system before anyone acts on it?
Evidence-index rows: s7-150-clusters-are-provisioning-concurrently-19 (unverified), s7-0-attendee-clusters-at-workshop-time (unverified)
The evidence for check 3
- "The fleet apply failed for every attendee cluster", in the commit before the agent's status report Self-reported, own incident record Source: Michael's own incident record: workshop repo commit 6f72f26, 29 June 2026 Verified, read 2026-11-13
- The second fix was verified on "a single-cluster plan" only, the last commit before the session Self-reported, own incident record Source: Michael's own incident record: workshop repo commit d15031a, 29 June 2026 Verified, read 2026-11-13
- The agent said "150 clusters are provisioning concurrently" from its own log lines Self-reported, own incident record Source: Michael's public incident record of the June 2026 workshop fleet, quoting a session transcript Unverified at the primary: read 2026-11-13 in Michael's public incident record, which quotes a session transcript that was not checked.
- Zero attendee clusters at workshop time Self-reported, own incident record Source: Michael's public incident record of the June 2026 workshop fleet, quoting a session transcript Unverified at the primary: read 2026-11-13 in Michael's public incident record, which quotes a session transcript that was not checked.
-
4. A written review rule by risk
Does every change that merges get a review under a written rule that says which changes need a human, which an agent and which both, and do you know what share of last month's merges had no review?
Evidence-index rows: s10-faros-q3-prs-merged-without-review, s1-rejected-agent-prs-228-of-600
The evidence for check 4
- "define which changes require human review, agentic review, or both" Recommendation, not a figure Source: Faros AI, Neely Dunlap, "How AI affects developer productivity and engineering workflows", 13 October 2026 Verified, read 2026-11-13
- PRs merged without review: a 31.3% rise in April 2026, 76.3% by Q3 Vendor telemetry Source: Faros AI, "The Speed Trap: 8 takeaways from our latest AI engineering research", 18 September 2026 Verified, read 2026-11-13
- Of 600 rejected agent PRs, 228 (38%) had no meaningful human review Observational study Source: Ehsani and others, "Where Do AI Coding Agents Fail?", arXiv 2601.15195, 21 January 2026 Verified, read 2026-11-13
-
5. Automated gates before human review
Must every agent-written change pass the same build, test, static analysis and secret checks as human code before a person reviews it?
Evidence-index rows: s5-veracode-2026-the-average-security-pass, s5-veracode-2026-syntax-correct-nearly-100, s5-gitguardian-claude-code-assisted-commits-3, s5-gitguardian-24-008-unique-secrets-in, s1-agent-prs-each-failed-ci-check
The evidence for check 5
- Security pass rate across models 56%, "barely changed from 55%" a year earlier Controlled study (benchmark, vendor-run) Source: Veracode blog, Natalie Tischler, 2026 GenAI code security report, 28 July 2026 Verified, read 2026-11-13
- The same models' code was syntactically correct "nearly 100% of the time" Controlled study (benchmark, vendor-run) Source: Veracode blog, Natalie Tischler, 2026 GenAI code security report, 28 July 2026 Verified, read 2026-11-13
- Claude Code-assisted public commits leaked secrets at 3.2%, against a 1.5% baseline Vendor telemetry Source: GitGuardian, "The State of Secrets Sprawl 2026", 17 March 2026 Verified, read 2026-11-13
- 24,008 unique secrets in MCP-related configuration files, 2,117 of them valid Vendor telemetry Source: GitGuardian, "The State of Secrets Sprawl 2026", 17 March 2026 Verified, read 2026-11-13
- Each failed CI check cuts an agent PR's odds of a merge by about 15% Observational study Source: Ehsani and others, "Where Do AI Coding Agents Fail?", arXiv 2601.15195, 21 January 2026 Verified, read 2026-11-13
-
6. Tests that check the behavior
Do agent-written changes come with tests that check the behavior that was asked for, written or reviewed by someone other than the agent that wrote the code?
Evidence-index rows: s11-metr-failure-table-runs-failing-tests, s11-mathews-and-nagappan-gpt-4-solved, s8-meta-testgen-llm-75-percent-of
The evidence for check 6
- Test coverage was inadequate in 91% of the agent PRs that failed the maintainers' tests and in 100% of those that passed Controlled study, agent evaluation Source: METR, David Rein, "Research Update: Algorithmic vs. Holistic Evaluation", 13 August 2025 Verified, read 2026-11-13
- Giving the model the tests solved an additional 12.0% of MBPP and 8.5% of HumanEval problems Controlled study, benchmark experiment Source: Mathews and Nagappan, "Test-Driven Development for Code Generation", arXiv 2402.13521 Verified, read 2026-11-13
- Of generated tests that survived Meta's build, pass and coverage filters, engineers accepted 73% Self-reported, company field report Source: Alshahwan and others, "Automated Unit Test Improvement using Large Language Models at Meta", arXiv 2402.09171, 14 February 2024 Verified, read 2026-11-13
-
7. Small changes, practiced rollback
Do you keep each change small, and can every team undo a bad change quickly because it has practiced doing so?
Evidence-index rows: s10-faros-q3-average-pr-size-up
The evidence for check 7
- Average PR size up 71.8% Vendor telemetry Source: Faros AI, "The Speed Trap: 8 takeaways from our latest AI engineering research", 18 September 2026 Verified, read 2026-11-13
- Small batches amplify AI's effect on product performance; frequent commits and rollbacks strengthen it Self-reported, survey Source: Storer and DeBellis, "Introducing the DORA AI Capabilities Model", Google Cloud blog, 23 September 2025 Verified, read 2026-11-13
- "Teams should make each change as small as possible to make the delivery process fast and stable" Recommendation, not a figure Source: DORA, "DORA's software delivery performance metrics", last updated 5 January 2026 Verified, read 2026-11-13
- "Encourage teams to become highly proficient in using rollback and revert features" Recommendation, not a figure Source: Storer and DeBellis, "Introducing the DORA AI Capabilities Model", Google Cloud blog, 23 September 2025 Verified, read 2026-11-13
-
8. Review capacity grown with volume
Has the capacity for review and QA grown in step with the volume of change agents produce, so that the time changes wait for review and spend in QA is not rising?
Evidence-index rows: s8-after-ai-agents-submission-to-merge, s10-faros-incidents-per-pr-up-242, s11-faros-q3-2026-22-000-developers
The evidence for check 8
- After coding agents arrived, submission to merge took 56.1% longer, across 655 firms Controlled study, quasi-experimental Source: Chen and Stratton, "Artificial Intelligence in the Firm: Bottlenecks in Software Production", page 37 Verified, read 2026-11-13
- Incidents per PR up 14.5% while monthly incidents rose 125.4%: in Faros's later sample the per-change spike faded and is still above the baseline, and the monthly total more than doubled. The Q3 figure comes from a different sample of organizations than April's. Vendor telemetry Source: Faros AI, "The Speed Trap: 8 takeaways from our latest AI engineering research", 18 September 2026 Verified, read 2026-11-13
- Time in QA up 300.6%, the largest deterioration among Faros's flow metrics Vendor telemetry Source: Faros AI, "The Speed Trap: 8 takeaways from our latest AI engineering research", 18 September 2026 Verified, read 2026-11-13
-
9. Done written down first
Is what done means written down before an agent starts a task, and do your repositories give agents short, human-written notes on what they could not work out from the code?
Evidence-index rows: s11-lulla-et-al-10-repositories-124, s11-ambig-swe-models-struggle-to-distinguish, s11-gloaguen-et-al-context-files-do
The evidence for check 9
- With a human-written AGENTS.md file, median runtime 28.64% lower and output tokens 16.58% lower Controlled study, benchmark experiment Source: Lulla and others, "On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents", arXiv 2601.20404 Verified, read 2026-11-13
- When agents asked about an underspecified task, performance rose by up to 74% Controlled study, benchmark experiment Source: Vijayvargiya and others, "Ambig-SWE", arXiv 2502.13069 Verified, read 2026-11-13
- Context files in general did not raise task success and raised inference cost by over 20% Controlled study, benchmark experiment Source: Gloaguen and others, "Evaluating AGENTS.md", arXiv 2602.11988 Verified, read 2026-11-13
-
10. Reviewers who can debug unfamiliar code
Can the engineers who approve agent changes read and debug code they did not write, and do your junior engineers still get practice debugging code themselves?
Evidence-index rows: s11-52-mostly-junior-developers-ai-group
The evidence for check 10
- Developers learning with an AI assistant averaged 50% on a quiz, against 67% for those coding by hand Controlled study, randomized, vendor-run Source: Anthropic, "How AI assistance impacts the formation of coding skills", 29 January 2026 Verified, read 2026-11-13
- "The largest gap in scores between the two groups was on debugging questions" Controlled study, randomized, vendor-run Source: Anthropic, "How AI assistance impacts the formation of coding skills", 29 January 2026 Verified, read 2026-11-13
Your result
What your first no means
Read down your answers. The first check you answer no to is the first thing to fix, and the other no answers follow in the order of the list.
- A no in checks 1 to 3. Fix this before anything else. Without a baseline you cannot tell whether a change helped; without limits the agent cannot lift, and checks on what it reports, one mistake can reach something you cannot undo.
- A no in checks 4 to 6. Your gate on what merges is the limit. More volume moves the cost into QA and production, where Faros saw monthly incidents up 125.4% (vendor telemetry).
- A no in checks 7 and 8. The gate exists but the volume will swamp it. Shrink the changes and grow the review capacity before you add agents.
- A no in checks 9 and 10. The pipeline is safe; what goes into it can be better. Start these early, because skills take the longest to build.
- Ten yeses. By this check, you are ready to raise the volume. Keep reading checks 1 and 8 every month, because the measures and the queue are where a rollout drifts first. The evidence does not show that a yes to every check means ready. That is argued, not measured: no study followed environments with all ten in place.
Why this order
Argued, not measured. No source read ranks these fixes. No study compares the order in which teams put these practices in place, or measures which one matters most. The order is built from measured findings in five steps, and each reason below is labeled measured or argued.
-
Step 1: Check 1
1. A baseline taken before scaling
- Measured People inside a rollout cannot tell whether it helped: developers believed they were 20% faster when their tasks took 19% longer Evidence: METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", 10 July 2025
- Argued Every later fix is judged against the baseline, and a baseline cannot be taken after the fact, so it is the first thing to put right
-
Step 2: Checks 2, 3
2. Limits the agent cannot lift; 3. Agent reports checked against reality
- Measured Controls that live in the agent's reasoning fail under adaptive attack, above 90% for most of 12 published defenses Evidence: Nasr, Carlini and others, "The Attacker Moves Second", arXiv 2510.09023, 10 October 2025
- Measured In Michael's own sessions a prompt rule did not stop a nine-node reset, and a status report went out after the fix was checked on a single-cluster plan only Evidence: Michael's own incident record: the rebuild prompt, August 2025, in his public cluster repo; Michael's own incident record: workshop repo commit d15031a, 29 June 2026
- Argued Deleted data and a false report acted on cannot be caught later by review, so the limits come before any gate on what merges
-
Step 3: Checks 4, 5, 6
4. A written review rule by risk; 5. Automated gates before human review; 6. Tests that check the behavior
- Measured The review gate is what erodes as volume rises: PRs merged without review went from a 31.3% rise to 76.3% Evidence: Faros AI, "The Speed Trap: 8 takeaways from our latest AI engineering research", 18 September 2026
- Measured Code that works is often not secure: syntax near 100%, security pass rate 56% Evidence: Veracode blog, Natalie Tischler, 2026 GenAI code security report, 28 July 2026
- Argued The written rule comes first because the automated gates and the tests enforce it; the gates come before tests because they are settings in the pipeline the team already has
-
Step 4: Checks 7, 8
7. Small changes, practiced rollback; 8. Review capacity grown with volume
- Measured Changes grew and waited longer: average PR size up 71.8%, submission to merge 56.1% longer Evidence: Faros AI, "The Speed Trap: 8 takeaways from our latest AI engineering research", 18 September 2026; Chen and Stratton, "Artificial Intelligence in the Firm: Bottlenecks in Software Production", page 37
- Argued Smaller changes and more review capacity make each check cheaper to pass, which matters only once the gates in step 3 exist
-
Step 5: Checks 9, 10
9. Done written down first; 10. Reviewers who can debug unfamiliar code
- Measured Written task context and clarified requirements improve what agents produce, and learning with an assistant left the widest gap on debugging Evidence: Lulla and others, "On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents", arXiv 2601.20404; Anthropic, "How AI assistance impacts the formation of coding skills", 29 January 2026
- Argued These raise the quality of what enters the pipeline rather than limit harm, and skills take the longest to build, so they come last in the list and are started early
Ready means a yes to all ten. That a yes to every check means ready is argued, not measured: no study shows that these ten together are enough.
What the check does not tell you
- It is not a ranking from a study. No source read ranks these fixes. The order is the talk's argument, built from findings that are each measured on their own.
- Ten yeses is not proof of readiness. That a yes to every check means ready is argued, not measured. No study followed environments with all ten in place.
- It does not measure your results. It asks whether the practices are in place, not whether they work in your system. Check 1's baseline is how you find that out.
- It does not cover cost, licensing or data residency, or whether the agents are worth what they cost.
- Its evidence is uneven. Some checks rest on randomized trials, some on vendors' telemetry from their own customers, some on surveys, and two on Michael's own incident records of one session each. The label on each piece of evidence says which.
- Secrets in agent configuration files. GitGuardian counted 24,008 unique secrets in MCP-related configuration files in public code, 2,117 of them valid (vendor telemetry). Check 5's secret check should cover those files too.
What to fix first
Start at your first no and work down this order. The order is argued, not measured: no source read ranks these fixes.
- A baseline first. 1. A baseline taken before scaling
- Cap what cannot be undone. 2. Limits the agent cannot lift; 3. Agent reports checked against reality
- Keep the gate on what merges. 4. A written review rule by risk; 5. Automated gates before human review; 6. Tests that check the behavior
- Make the volume hold. 7. Small changes, practiced rollback; 8. Review capacity grown with volume
- Improve what goes in. 9. Done written down first; 10. Reviewers who can debug unfamiliar code
Ten yeses means ready by this check. That a yes to every check means ready is argued, not measured.