Is
your environment ready to open the throttle? A self-check, and what to
fix first
This page closes the second half of Deploying Agents at Scale:
What Enterprise Adoption Actually Looks Like, Agentic Engineering
Days Zurich, 20 November 2026. The talk promised that you would be "able
to tell whether your own environment is ready to open the throttle, and
what to fix first if it isn't". The interactive version asks the same ten
questions and names your first fix as you answer; it loads, sends and
stores nothing. Printed, it is a one-page checklist of the same ten
questions and the fix-first order, also here as a PDF.
Is your environment ready to open the throttle on coding agents?
Answer these ten questions yes or no, for one team or one system, as
things stand today. Each question is about your own environment and
needs no tool to answer. They are in fix-first order: if you answer no
to any, the first no is the first thing to fix. Each comes with the
evidence behind it and the kind of evidence it is, so you can check the
work.
Answer yes only if it is true today, not planned. Every piece of
evidence carries its label in words. Each one that states a figure links
its row in the evidence index, which
gives the source, the date it was read and its status; the rest link
their source. Two things here are argued, not measured:
the order of the checks, and that a yes to all ten means ready.
The ten checks
1. A baseline taken before
scaling
Do you have delivery measures recorded from before agent use grew,
such as lead time, change fail rate, rework rate, recovery time and
total monthly incidents, and do you still collect them the same way?
Evidence:
Experienced developers expected a 24% speedup and believed afterward
they had been 20% faster, while their tasks took 19% longer with AI
allowed. Controlled study (metr-expected-24-believed-20,
metr-19-percent-longer).
DORA's five delivery metrics are change lead time, deployment
frequency, failed deployment recovery time, change fail rate and
deployment rework rate. Self-reported, survey (DORA's metrics
guide).
Each 25% increase in AI adoption went with an estimated 7.2% drop in
delivery stability, a drop only a before-and-after measure shows.
Self-reported, survey (s10-dora-2024-each-25-percent-increase).
Faros's code churn still up 71.6% in Q3 2026 comes from a report
behind a sign-up form. Vendor telemetry, Upstream:
unverified at the primary, and no check rests on it
alone (faros-q3-code-churn-71-6).
2. Limits the agent cannot
lift
Are the things an agent must never do, such as deleting data,
resetting infrastructure or using production credentials, blocked by
permissions it does not hold, and not only by instructions in its prompt
or by an approval a person clicks through?
Evidence:
Adaptive attacks beat 12 recent published defenses that work inside
the model's reasoning, with attack success above 90% for most.
Controlled study, adaptive attack evaluation (s9-adaptive-attacks-bypass-12-recent-defenses).
My own 481-line prompt had eight rules, rule 6 "ASK WHEN STUCK", and
it did not stop a reset script aimed at all nine nodes; the reset log
records the etcd data directory deleted. Self-reported, own incident
record (s7-prompt-of-481-lines-and-8,
s7-reset-script-targets-9-nodes).
Claude Code got past its deny rule by path, then asked for approval
to run outside its sandbox: "Approval fatigue turns a security boundary
into a rubber stamp." Self-reported, a vendor's own
demonstration, one run, no figure (Ona,
3 March 2026).
Sandboxing cut permission prompts by 84% in Anthropic's internal
usage, so fewer approvals reach a person. Vendor telemetry, the
vendor's own internal usage (s9-sandboxing-cut-permission-prompts-by-84).
3. Agent reports
checked against reality
Is every claim an agent makes about what it built, changed or
deployed checked against the real system before anyone acts on it?
Evidence:
"The fleet apply failed for every attendee cluster", in the commit
before the agent's status report, and the second fix was verified on "a
single-cluster plan" only. Self-reported, own incident record
(teardown B's sources).
The agent then said "150 clusters are provisioning concurrently",
counting its own log lines, and there were zero attendee clusters at
workshop time. Self-reported, own incident record, public
record only: unverified at the primary, because the
session transcript is not public (s7-150-clusters-are-provisioning-concurrently-19,
s7-0-attendee-clusters-at-workshop-time).
4. A written review rule by
risk
Does every change that merges get a review under a written rule that
says which changes need a human, which an agent and which both, and do
you know what share of last month's merges had no review?
Evidence:
Faros advises: "define which changes require human review, agentic
review, or both". Recommendation, not a figure (see the review, testing and
measures page).
24,008 unique secrets sat in MCP-related configuration files in
public code, 2,117 of them valid, so the secret check should cover those
files too. Vendor telemetry (s5-gitguardian-24-008-unique-secrets-in).
Each failed CI check cuts an agent PR's odds of a merge by about
15%. Self-reported, observational study (s1-agent-prs-each-failed-ci-check).
6. Tests that check the
behavior
Do agent-written changes come with tests that check the behavior that
was asked for, written or reviewed by someone other than the agent that
wrote the code?
Evidence:
Test coverage was inadequate in 91% of the agent PRs that failed the
maintainers' tests and in 100% of those that passed. Controlled
study, agent evaluation (s11-metr-failure-table-runs-failing-tests).
Giving the model the tests solved an additional 12.0% of MBPP and
8.5% of HumanEval problems. Controlled study, benchmark
experiment (s11-mathews-and-nagappan-gpt-4-solved).
Of generated tests that survived Meta's build, pass and coverage
filters, engineers accepted 73%. Self-reported, company field
report (s8-meta-testgen-llm-75-percent-of).
7. Small changes, practiced
rollback
Do you keep each change small, and can every team undo a bad change
quickly because it has practiced doing so?
Small batches amplify AI's effect on product performance, and
frequent commits and rollbacks strengthen it. Self-reported,
survey (DORA
AI Capabilities Model).
DORA advises: "Teams should make each change as small as possible to
make the delivery process fast and stable", and "Encourage teams to
become highly proficient in using rollback and revert features".
Recommendation, not a figure (see the review, testing and
measures page and the skills page).
8. Review capacity grown
with volume
Has the capacity for review and QA grown in step with the volume of
change agents produce, so that the time changes wait for review and
spend in QA is not rising?
Evidence:
After coding agents arrived, submission to merge took 56.1% longer,
across 655 firms. Controlled study, quasi-experimental (s8-after-ai-agents-submission-to-merge).
Incidents per PR up 14.5% while monthly incidents rose 125.4%: in
Faros's later sample the per-change spike faded and is still above the
baseline, and the monthly total more than doubled. The Q3 figure comes
from a different sample of organizations than April's. Vendor
telemetry (s10-faros-incidents-per-pr-up-242).
Is what done means written down before an agent starts a task, and do
your repositories give agents short, human-written notes on what they
could not work out from the code?
Evidence:
With a human-written AGENTS.md file, median runtime was 28.64% lower
and output tokens 16.58% lower. Controlled study, benchmark
experiment (s11-lulla-et-al-10-repositories-124).
Context files in general did not raise task success and raised
inference cost by over 20%. Controlled study, benchmark
experiment (s11-gloaguen-et-al-context-files-do).
10. Reviewers who
can debug unfamiliar code
Can the engineers who approve agent changes read and debug code they
did not write, and do your junior engineers still get practice debugging
code themselves?
Evidence:
Developers learning with an AI assistant averaged 50% on a quiz,
against 67% for those coding by hand, and "The largest gap in scores
between the two groups was on debugging questions". Controlled
study, randomized, vendor-run (s11-52-mostly-junior-developers-ai-group,
Anthropic,
29 January 2026).
What to fix first
Argued, not measured. No source read ranks these
fixes. The order is built from the measured findings above in five
steps, and each step says which part is measured and which is
argued.
A baseline first (check 1).Measured:
people inside a rollout cannot tell whether it helped; developers
believed they were 20% faster when their tasks took 19% longer (metr-expected-24-believed-20,
metr-19-percent-longer).
Argued: every later fix is judged against the baseline, and a
baseline cannot be taken after the fact.
Cap what cannot be undone (checks 2 and 3).Measured: controls that live in the agent's reasoning fail
under adaptive attack, above 90% for most of 12 published defenses (s9-adaptive-attacks-bypass-12-recent-defenses);
in my own sessions a prompt rule did not stop a nine-node reset, and a
status report went out after the fix was checked on a single-cluster
plan only (s7-prompt-of-481-lines-and-8,
teardown B's sources).
Argued: deleted data and a false report acted on cannot be
caught later by review, so the limits come before any gate on what
merges.
Keep the gate on what merges (checks 4 to 6).Measured: the review gate is what erodes as volume rises, with
PRs merged without review going from a 31.3% rise to 76.3% (s10-faros-q3-prs-merged-without-review),
and code that works is often not secure, with syntax near 100% and a
security pass rate of 56% (s5-veracode-2026-the-average-security-pass).
Argued: the written rule comes first because the automated
gates and the tests enforce it; the gates come before tests because they
are settings in the pipeline the team already has.
Make the volume hold (checks 7 and 8).Measured: changes grew and waited longer, with average PR size
up 71.8% and submission to merge 56.1% longer (s10-faros-q3-average-pr-size-up,
s8-after-ai-agents-submission-to-merge).
Argued: smaller changes and more review capacity make each
check cheaper to pass, which matters only once the gates in step 3
exist.
Improve what goes in (checks 9 and 10).Measured: written task context and clarified requirements
improve what agents produce, and learning with an assistant left the
widest gap on debugging (s11-lulla-et-al-10-repositories-124,
Anthropic,
29 January 2026). Argued: these raise the quality of what
enters the pipeline rather than limit harm, and skills take the longest
to build, so they come last in the list and are started early.
Ready means a yes to all ten. That a yes to every check means ready
is argued, not measured: no study shows that these ten together are
enough.
What your answers mean
Ten yeses. By this check, you are ready to raise
the volume. Keep reading checks 1 and 8 every month, because the
measures and the queue are where a rollout drifts first.
A no in checks 1 to 3. Fix this before anything
else. Without a baseline you cannot tell whether a change helped;
without limits the agent cannot lift, and checks on what it reports, one
mistake can reach something you cannot undo.
A no in checks 4 to 6. Your gate on what merges is
the limit. More volume moves the cost into QA and production, where
Faros saw monthly incidents up 125.4% (vendor telemetry, s10-faros-incidents-per-pr-up-242).
A no in checks 7 and 8. The gate exists but the
volume will swamp it. Shrink the changes and grow the review capacity
before you add agents.
A no in checks 9 and 10. The pipeline is safe; what
goes into it can be better. Start these early, because skills take the
longest to build.
What the check does not tell
you
It is not a ranking from a study. No source read
ranks these fixes. The order is the talk's argument, built from findings
that are each measured on their own.
Ten yeses is not proof of readiness. That a yes to
every check means ready is argued, not measured. No study followed
environments with all ten in place.
It does not measure your results. It asks whether
the practices are in place, not whether they work in your system. Check
1's baseline is how you find that out.
It does not cover cost, licensing or data
residency, or whether the agents are worth what they cost.
Its evidence is uneven. Some checks rest on
randomized trials, some on vendors' telemetry from their own customers,
some on surveys, and two on my own incident records of one session each.
The label on each piece of evidence says which.
It says nothing about any particular company's
systems. Every source is published research, a vendor's public
report, or my own public incident record.