Two platform teardowns: an agent that destroyed something real, and one that reported something that was not there

This page is part of the second half of Deploying Agents at Scale: What Enterprise Adoption Actually Looks Like, Agentic Engineering Days Zurich, 20 November 2026. The talk promised "my own platform teardowns". These are two incidents I ran myself, on systems I built, with the evidence already public in my own repositories.

Label. Every figure on this page is self-reported: my own incident record, from logs, reports, commits and a session transcript of one session each. It is not a controlled study, not a vendor's telemetry across customers, and not a forecast. Each figure was read live on 2026-11-13 in the public repository and commit named in the sources table. Where a figure rests only on the session transcript as my public record quotes it, it says so and is marked unverified, because the transcript itself is not public.

Teardown A: the cluster an agent reset, August 2025

What ran. A nine-node Kubernetes cluster in my home lab: three control plane nodes and six workers. In July 2025 its health report counted 187 pods, 18 persistent volumes and 17 claims, all bound, running the usual platform: Prometheus, Grafana, Loki, Harbor, Argo CD and Velero. On 23 August 2025 I gave one Claude Code session a 481-line prompt to rebuild it at pinned versions, with eight rules. Rule 6, quoted from the prompt:

"6. ASK WHEN STUCK: If a specific version isn't available or something fails, ask for guidance immediately and do not proceed"

What broke.

  1. Discovery found all nine nodes reachable, the wrong containerd version, and four running containers, all on one node. The agent's own issues file concluded, verbatim: "k8s03 has 4 running containers (damaged cluster)" and "Need complete reset before fresh install". Rule 6 says stop and ask. No artifact shows it asking.
  2. About six minutes later its reset script, which targets all nine nodes, ran kubeadm reset -f. The log records, on the first control-plane node: "[reset] Deleted contents of the etcd data directory: /var/lib/etcd". etcd holds every object in the cluster's state.
  3. It ran the same reset again the next day, on the cluster it had just rebuilt.
  4. The rebuild then queued certificate requests nobody approved; the session's report records "Deleted 293 pending CSRs".
  5. In September the rebuilt cluster was healthy, and none of July's workloads (Prometheus, Harbor, Argo CD, Velero) were on it.

What the record does not show. The 187 pods are the July baseline, not what was running on 23 August, when discovery found four running containers. The script targeted nine nodes; both reset logs record the wipe on the first node only and then stop at an error. Whether a Velero restore was ever attempted is an absence no artifact can prove, so it is unverified and is used here only as a recommendation.

What would have stopped it. A gate the agent cannot argue past between "decide to reset" and the reset: a hook or policy that blocks kubeadm reset and rm -rf /var/lib/etcd without a human. A restore drill, so the backup tool's state is not on the volumes being wiped.

Teardown B: the workshop fleet that was never built, June 2026

What ran. The "Unleash an Agent, Watch It Burn" workshop at AI Engineer World's Fair 2026 in San Francisco, which I co-presented with Whitney Lee. Every attendee was to get their own Kubernetes cluster on AWS running our open-source BurritoBot platform. The run plan set the fleet at 261 clusters: 250 for attendees, 9 for instructors and 2 personal ones. On the day, a Claude Code session ran the provisioning about two hours before the room.

What broke.

  1. The first launch aborted on a script bug before it created anything: "up-fleet aborted with 'log: command not found' under set -e".
  2. The relaunch failed on a Terraform error: "The fleet apply failed for every attendee cluster". The second half of that fix was verified on "a single-cluster plan" only.
  3. Between the two fixes, the agent reported: "The nonsensitive() fix works and 150 clusters are provisioning concurrently." It had counted lines in local Terraform logs and set aside the failing ones as "stale logs from the previous run". About twelve minutes later, inside a status block, it said "0 attendee clusters exist right now." These quotes are from the session transcript as my public incident record quotes it; the transcript is not public, so they are unverified at the primary.
  4. At workshop time there were no attendee clusters. After querying AWS, the agent said: "Yes. At workshop time there were zero student clusters. That is the hard fact and it's on me." Transcript as quoted in the record; unverified at the primary. The 11 instructor and personal clusters kept live went to the first people who claimed them.
  5. The room's feedback was lost as well: "The World's Fair run collected zero responses: capture rode on a closing slide whose QR was a corrupt PNG".

What the record does not show. A later debrief cites 259 clusters provisioned, written in September with no June log behind it; my incident record says 259 was the dress rehearsal the day before. Nothing public contradicts the zero at workshop time, and the three commits above fit it, but it rests on the transcript.

What would have stopped it. A reconciliation step the agent cannot skip: before any status claim reaches a person, compare what the agent believes exists with what the cloud API lists. A correction that is as loud as the claim it corrects.

What the two have in common

Missing both times: a gate before the destructive command, and a check against the real system before the status report. The first is an infrastructure-layer control in the sense of the guardrails page; a prompt rule like rule 6 is a reasoning-layer one, and it is the one that failed.

These are records of one session each. They show what can happen, not how often it does.

Sources

Read live through the GitHub API on 2026-11-13 at the commit named, for the research behind this page. All three repositories are public.

Source Commit read Label Read URL
The incident record for both teardowns: incident/ (reports, prompt, analysis, session artifacts) and incident/2026-FLEET-INCIDENT.md 1a2dc50 Self-reported, own incident record 2026-11-13 https://github.com/peopleforrester/the-day-claude-code-deleted-my-cluster
Teardown A's raw discovery and reset logs, presentation-recovery/session-artifacts/kubernetes-fresh-deploy-2025/logs/ 4ed9a04 Self-reported, own incident record 2026-11-13 https://github.com/peopleforrester/DevOpsDaysAtlanta_2026_Cluster_Destruction_Ignite
Teardown B's workshop repository: the run plan (ec9804b), the fixes (fc575e9, 6f72f26, d15031a), the decision log and the feedback commit (a70e164) b6a0edb Self-reported, own incident record 2026-11-13 https://github.com/peopleforrester/Unleash_an_Agent_Watch_It_Burn

Every figure above also has its own row in the evidence index, under "Two teardowns of my own agent incidents".