What AI-generated code ships: the security data

This page is part of the second half of Deploying Agents at Scale: What Enterprise Adoption Actually Looks Like, Agentic Engineering Days Zurich, 20 November 2026. The talk promised "the security data on what AI-generated code actually ships". Here is what ships, at what rate, and against what baseline.

Every figure carries a label, so you can judge it yourself. A controlled study fixes the tasks and measures the output: a benchmark prompts models with set tasks and scans what comes back, and a user trial gives one group of people an assistant and another group none. A large-sample study measures code that was already published, not code written for the study; it counts as self-reported, because which code is AI-written rests on the code's own marking. Vendor telemetry is a company reporting on data it holds, usually about a problem its product addresses. No forecast is used here. Every source was read live in its primary on 2026-11-13; the table at the end gives each one with its date and link. Anything that could not be read in a primary is listed at the end as unverified, and nothing on the page rests on it.

The short answer

What ships. The same weakness classes recur across independent studies: cross-site scripting and log injection, SQL injection, insufficiently random values, code injection, hardcoded secrets, and, in filed vulnerabilities, command injection, authentication bypass and server-side request forgery.

At what rate. In benchmarks, roughly two in five generated programs are vulnerable, and the rate has barely moved while correctness rose.

Against what baseline. This is the thin part. Benchmarks have no human baseline, the two user trials disagree on the size of the gap, and the comparisons of shipped code depend on knowing which commits had AI help.

The rate has barely moved

Controlled study (benchmark). In 2021, Pearce and colleagues had GitHub Copilot complete 89 security-relevant scenarios and found "approximately 40%" of the 1,689 resulting programs vulnerable (Pearce et al., IEEE S&P 2022).

Controlled study (benchmark). A 2023 replication with a newer Copilot found the vulnerable share of Python suggestions "reduced from 36.54% to 27.25%" (Majdinasab et al.).

Controlled study (benchmark, run by a vendor that sells the scanner). Veracode's benchmark of more than 100 models found vulnerabilities "in 45 percent of cases" in 2025. In 2026: "The average security pass rate across models is 56%", "barely changed from 55% in the first report", while syntax was correct "nearly 100% of the time". The best model reached 68%.

Some weakness classes fare far worse than others

Controlled study (benchmark, run by a vendor). In Veracode's 2026 data, models passed SQL injection tasks 83% of the time, cryptographic algorithms 87%, cross-site scripting 15% and log injection 12%. In 2025, cross-site scripting and log injection failed in 86% and 88% of cases, and Java was the weakest language, with a failure rate over 70% against 38% to 45% for Python, C# and JavaScript.

Whole applications and agent tasks do worse

Controlled study (benchmark). BaxBench asked models to build 392 backends and then attacked them. The best model got 62% of them working, and the researchers could run exploits on "around half of the correct programs" (Vero et al., 2025).

Controlled study (benchmark). SusVibes gave 12 agent setups 186 real feature requests from open-source projects where a human had once shipped a vulnerable version. For one agent, 57% of the solutions were functionally correct and 11.8% were secure (Zhao et al., ICML 2026). Because the tasks were chosen where humans got it wrong, this is a hard test, not a comparison with humans.

Against people without AI, the trials disagree

Controlled study (user trial). Stanford's trial ended with 47 participants, 33 with an assistant and 14 without. With it, 36% wrote code open to SQL injection against 7% without; on a file-path task, 12% wrote secure code against 29%. Participants with the assistant "were more likely to believe they wrote secure code" (Perry et al., ACM CCS 2023).

Controlled study (user trial). Sandoval and colleagues' trial of 58 student programmers writing C found critical bugs "at a rate no greater than 10% more than the control" (Sandoval et al., USENIX Security 2023).

Both trials tested early OpenAI Codex models with mostly student participants.

Against humans' own mistakes, Copilot was not worse

Controlled study (benchmark against human commits). Asare and colleagues prompted Copilot at the exact points where human developers had introduced known C and C++ vulnerabilities. Copilot reproduced the vulnerable code "about 33% of the time" and the fixed code "at a 25% rate" (Asare et al., Empirical Software Engineering).

In code that shipped

Self-reported, large-sample study. Of 733 AI-generated snippets found in published GitHub projects, 29.5% of the Python and 24.2% of the JavaScript had weaknesses, across 43 CWE categories. Given the scanner's warnings, Copilot Chat fixed up to 55.5% of them (Fu et al., ACM TOSEM 2025).

Vendor telemetry. GitGuardian: "Claude Code-assisted commits showed a 3.2% secret-leak rate, versus a 1.5% baseline across all public GitHub commits." The same report found 24,008 unique secrets in MCP-related configuration files on public GitHub, 2,117 of them valid (State of Secrets Sprawl 2026). GitGuardian's blog does not say how it identified Claude Code-assisted commits.

Vendor telemetry. CodeRabbit compared 470 open-source pull requests, 320 AI-co-authored and 150 human-only, and found 10.83 issues per pull request against 6.45, with security issues "up to 2.74× higher". CodeRabbit says it "cannot guarantee" the human-only set had no AI in it, and the blog does not say which category the 2.74× is.

In filed vulnerabilities

Self-reported, large-sample study. Georgia Tech's Vibe Security Radar scanned over 43,000 security advisories for vulnerabilities whose introducing commit carries an AI tool's signature. As of 13 April 2026 it had 74 confirmed cases, 14 critical and 25 high: about 18 in the second half of 2025, and 56 in the first three months of 2026, 35 of them in March. The count is a floor, because code with its AI signature removed is not counted.

What this evidence does not show

Checked and not used

Each of these was looked for and could not be read in a primary on 2026-11-13. They are unverified and nothing on this page rests on them.

Claim Why it is not used
Apiiro, "4x velocity, 10x vulnerabilities" (2025) Unverified: the blog returned HTTP 403 and a script-only page
A Veracode October 2026 edition with a 62% pass rate Unverified: on a reseller page only; Veracode's own pages give the July edition
Veracode: "AI now authors roughly half of all committed code" Unverified: stated on Veracode's blog with no source or method
CodeRabbit: 1.57 times as many security findings, 2.74 times for cross-site scripting Unverified, secondary: in press coverage only, not in CodeRabbit's blog
Georgia Tech radar: the true count five to ten times higher Unverified, secondary: in write-ups only, not in Georgia Tech's article

Sources

Every source was read live on 2026-11-13 for the research behind this page.

Source Published Label Read URL
Pearce, Ahmad, Tan, Dolan-Gavitt, Karri, "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions", IEEE S&P 2022 arXiv v3 Controlled study, benchmark 2026-11-13 https://arxiv.org/abs/2108.09293
Majdinasab, Bishop, Rasheed, Moradidakhel, Tahir, Khomh, "Assessing the Security of GitHub Copilot Generated Code: A Targeted Replication Study" 18 November 2023 Controlled study, benchmark 2026-11-13 https://arxiv.org/abs/2311.11177
Veracode, 2025 GenAI Code Security Report, press release 30 July 2025 Controlled study, benchmark, vendor-run 2026-11-13 https://veracode.com/press-release/ai-generated-code-poses-major-security-risks-in-nearly-half-of-all-development-tasks-veracode-research-reveals/
Natalie Tischler, Veracode, "2026 GenAI Code Security Report: AI Is Writing More of Your Code but Security Hasn't Caught Up" 28 July 2026 Controlled study, benchmark, vendor-run 2026-11-13 https://veracode.com/blog/2026-genai-code-security-report-ai-risk
Vero, Mündler, Chibotaru, Raychev, Baader, Jovanović, He, Vechev, "BaxBench: Can LLMs Generate Correct and Secure Backends?" 2025, arXiv v3 Controlled study, benchmark 2026-11-13 https://arxiv.org/abs/2502.11844
Zhao, Wang, Zhang, Luo, Li, Li, "Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks" (SusVibes), ICML 2026 arXiv v4, 21 September 2026 Controlled study, benchmark 2026-11-13 https://arxiv.org/abs/2512.03262
Perry, Srivastava, Kumar, Boneh, "Do Users Write More Insecure Code with AI Assistants?", ACM CCS 2023 arXiv v3 Controlled study, user trial 2026-11-13 https://arxiv.org/abs/2211.03622
Sandoval, Pearce, Nys, Karri, Garg, Dolan-Gavitt, "Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants", USENIX Security 2023 arXiv v4 Controlled study, user trial 2026-11-13 https://arxiv.org/abs/2208.09727
Asare, Nagappan, Asokan, "Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?", Empirical Software Engineering arXiv v5 Controlled study, benchmark against human commits 2026-11-13 https://arxiv.org/abs/2204.04741
Fu, Liang, Tahir, Li, Shahin, Yu, Chen, "Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study", ACM TOSEM 2025 arXiv v4 Self-reported, large-sample study 2026-11-13 https://arxiv.org/abs/2310.02059
Georgia Tech School of Cybersecurity and Privacy, "Bad Vibes: AI-Generated Code is Vulnerable, Researchers Warn" 13 April 2026 Self-reported, large-sample study 2026-11-13 https://hg.gatech.edu/node/689636
GitGuardian, "The State of Secrets Sprawl 2026" 17 March 2026 Vendor telemetry 2026-11-13 https://blog.gitguardian.com/the-state-of-secrets-sprawl-2026/
CodeRabbit, "Our new report: AI code creates 1.7x more problems" 17 December 2025 Vendor telemetry 2026-11-13 https://coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report

Every figure above also has its own row in the evidence index, under "The security of AI-generated code".