AI Engineer · Published 2026-09-27

AI-Generated Code Is Already Competing With Human Code — Daksh Gupta, Greptile

privateReview: unreviewedConfidence: high
ai codepull requestsCode reviewgreptileCoding agentscoding-agentsai-code-reviewpr-telemetryfailure-modessql-injectionagent-evaluationci-cdstatic-analysis
Open on YouTube ↗

Summary

Overview

  • Speaker: Daksh Gupta
  • Channel: AI Engineer
  • Main topic: Analysis of 1 million AI-generated pull requests and their quality compared to human pull requests
  • Purpose: To share data-driven insights on the quality, failure modes, and industry adoption of end-to-end AI coding agents. Daksh Gupta, co-founder of Greptile, analyzes over one million AI-generated pull requests from companies like Nvidia, Coinbase, and Datadog. By examining Github author fields, PR description footers, and branch name prefixes, the talk investigates whether autonomous coding agents produce code that is competitive with human developers. Using methods such as revert rates, automated code review comments, and review iteration rounds, the findings reveal that AI-generated PRs are growing rapidly—reaching 27.6% by April—and perform comparably to or even slightly better than human-written PRs in various metrics.

Topic Map

Introduction to Greptile and Agentic Coding Evolution

  • Explanation: Greptile builds AI agents that review and test pull requests with full codebase context. The talk outlines the evolution of AI coding from autocomplete (2021-2023) to multi-file editing (2024) and fully autonomous end-to-end coding agents (2025-today).
  • Key claims:
    • December marked a watershed moment when coding agents became fully autonomous.
    • A quarter of all pull requests reviewed by Greptile in April showed strong evidence of full AI authorship.
  • Examples:
    • Autocomplete with GPT-3.5 in 2022
    • Multi-file editing via Cursor in 2024
  • Terminology:
    • Greptile
    • pull requests
    • autonomous coding agents
  • Why it matters: Shows the rapid acceleration and adoption of AI-generated code in real commercial codebases.

Identifying Fully Vibe PRs via Signals

  • Explanation: Identifying AI-generated pull requests is difficult. Three signals were used: GitHub author fields (less than 1%), PR description footers (e.g., Co-authored by Claude or Cursor), and branch name prefixes (e.g., codestral or cursor).
  • Key claims:
    • GitHub author fields only capture less than 1% of AI-authored PRs.
    • PR footers and branch name prefixes provide much clearer signals of AI authorship.
  • Examples:
    • Codex and Claude PR footers stating 'Co-authored by Claude'
    • Codex branch name conventions
  • Terminology:
    • GitHub author field
    • PR description footer
    • branch name prefix
  • Why it matters: Demonstrates a robust methodology for measuring the prevalence of AI code in enterprise repositories.

Evaluating PR Quality: Revert Rates, Comments, and Review Rounds

  • Explanation: To answer whether AI-generated PRs are any good, three methods were analyzed: revert rates, Greptile review comments across severity levels (P0, P1, P2), and the number of review rounds before merging.
  • Key claims:
    • There is very little correlation between AI authorship and higher revert rates compared to humans.
    • Three out of four AI coding agents tested produced fewer P0 review comments than humans.
    • There is negligible statistical difference in the number of review rounds required for AI vs human PRs to merge.
  • Examples:
    • Codex revert rate of 1.2 per thousand vs human revert rate of 2.7
    • P0, P1, and P2 error frequency tracking in Greptile code reviews
  • Terminology:
    • revert rates
    • P0 comments
    • review rounds
  • Why it matters: Challenges the skepticism around AI-generated code quality in production enterprise environments.

Patterns of Failure

  • Explanation: Analyzing qualitative differences in failure modes between AI agents and humans, such as SQL injection, auth bypass, IDOR, N+1 queries, regressions, and missing tests.
  • Key claims:
    • Claude is 1.5x more likely to produce SQL injection errors than humans.
    • Devin is about half as likely as humans to produce auth bypass issues.
    • There is significant variation in failure modes across different AI agents.
  • Examples:
    • SQL injection error rates
    • Auth bypass and N+1 query patterns
  • Terminology:
    • SQL injection
    • auth bypass
    • N+1 query
    • regression
  • Why it matters: Provides granular insights into specific security and reliability risks associated with different coding agents.

Key Points

Rapid Growth of AI-Authored PRs

  • Explanation: The percentage of AI-generated pull requests is growing exponentially month over month, reaching 27.6% by April.
  • Evidence: Historical chart showing exponential monthly growth starting from under 1% in early last year.
  • Practical implication: Engineering teams must adapt code review workflows to handle a high volume of AI-generated code.

Comparable Quality to Human Code

  • Explanation: Quantitative metrics like revert rates and review rounds show that AI-generated PRs are roughly equal in quality to human-written PRs.
  • Evidence: Revert rate comparisons showing Codex at 1.2 and humans at 2.7 per thousand PRs.
  • Practical implication: Autonomous coding agents are viable and reliable for production use in large enterprises.

Agent-Specific Failure Modes

  • Explanation: Different AI agents exhibit distinct patterns of failure, such as higher SQL injection risks in some models compared to others.
  • Evidence: Patterns of failure chart tracking SQL injection, auth bypass, IDOR, and N+1 queries across Claude, Codex, Devin, and Cursor.
  • Practical implication: Teams need targeted guardrails and linters tailored to the specific failure tendencies of the AI agent they use.

Frameworks, Models & Processes

Greptile PR Evaluation Framework

  • How it works: Measures pull request quality and agent performance using quantitative signals from GitHub metadata, review comments, and merge cycles.
  • Components:
    • GitHub author field analysis
    • PR description footer parsing
    • Branch name prefix tracking
    • Revert rate tracking
    • Review comment severity classification (P0, P1, P2)
    • Review iteration round counting
  • When to use: When auditing enterprise codebase health and evaluating the reliability of autonomous coding agents.

Examples & Case Studies

Analyzing 1 million pull requests across large companies like Nvidia, Coinbase, and Datadog.

  • Illustrates: Real-world adoption and performance of AI-generated code in production systems.
  • Lesson: AI code is not just a hobbyist trend; it is actively used in mission-critical commercial codebases.

Actionable Takeaways

  • Immediate:
    • Track PR description footers and branch prefixes to monitor AI code adoption in your organization.
    • Understand specific failure patterns (like SQL injection or auth bypass) for the AI agent you use.
  • Strategic:
    • Autonomous coding agents are ready for enterprise production use with quality comparable to human developers.
    • Traditional code review and validation pipelines need to adapt to handle high volumes of agent-generated code.
  • Questions to investigate:
    • How can we scale code validation and sandboxing to match agent output velocity?
    • What are the best guardrails to prevent agent-specific failure modes like SQL injection?

Claims Worth Verifying

  • 27.6% of pull requests reviewed by Greptile in April had strong evidence of full AI authorship. (statistical)
  • Codex PRs had a revert rate of 1.2 per thousand compared to 2.7 for humans. (statistical)

Notable Quotes

"What we learned by analyzing 1M AI-generated PRs"

Compressed Summary

  • Analyzed over 1 million pull requests from enterprise codebases.
  • AI-generated PR authorship grew to 27.6% by April.
  • Revert rates and review rounds show AI code quality is comparable to human code.
  • Different coding agents exhibit unique failure patterns like SQL injection and auth bypass.
  • Keywords: ai code, pull requests, code review, greptile, coding agents
  • Core insight: Analysis of 1 million pull requests reveals that autonomous AI coding agents produce code with quality and revert rates comparable to human developers, despite having distinct failure patterns.

Core insights

5
Empirical Resulthigh noveltystrong evidence

Aggregate quality metrics (revert rate, review rounds) do not discriminate AI-authored from human-authored PRs, but the *distribution of failure modes* does — and it varies per agent. Codex-posted PRs showed a lower revert rate than human PRs, and three of four tested agents produced fewer P0 review comments than humans, yet Claude was ~1.5x more likely to introduce SQL injection while Devin was ~half as likely to introduce auth bypass.

Why it matters

If aggregate defect or revert metrics converge between agents and humans, a harness that evaluates an agent on 'does it break less often' will report false parity. Evaluation must stratify by vulnerability class (injection, authz, data-access, regression, test coverage) rather than reporting a single quality score.

Generalization

For any code-producing or action-producing agent, per-category error taxonomies are more discriminative than aggregate success/failure rates; aggregate metrics saturate once the agent is 'competent enough' while the residual risk concentrates in specific classes.

There is very little correlation between AI authorship and higher revert rates compared to humans.
Open source video
Three out of four AI coding agents tested produced fewer P0 review comments than humans.
Open source video
Claude is 1.5x more likely to produce SQL injection errors than humans.
Open source video
Mechanismmedium noveltystrong evidence

AI authorship cannot be detected from the GitHub author field (captures <1% of AI PRs); the reliable signals are incidental side channels in the artifact metadata — PR description footers ('Co-authored by Claude') and branch name prefixes (e.g. 'codestral', 'cursor').

Why it matters

Any organization trying to measure agent adoption or attribute quality regressions must instrument commit/PR metadata at the platform layer, not rely on identity fields. It also implies current adoption statistics are a lower bound, produced by scraping conventions that agents happen to emit.

Generalization

Agent provenance in shared systems will first appear as unstructured artifacts of the tool's scaffolding (footers, branch naming, commit trailers) rather than as first-class identity fields; measurement infrastructure should be built on pluggable, heuristic provenance classifiers that can be updated as conventions drift.

GitHub author fields only capture less than 1% of AI-authored PRs.
Open source video
PR footers and branch name prefixes provide much clearer signals of AI authorship.
Open source video
Practicehigh noveltymoderate evidence

Guardrails should be specialized per agent rather than shared: because each coding agent has a distinct empirical failure signature (Claude → injection, Devin → auth bypass relative advantage, plus IDOR / N+1 / regression / missing-test patterns), a single universal linter set is a mismatched allocation of review effort.

Why it matters

Shifts static analysis and review configuration from a repo-wide constant to a per-agent (and per-model-version) policy layer. Practically: maintain an agent→failure-profile mapping, pin it to the model version in use, and re-measure when the model is upgraded.

Generalization

In agentic systems, post-generation validation should be configured against the generator's measured weakness profile; capability changes at the model layer imply a corresponding change to the validation layer.

Team need targeted guardrails and linters tailored to the specific failure tendencies of the AI agent they use.
Open source video
There is significant variation in failure modes across different AI agents.
Open source video
Mental Modelmedium noveltymoderate evidence

The marginal cost of AI code is not reviewer attention: review iteration rounds before merge were statistically indistinguishable between AI and human PRs. The binding constraint moves downstream to validation capacity — sandboxing, testing, and CI throughput.

Why it matters

Redesigning the pipeline around 'AI PRs will need many more human review cycles' would target the wrong bottleneck. The throughput limiter is compute-bound verification (build/test/sandbox execution), not human judgment turns.

Generalization

When generation becomes cheap and roughly correct, the system bottleneck relocates from review latency to verification throughput; architectures should therefore invest in parallel sandbox execution, incremental test selection, and machine-checkable properties rather than in reviewer UI.

There is negligible statistical difference in the number of review rounds required for AI vs human PRs to merge.
Open source video
How can we scale code validation and sandboxing to match agent output velocity?
Open source video
Predictionmedium noveltymoderate evidence

Full-autonomy agentic coding crossed an adoption threshold recently (framed as December) and AI-authored PRs then grew monotonically to 27.6% of reviewed PRs by April — i.e., roughly a quarter of enterprise PR volume now carries strong evidence of full AI authorship.

Why it matters

At ~25%+ of PR volume, agent-authored code is no longer an exception path: review rules, branch protection, blame/ownership, incident attribution, and compliance evidence must treat machine authorship as a default case rather than a special case.

Generalization

Infrastructure thresholds should be defined by the volume fraction at which an exception becomes a normal path; once agent output exceeds ~10-25% of a queue, any human-only assumption in that queue is a capacity bug.

December marked a watershed moment when coding agents became fully autonomous.
Open source video
A quarter of all pull requests reviewed by Greptile in April showed strong evidence of full AI authorship.
Open source video

Deep dives

4

Per-vulnerability-class failure signatures as the discriminative eval signal

Research question

When aggregate metrics (revert rate, P0 count, review rounds) converge between agent and human output, do per-vulnerability-class rates remain a stable, reproducible signature of a given agent and model version — and does that signature transfer across repositories and domains?

Why

If parity on aggregate metrics is real, then the only remaining signal separating agents is the shape of their residual error distribution. Building evals, routing policy, or guardrails on a single scalar quality score would silently hide the classes where a given agent is materially worse than a human.

There is very little correlation between AI authorship and higher revert rates compared to humans.
Open source video
Three out of four AI coding agents tested produced fewer P0 review comments than humans.
Open source video
Source video

Circularity when the reviewer is also the measuring instrument

Research question

How much of the reported AI-vs-human quality parity survives when the same PR sample is scored by independent reviewer models and human experts rather than by the vendor's own review agent, and what is the inter-rater agreement on P0 classification?

Why

Every quality number in the talk is produced by an automated reviewer whose blind spots are correlated with the generators being judged. Without cross-rater agreement data, 'AI code is competitive' is a claim about one reviewer's preferences, not about code quality.

Three out of four AI coding agents tested produced fewer P0 review comments than humans.
Open source video
There is negligible statistical difference in the number of review rounds required for AI vs human PRs to merge.
Open source video
Source video

Where the bottleneck actually relocates when generation becomes cheap

Research question

If review iteration rounds stay flat while agent-authored PR share rises past a quarter of volume, is the binding constraint compute-bound verification (build, test, sandbox execution) rather than human review turns — and at what PR volume does sandbox/CI throughput become the throughput limiter?

Why

Architectural investment follows the bottleneck. Building reviewer UI and review-assist tooling targets a constraint the data says is not binding; investing in parallel sandbox execution, incremental test selection, and machine-checkable properties targets the one that is.

There is negligible statistical difference in the number of review rounds required for AI vs human PRs to merge.
Open source video
A quarter of all pull requests reviewed by Greptile in April showed strong evidence of full AI authorship.
Open source video
Source video

Provenance as a heuristic classifier over artifact side channels

Research question

How reliable and how drift-prone are footer and branch-prefix heuristics as agent-provenance classifiers, and what fraction of true agent-authored PRs do they miss relative to identity fields?

Why

Both adoption statistics and per-agent quality comparisons inherit the recall of the detection rule. If provenance lives in incidental scaffolding conventions rather than first-class identity, every reported rate is a convention-dependent lower bound, and the classifier must be re-validated as tool conventions change.

GitHub author fields only capture less than 1% of AI-authored PRs.
Open source video
PR footers and branch name prefixes provide much clearer signals of AI authorship.
Open source video
Source video

Article ideas

4

Your Agent's Quality Score Is Hiding Its Worst Bug

Aggregate quality metrics saturate as soon as an agent is competent enough, so a single 'does the agent break less often' score is a measurement failure: the residual risk concentrates in specific vulnerability classes, and a harness that reports one number will declare false parity exactly when the agent's weaknesses matter most.

Angle

Take the reported parity on revert rate and P0 comments at face value, then show what it conceals: an agent at ~1.5x human SQL injection rate while beating humans on aggregate. Argue for stratified reporting as the default eval contract.

Source video

The Reviewer Is the Instrument: Why Vendor-Scored Code Quality Doesn't Settle the AI-vs-Human Question

Quality claims scored by the same class of automated reviewer that gates the code under test are circular; defensible parity claims require multi-rater panels with published agreement statistics, particularly on the highest-severity classification.

Angle

Methodology critique: separate the reviewer's judgment from the measurement of that judgment, and specify the minimal study design (shared sample, independent raters, agreement on P0) that would make the claim falsifiable.

Source video

Stop Sizing Your Review Process for a Bottleneck That Moved

If review iterations to merge are statistically indistinguishable between agent and human PRs while agent output reaches a quarter of volume, then review latency is not the constraint — verification throughput is, and pipeline investment should shift from reviewer experience to sandbox parallelism, incremental test selection, and machine-checkable properties.

Angle

Capacity-planning argument with an explicit threshold: find the PR-volume fraction at which your sandbox/CI pool, not your reviewers, becomes the limiter, and re-architect before crossing it.

Source video

The Agent's Name Is in the Footnotes

Agent provenance is not an identity problem but a metadata-archaeology problem: identity fields capture almost nothing, provenance leaks through scaffolding conventions, and any adoption or quality metric built on those conventions is a lower bound that silently decays as tools change their footers.

Angle

Instrumentation essay: treat provenance as a pluggable, versioned classifier with measured recall, and show why teams that wait for a first-class field will undercount their own agent adoption by an order of magnitude.

Source video

Project ideas

4

VulnClass Profile Harness

beyond-evals

Per-vulnerability-class failure rates form a stable signature per agent+model-version that is discriminative even when aggregate revert rates and P0 counts show parity, and the signature shifts measurably (not marginally) across model version bumps.

Proof of concept

Replay a fixed corpus of repository tasks through two or more coding agents on pinned model versions; classify resulting diffs into vulnerability classes (injection, authz/IDOR, data-access, N+1, regression, missing tests) with a rubric scored by two independent reviewer models plus one human adjudicator; emit a per-agent failure-profile vector with confidence intervals.

Measurement

Per-class defect rate per thousand diffs with inter-rater agreement; rank-order stability of the profile across model snapshots and across two repository domains; discriminative power (AUC) of the per-class vector versus a single aggregate quality score.

Source video

Guardrail Co-Versioning Layer

gatehouse

Validation policy conditioned on the producing agent/model version (class-specific checks targeted at that agent's measured weakness profile) reduces that class's defect rate without a compensating increase in other classes, and a model upgrade implicitly invalidates the calibration until re-measured.

Proof of concept

Maintain a versioned agent→failure-profile artifact; gate PRs with class-specific checks (e.g. parameterized-query lint for an injection-prone agent, authz invariant tests for an authz-prone one); A/B rollout on a real repo with agent attribution from metadata heuristics.

Measurement

Target-class defect rate before/after rollout versus a control branch; compensating-regression rate (defects introduced in other classes); delta in guardrail calibration accuracy immediately after a pinned model version change.

Source video

Provenance Recall Auditor

movement-lab

A classifier ensemble over PR metadata side channels (description footers, branch prefixes, commit trailers) detects agent authorship with materially higher recall than author identity fields, and its recall is convention-dependent such that reported adoption share is a lower bound drifting over time.

Proof of concept

Build a pluggable classifier over a large PR corpus; hand-label a stratified sample to estimate precision/recall per signal and per ensemble; track recall of each signal monthly to detect convention drift.

Measurement

Recall/precision against hand-labeled ground truth versus the author-field baseline; estimated undercount factor on headline adoption percentages; monthly drift in per-signal recall.

Source video

Verification Throughput Planner

new

Once agent-authored PRs exceed a measurable share of a queue, end-to-end merge latency is dominated by sandbox/build/test execution capacity rather than human review turns, so adding parallel sandbox capacity and incremental test selection reduces merge latency more per unit cost than adding review tooling.

Proof of concept

Instrument a pipeline to decompose merge latency into review-turn time vs verification-execution time as a function of agent-authored PR share; simulate added sandbox parallelism and incremental test selection against the observed queue.

Measurement

Share of merge latency attributable to verification execution at increasing agent-PR fractions; predicted vs observed merge-latency reduction per unit of added sandbox capacity versus per unit of reviewer-tooling investment.

Source video

Architectural implications

3

PR quality was evaluated with three measurable signals: revert rate, severity-graded review comments (P0/P1/P2), and count of review rounds to merge.

Before

Merge/no-merge and human reviewer judgment were the de facto quality signal for code contributions.

After

Quality becomes a continuously instrumented vector (post-merge revert rate + severity-classified findings + iteration count) attached to each contribution and each agent.

Consequence

The CI/review layer needs an agent-performance telemetry store and severity taxonomy as first-class infrastructure, not a dashboard bolted on afterwards.

Source video

AI code is validated downstream by automated review agents operating with full codebase context on enterprise repos.

Before

Review responsibility sat with humans who reconstructed intent from a diff.

After

Verification responsibility shifts toward machine reviewers with repository-wide context, with humans adjudicating escalated findings.

Consequence

Context retrieval over the whole codebase becomes a correctness dependency of the review stage; a reviewer agent's recall of cross-file invariants (auth checks, query patterns) is now what gates production code.

Source video

Failure profiles differ per agent, requiring agent-specific lint and guardrail configuration.

Before

One repository-wide static-analysis ruleset applied uniformly to all contributors.

After

Validation policy is conditioned on the producing agent/model version, with rule emphasis tied to that agent's measured weakness profile.

Consequence

Policy becomes a versioned artifact coupled to model versions; a model upgrade silently invalidates the guardrail calibration unless the failure profile is re-measured.

Source video

Tradeoffs and failure modes

3

Autonomous code generation vs. security-class exposure

Benefit

AI-authored PRs matched or beat humans on revert rate and P0 find rate, making autonomous generation viable for production repos.

Cost or risk

Parity on aggregate does not mean parity per class — a specific model is materially more likely to emit one vulnerability class (SQL injection) while another is comparatively less likely to emit another (auth bypass).

Claude is 1.5x more likely to produce SQL injection errors than humans.
Open source video
Source video

Cheap provenance heuristics vs. measurement validity

Benefit

Footer and branch-prefix heuristics make it possible to detect AI authorship at scale without platform cooperation.

Cost or risk

Detection depends on incidental conventions; the author field captures almost nothing, so reported adoption and per-agent quality numbers are convention-dependent and likely undercount.

GitHub author fields only capture less than 1% of AI-authored PRs.
Open source video
Source video

Automated review at agent output volume

Benefit

Machine review with full-codebase context keeps iteration rounds flat even as AI PR share grows to roughly a quarter of volume.

Cost or risk

The review gate is itself an agent, so review capacity and review calibration become new dependencies; scaling verification is the open constraint rather than scaling generation.

There is negligible statistical difference in the number of review rounds required for AI vs human PRs to merge.
Open source video
Source video

Open questions

4

Do revert rates and review-comment counts fully capture agent-caused defects, or do they miss failures that survive to production without an explicit revert?

Why unresolved

The summary reports revert rates and reviewer comments as the quality proxies but gives no data on silent incidents, post-merge bug linkage, or defects caught before PR creation.

Research direction

Link merged-CHANGE attribution to incident/bug-tracker records and compare defect-escape rates stratified by contributor type.

Source video

How much of the reported AI-vs-human quality parity is an artifact of the reviewing model's own blind spots, given the reviewer and the measurement instrument are the same automated system?

Why unresolved

Quality was scored using the vendor's own severity-graded review comments; no independent review panel or cross-reviewer agreement data is described.

Research direction

Score a shared PR sample with multiple independent reviewer models plus human experts and report inter-rater agreement, especially on P0 classification.

Source video

Are agent-specific failure profiles stable across model versions and repository domains, or do they drift on each upgrade and by codebase?

Why unresolved

The summary presents a single cross-agent failure-pattern comparison without version pinning or per-repo breakdown.

Research direction

Re-run the failure-class measurement per model snapshot and per repository, treating the failure profile as a tracked metric rather than a fixed property.

Source video

What guardrails actually reduce agent-specific failures like SQL injection without introducing compensating regressions elsewhere?

Why unresolved

The talk recommends tailored guardrails as an implication but presents no before/after data on guardrail effectiveness.

Research direction

A/B rollout of class-specific checks (parameterized-query linters, authz invariant tests) against the measured baseline failure rates per agent.

Source video

Key claims

7
factualVerification needed

AI-authored pull requests reached 27.6% of PRs reviewed by April, up from under 1% in early the prior year.

Evidence

The percentage of AI-generated pull requests is growing exponentially month over month, reaching 27.6% by April.

Question

What is the exact denominator, sampling window, and detection rule behind the 27.6% figure, and does it hold outside Greptile's customer base?

Source video
comparativeVerification needed

Codex-authored PRs are reverted less often than human-authored PRs (1.2 vs 2.7 per thousand).

Evidence

Codex revert rate of 1.2 per thousand vs human revert rate of 2.7

Question

Is the comparison controlled for PR size, change criticality, repo, and team, and does it hold across agents other than Codex?

Source video
comparativeVerification needed

Three of four tested coding agents generated fewer P0-severity review findings than human contributors.

Evidence

Three out of four AI coding agents tested produced fewer P0 review comments than humans.

Question

Which agent failed this comparison, and does P0 classification remain consistent when assigned by an independent reviewer rather than the same automated reviewer?

Source video
comparativeVerification needed

Number of review iterations before merge is statistically indistinguishable between AI and human PRs.

Evidence

There is negligible statistical difference in the number of review rounds required for AI vs human PRs to merge.

Question

Is this measured on matched PR-size and complexity strata, and does it hold in repos where AI-authored PRs are a minority?

Source video
comparativeVerification needed

Claude-authored code carries roughly 1.5x the SQL injection rate of human-authored code.

Evidence

Claude is 1.5x more likely to produce SQL injection errors than humans.

Question

What is the absolute base rate per thousand PRs, and is the ratio stable across model versions and languages/frameworks?

Source video
comparativeVerification needed

Devin-authored code is about half as likely as human code to produce auth bypass issues.

Evidence

Devin is about half as likely as humans to produce auth bypass issues.

Question

Is the reduced auth-bypass rate a genuine behavioral property of the agent, or an artifact of the task types it is assigned in the sampled repos?

Source video
predictionVerification needed

Full autonomy in coding agents began around December, marking a discontinuity in AI code volume.

Evidence

December marked a watershed moment when coding agents became fully autonomous.

Question

What operational threshold defines 'fully autonomous' here, and does the volume curve show a genuine breakpoint at December rather than smooth exponential growth?

Source video

Connections

4