The 80/20 Boundary Is the Real Agentic Engineering Problem
The limiting factor in agentic software development is not autonomy but the human review boundary. Vendors claiming whole-repo context and 5x velocity are really selling a new review architecture, and unless the retained 20% human-guided work scales sublinearly, the velocity claim collapses under review throughput.
AngleArchitectural argument that reframes agent benchmarks around the human review fraction rather than benchmark task completion, with implications for context ingest cost, latency, and team topology.
Source video ↗Liability Is the Only Alignment Guardrail That Scales
Voluntary incident reports and alignment documentation are insufficient safety mechanisms at production scale. Enforceable strict liability on labs and operators is what forces provenance, replayability, and auditability into agent architecture, because those artifacts are the evidence used to assign responsibility after a failure.
AngleInverts the standard safety narrative by treating audit logging and version pinning as legal requirements rather than engineering preferences, and argues that liability exemptions requested alongside slowdowns function as regulatory capture.
Source video ↗Scaling Curves Do Not Prove Generalization: Environment Holdouts Do
Smooth validation-loss curves in robotics and code agents measure in-distribution fit, not deployment readiness. For systems dropped into environments they were never prepared for, the only meaningful evaluation holds out entire environments and measures zero-shot success, not held-out samples from seen environments.
AngleMethodological critique of scaling-law-as-deployment-evidence claims, with a concrete evaluation design that partitions by environment novelty rather than by data split.
Source video ↗Tokenizing Equities Does Not Remove Risk, It Moves It
The composability gain of tokenized securities is real, but it swaps settlement risk for smart-contract, oracle, and liquidity risk while creating unresolved cross-border enforcement gaps. No chain-level primitive currently solves the legal-ownership and counterparty questions that legacy clearing plumbing implicitly answered.
AngleRisk-transfer analysis of on-chain equities, contrasting legacy clearinghouse guarantees with composable DeFi failure modes, and mapping where regulation breaks under 24/7 settlement.
Source video ↗RepoScale Bench
beyond-evalsFor legacy migration tasks on repositories exceeding 10M lines of code, a single-pass full-context agent achieves higher end-to-end correctness than a retrieval-augmented agent at comparable human review hours, but at higher token cost and wall-clock latency, and the human review fraction becomes the throughput bottleneck regardless of grounding strategy.
Proof of conceptSelect an open-source multi-million-line monorepo, define a deterministic migration or refactor task (e.g., a language-version upgrade or a mechanical API migration), run two agent configurations (single-pass full-context vs retrieval-augmented) under identical human review budgets, and log every human edit per accepted change.
MeasurementEnd-to-end task success rate after human review, total token cost, wall-clock latency, and fraction of human edits per accepted change, compared across the two grounding strategies.
Source video ↗Liability-Replay Agent Trace
gatehouseA minimal decision-provenance log (model version, tool-call sequence, inputs, outputs, confidence, and human approvals) is sufficient for an independent third party to reconstruct and attribute an autonomous agent failure in a post-hoc review with high inter-reviewer agreement.
Proof of conceptInstrument a production-like agent (e.g., a code-migration or customer-workflow agent) to emit a structured trace, inject a controlled failure scenario, and have multiple independent reviewers attempt attribution using only the trace, without access to the agent's internal state.
MeasurementAttribution accuracy, time-to-attribution, and inter-reviewer agreement across independent reviewers using only the emitted trace.
Source video ↗Unseen-Home Eval
beyond-evalsPre-training data volume predicts zero-shot task success in held-out homes only up to a plateau; beyond a threshold, environment-specific factors such as layout diversity and novel object categories dominate task success more than additional data volume.
Proof of conceptBuild a simulation-plus-real-home holdout suite in which no evaluation home shares layout or object categories with any training home, vary the effective pre-training data volume, and measure zero-shot task success stratified by an environment-novelty score.
MeasurementZero-shot task success rate as a function of pre-training data scale, stratified by environment novelty score, and the point at which additional data yields diminishing gains in genuinely novel homes.
Source video ↗Autonomy Ladder Instrument
movement-labDelegating specific ML R&D pipeline stages to AI, graded per AL0-AL5, measurably shortens the next stage's cycle time, but only beyond a threshold of delegation density; below that threshold, partial automation introduces handoff overhead that negates the cycle-time gain.
Proof of conceptInstrument an actual ML pipeline (data curation, feature engineering, training, evaluation, deployment) to record per-stage AI-led versus AI-assisted status and per-stage cycle times over multiple iterations, then correlate stage i delegation level with stage i+1 cycle-time change.
MeasurementCorrelation between delegation level at stage i and cycle-time reduction at stage i+1 across iterations, and the delegation threshold below which handoff overhead outweighs the gain.
Source video ↗