A Benchmark Without a Price Tag Is Not a Deployment Decision
Leaderboards that rank models on success rate alone cannot answer the only operational question teams have — which model to run — and the shift toward cost-normalized comparisons on named task tiers makes a whole class of 'we picked the best model' decisions retroactively unjustifiable.
AngleTake the reported 84% cost drop and show that a pure success-rate ranking would have produced the same model choice before and after the cost collapse, despite the deployment economics having inverted.
Source video ↗Alignment Is a Scoring Bug, Not a Prompt Bug
If agents and humans are both trained to win, then alignment failures emerge from the selection mechanism — who is scored on what, and what happens to losers — and system prompts and content filters cannot fix an objective that is embedded in a tournament.
AnglePair the claim that humanity lacks alignment among itself with the historical prize examples to argue that outcome-defined rewards shift where misalignment can occur rather than removing it, and that orchestration-layer scoring must itself be evaluated.
Source video ↗When Inference Becomes Free, Retries Become the Failure Mode
A three-week halving curve destroys the implicit budget that currently bounds agent loops; the constraint on agent design migrates from spend to latency, tool-call storms, and unbounded self-verification, which means cost-based guardrails are the wrong control surface going forward.
AngleArgue that near-zero marginal cost per subtask permits aggressive retry and multi-sample strategies but removes the economic signal that kept runaway loops in check, so new boundedness must come from explicit step budgets and latency budgets.
Source video ↗Put a Reviewable Artifact in the Middle of Your Code Agent
The most valuable structural choice in a long-horizon code agent is not the model but the placement of a generated specification between ingestion and generation, because it converts an unattributable failure into a localizable one and makes the human's residual share of the work measurable rather than rhetorical.
AngleUse the 80% autonomy claim as a prompt to ask what the remaining 20% costs in review time, and argue that the spec layer is what makes that question answerable at all.
Source video ↗