Your AI Productivity Metric Is Measuring the Wrong Stage
Instrumenting productivity at the generation layer is not merely incomplete but actively misleading: 20x more generated code alongside 30% more releases means the measured gain is real at one stage and unreal end-to-end, so any harness or dashboard that reports generation volume will misdirect investment toward the stage that stopped being the constraint.
AngleEngineering-metrics teardown: use the 20x/30% ratio to argue that every AI productivity dashboard needs downstream attribution before it needs more throughput.
Source video ↗Capability Is Necessary but Not Sufficient: Why Diffusion Is the Real Gate
Extreme AI economic scenarios are not capability-constrained but adoption-constrained, so engineering and infrastructure roadmaps that depend on those scenarios should budget for rollout, integration, and change-management scaffolding rather than for the next capability step.
AngleContrarian planning argument framing deployment scaffolding as the dominant term in realized value.
Source video ↗Stop Modeling Roles as Replaceable Units: The Case for Task-Level Automation Coverage
The finding that half of jobs contain AI-used tasks while no occupation is fully automated is not a transitional curiosity but a structural property of automation, and it invalidates role-level abstractions for scoping agents, evals, and handoffs — the engineering problem is the seam between automated and human tasks, and role-level models cannot see it.
AngleArchitecture argument that reframes the unit of design from role to task and makes handoff a first-class primitive.
Source video ↗Human-in-the-Loop Is a Capability, Not a Safety Net
Treating human review as an exception path after agent completion discards its actual value: HITL raises success rates and unlocks harder tasks, which means checkpoint placement is a performance optimization problem bounded by human attention, not a compliance checkbox.
AngleHarness design piece arguing checkpoint density, escalation policy, and partial-state handoff belong in the runtime, not the UI.
Source video ↗Downstream Loss Attribution Harness
beyond-evalsIn an agent-assisted code pipeline, the majority of the gap between generated artifacts and shipped releases is attributable to a single downstream stage (review or CI/merge), and automating or adding back-pressure at that stage increases release cadence materially more than further generation throughput increases.
Proof of conceptInstrument one real repository end-to-end: count generated artifacts, then attribute loss across review queue time, CI runtime, merge conflicts, integration failures, and release approval. Then add a single intervention (e.g., review-side batching or CI parallelization) and re-measure.
MeasurementPercent of generated artifacts reaching release, before and after the intervention, plus a stage-level attribution breakdown showing which stage accounts for the largest share of the generation-to-release gap.
Source video ↗Checkpoint Density Sweep
movement-labSuccess rate on long-horizon tasks is a non-monotonic function of human checkpoint density: it rises with added checkpoints up to a crossover point and then declines as human attention becomes the dominant failure and latency term.
Proof of conceptTake a fixed suite of multi-hour tasks and run it at several checkpoint densities (no human, per-step, per-subtask, per-phase), recording success rate, wall-clock latency, and human minutes consumed at each density.
MeasurementSuccess rate versus checkpoint density curve with a located crossover point, plus human-minutes-per-success at each density to test whether the peak-success density is also the most efficient one.
Source video ↗Per-Task Token Attribution Meter
gatehouseToken consumption can be attributed reliably to individual tasks, tenants, and workflows at runtime, and doing so changes operational decisions (routing, model selection, agent depth) in ways that aggregate dashboards do not surface.
Proof of conceptWrap an agent runtime so every model call carries task, tenant, and workflow identifiers; emit per-task token ledgers and expose a per-task cost view. Then run an A/B where routing decisions are made from per-task costs instead of aggregate spend.
MeasurementAttribution accuracy (share of tokens traceable to a specific task/tenant with no unallocated residual) and the measured change in per-workflow cost after routing decisions switch to per-task signals.
Source video ↗Diffusion Threshold Solver
newFor a fixed capability trajectory, the adoption and diffusion rates required to reach the extreme AI economic scenario are quantifiably higher than observed historical general-purpose technology adoption curves, making the extreme scenario falsifiable against current adoption data.
Proof of conceptBuild a minimal parameterized adoption model over a historical productivity baseline, solve for the adoption/diffusion curve implied by each scenario, and compare it against observed task-level AI usage adoption rates over time.
MeasurementThe implied required adoption rate per scenario versus observed adoption rate, expressed as a gap factor, and the sensitivity of scenario outcomes to a one-standard-deviation change in diffusion rate.
Source video ↗