Your Token Bill Is 2.5% of the Problem
Teams that instrument and optimize inference spend are optimizing the wrong line item: in a blended cost-per-merged-PR model, AI is a rounding error next to salary, so token-efficiency work can be locally successful and globally negative-value while interventions that raise artifacts per human cycle capture almost all the leverage.
AngleTake the 2.5%/97.5% split seriously and walk through what it invalidates — per-call cost dashboards, request-level rate limits, model-routing micro-optimizations — then specify the metric that should replace them.
Source video ↗A 60x Cost Increase That Lowered Unit Cost
A 60x rise in AI spend per PR coexisting with falling total cost per PR is not a paradox but arithmetic: when the dominant cost is quasi-fixed salary, any intervention that raises shipped artifacts per person swamps order-of-magnitude growth in the marginal cost term, which means 'spend growth' is the wrong budget alarm and 'artifacts per human cycle' is the right one.
AngleDecompose the ratio into numerator and denominator, show why finance intuition fails on fixed-cost amortization, and give the sensitivity test leaders should run before cutting AI budgets.
Source video ↗Stop Throttling Your Best Users
Blanket token caps are a governance default that suppresses exactly the cohort with the best unit economics: cohorting by consumption quintile turns a raw spend dashboard into a dose-response curve, and that curve argues for usage-tiered budgets rather than one global limit.
AngleContrast aggregate spend reporting with quintile outcome-per-cost reporting, and show how a single global cap conflates a runaway loop with a high-throughput power user.
Source video ↗Cost Observability Lives One Layer Too Low
Per-request token accounting cannot answer ROI questions because the decision-useful cost surface is the join of inference telemetry, workflow acceptance events, and human cost; until agent sessions are resolvable to a person and an accepted artifact, every AI cost dashboard is a billing artifact masquerading as an economics tool.
AngleSpecify the event contracts and identity keys required for the join, and show which questions become unanswerable when the join is absent.
Source video ↗Blended Unit-Cost Joiner
gatehouseA pipeline joining inference telemetry, acceptance events, and loaded human cost produces a cost-per-accepted-artifact number whose rank ordering of teams differs materially from a token-spend ranking, and whose variance is dominated by the artifact-count denominator rather than the AI-spend numerator.
Proof of conceptIngest gateway token logs, VCS merge events, and synthetic compensation bands; resolve agent session to developer to merged change; emit daily blended cost per accepted artifact with a numerator/denominator variance decomposition. Run on 6 months of backfilled data and compare team rankings under blended cost versus token spend.
MeasurementSpearman rank correlation between token-spend ranking and blended cost-per-artifact ranking (hypothesis: weak or negative); share of cost-per-artifact variance attributable to denominator movement; percentage of agent sessions successfully joined to an accepted artifact.
Source video ↗Budget-Raising Rollout Trial
movement-labRaising context, tool-call, and step budgets for a matched cohort raises accepted artifacts per human review cycle enough that blended cost per accepted artifact does not increase, and the effect is not explained by pre-existing team velocity.
Proof of conceptStaged enablement: randomly assign teams to raise harness budgets at staggered dates; measure reviewed-and-merged changes per developer-week and human review iterations per accepted change before and after each switch, with a matched holdout.
MeasurementPre/post change in accepted artifacts per developer-week and in human touches per accepted artifact; blended cost per accepted artifact; test for pre-trend differences between treated and holdout to separate effect from selection.
Source video ↗Human-Touch Budget Harness
beyond-evalsCapping human review iterations and rework loops per accepted change reduces blended cost per artifact more than an equivalent tightening of token budgets, because human touches carry the dominant cost weight.
Proof of conceptInstrument an agent harness to count human interventions, review round-trips, and rework cycles per candidate change; run two policies against the same task stream — token caps versus touch caps — and compare blended cost per accepted artifact at equal acceptance quality.
MeasurementHuman touches per accepted artifact, acceptance rate, and blended cost per accepted artifact under each policy; sensitivity of total cost to a 10x token-cost change versus a 10% change in touches per artifact.
Source video ↗Quintile Dose-Response Dashboard
newBucketing users or agents into consumption deciles and reporting outcome-per-cost per bucket reveals an interior saturation point that a single aggregate spend number conceals, and the saturation point differs by workflow tier.
Proof of conceptBuild a reporting layer over existing telemetry that plots throughput gain and cost per accepted artifact against continuous spend deciles (p10/p50/p90/p99 rather than quintile labels), and check whether the reported spread is consistent with per-developer versus per-org aggregation.
MeasurementCurve shape and location of any plateau or inflection; consistency of the spend spread across aggregation levels; number of teams whose optimal budget recommendation differs under deciles versus quintiles.
Source video ↗