Your Inference Bill Is a Power Bill: Why Tokens-Per-Joule Replaces Cost-Per-Million-Token
Once the data center is modeled as a factory with energy in and tokens out, the controlling variable for agent serving economics is power and thermal headroom, not GPU count or API price, so schedulers, admission control, and capacity commitments built around dollar cost are optimizing a proxy target and will degrade under grid or cooling constraints.
AngleReframe inference cost modeling around physical input constraints and show which runtime design decisions (batching, deferral, routing, throttling) change when the objective becomes tokens-per-joule at fixed SLO.
Source video ↗The Compliance Surface You Already Have: Extending Existing Control Planes to Autonomous Systems
Applying cybersecurity, product liability, and SLA regimes to AI before writing new rules is not just a regulatory stance but an architectural one: teams should extend proven audit, access-control, and incident-response abstractions to agents, and the honest engineering work is finding the subset of autonomous behaviors, chiefly unattributed actions, that those abstractions cannot reach.
AngleTreat existing-law-first as a control-plane reuse principle, then enumerate the concrete agent failure modes that fall outside it.
Source video ↗Your Guardrails Passed Review and Failed Production
Controls validated under low-volume, human-reviewed research conditions do not become automated enforcement at production volume by default; the gap between validation method and enforcement method is where high-throughput safety failures originate, and closing it requires statistically monitored policy enforcement with rollback rather than additional sample-based evaluation.
AngleContrast sampled validation and scaled automated enforcement as two distinct systems that must be engineered separately for any control that ships to production.
Source video ↗JouleScheduler
movement-labAn admission-control scheduler whose policy is parameterized on power headroom and measured tokens-per-joule will hold p99 latency SLOs equal to cost-only scheduling while consuming measurably fewer joules per served request under simulated grid-capacity variation.
Proof of conceptInstrument a small multi-model agent serving stack with per-request energy proxies derived from GPU power draw and utilization, implement dual schedulers (cost-only vs power-headroom-aware) behind a shared queue, and replay a synthetic variable-capacity workload trace through both.
MeasurementJoules per completed request, p99 and p50 latency, and SLO violation rate for each scheduler across identical workload traces and capacity profiles.
Source video ↗LabToProd Guardrail Degradation Harness
beyond-evalsFor a given guardrail, breach rate under production-volume automated enforcement will exceed the breach rate implied by low-volume sampled review, and the divergence will be detectable by continuous statistical monitoring before it is detectable by periodic review.
Proof of conceptTake two or three sample-validated agent guardrails (tool-allowlist enforcement, prompt-injection refusal, output-policy filtering), replay a fixed adversarial corpus at 1x, 100x, and 10000x request volume with and without automated enforcement, and compare sequential-detection monitors against batch review cadence.
MeasurementDivergence between sampled-review breach estimate and observed automated-enforcement breach rate per volume tier, plus mean requests-to-detection for each monitoring method.
Source video ↗Unattributed-Action Coverage Map
gatehouseA defined subset of autonomous-agent failure modes (unattributed action, unverifiable state, cascading tool calls) will map to no existing liability, contract, or unauthorized-access regime, and that uncovered subset can be enumerated exhaustively for a given agent capability set.
Proof of conceptBuild a failure-mode taxonomy for a concrete tool-using agent, map each mode against product liability, SLA and breach-law criteria expressed as checkable predicates (identifiable actor, discrete defective artifact, enforceable contract term), and flag modes failing all predicates.
MeasurementCount and classification of failure modes that satisfy no existing-regime predicate, plus inter-rater agreement on the mapping to test whether the uncovered set is stable or definitional.
Source video ↗Generation-Isolated Serving Benchmark
movement-labWhen model, quantization, batch size and serving stack are held constant, the tokens-per-second-per-watt gap between adjacent accelerator generations is measurably smaller than the multi-year generational lead claim implies.
Proof of conceptDeploy one fixed open-weight model on a fixed serving stack across two accelerator generations, sweep batch size and concurrency, and record throughput, latency and power draw to compute tokens-per-second-per-watt and cost-per-million-token under matched conditions.
MeasurementTokens-per-second-per-watt and cost-per-million-token ratios between generations at fixed latency targets, with sensitivity analysis over batch size and utilization.
Source video ↗