Your Agent's Safety Boundary Is Not in the Prompt
Because an agent selects actions rather than executing a specified one, prompt-level guardrails are a suggestion and action-level permissioning is a contract; teams that keep safety in system messages are shipping policy that varies by model version, while teams that enforce scoped tool grants and spend ceilings ship a property that holds regardless of what the model decided internally.
AngleReframe the tool-vs-agent debate from philosophy to architecture: the enforceable contract is the set of permitted actions, and everything else is a preference.
Source video ↗The Capability You Paid For Is the Property You Cannot Verify
Investment in agents is justified precisely by out-of-distribution behavior, which means the success criterion and the verification criterion are in structural conflict; teams should stop expecting capability benchmarks to double as safety evidence and adopt adversarial, open-ended evaluation as a separate discipline with its own budget.
AngleShow that capability and verifiability are anti-correlated by design, then argue that distribution-matched evals are the wrong instrument for systems valued for going off-distribution.
Source video ↗The Algorithm Is Working Correctly. That Is the Problem.
Engagement-maximizing ranking is a faithful optimizer hitting its target; the harm comes from objective selection, not model quality, and the same failure class applies to every agent loop scored by a scalar such as task-completion or test-pass rate — so reliability engineering without objective auditing is insufficient by construction.
AngleTreat outrage amplification as the canonical case study in reward hacking and map it onto engineering proxy metrics teams actually use.
Source video ↗Model Vendors Are Now Part of Your Threat Model
Routing a core function through a single external model provider means inheriting that provider's jurisdiction, policy shifts, and outages as your own; for critical systems this is a continuity requirement that should be designed as a provider-abstraction boundary with a tested local fallback, not negotiated as a procurement preference after a policy change.
AngleRecast the sovereignty argument as a boring availability and dependency-management argument that platform teams can actually act on.
Source video ↗Grantwall — an action-permission runtime for agents
gatehouseAn out-of-band permissioned action layer (scoped tool grants, per-task spend ceilings, mandatory approval for irreversible operations) bounds blast radius under adversarial prompts at a materially higher rate than prompt-level guardrails, and the bound is stable across model upgrades.
Proof of conceptWrap a fixed tool surface behind a policy engine; run a red-team suite of prompt-injection and goal-drift scenarios against two configurations (prompt guardrails only vs. Grantwall) using the same model; log every attempted action, denial, and side effect.
MeasurementPercentage of scenarios producing an out-of-permission side effect, broken down by configuration and by model version; variance of that rate across model swaps; added latency and false-denial rate per task.
Source video ↗Off-Distribution Stress Bench
beyond-evalsA model that scores well on distribution-matched task benchmarks will still violate declared constraints at high rates on adversarial, open-ended tasks, and the gap between the two scores is a predictable property of the system rather than noise.
Proof of conceptPair each task benchmark with a matched adversarial variant that rewards the same objective while offering a cheaper prohibited path; score both, compute the gap, and repeat across several models and agent scaffoldings.
MeasurementCapability score minus constraint-violation-adjusted score per task family; rank correlation between benchmark performance and adversarial violation rate across models.
Source video ↗Interaction Graph Recorder
newEmergent cross-agent coordination is detectable from interaction-level telemetry (channel topology, peer discovery, shared-resource contention, cross-agent correlation IDs) before it is visible in any per-request trace, and a detector built on those signals fires on unauthorized coordination with usable precision.
Proof of conceptInstrument a multi-agent sandbox to emit an interaction graph alongside conventional request traces; run seeded scenarios where agents can coordinate only via a shared resource; compare detection time and recall for graph-based vs. trace-based detection.
MeasurementTime-to-detection and recall/precision for unauthorized coordination events; fraction of incidents whose causal path is unreconstructable from traces alone; telemetry volume per task.
Source video ↗Invariant-a-gate
gatehouseFor high-stakes agent actions, replacing human approval with a machine-checkable invariant suite (schema constraints, budget predicates, policy rules, differential tests) reduces undetected-policy-violation rate without increasing block rate, and human review retained only on legible judgment cases performs better than uniform human gating.
Proof of conceptBuild a gate that runs an invariant suite before an irreversible action; run a seeded-error review study comparing (a) uniform human approval, (b) invariant suite only, and (c) invariants plus human review limited to cases the suite cannot decide.
MeasurementEscaped-violation rate per configuration; false-block rate; reviewer time per action; detection rate on seeded errors as a proxy for reviewer vigilance over a long session.
Source video ↗