Your Agent Doesn't Need a Better Model, It Needs a State Observation Step
Open-loop agent pipelines fail on long-horizon tasks not because the model is weak but because no stage observes the resulting system state; the highest-leverage architectural change is inserting continuous state estimation and external-actor modeling into the loop, not upgrading the model inside it.
AngleTake the standard read-task → plan → tool calls → emit-result harness and re-draw it as Goals → Decisions → State → Side Effects → Actions of Others, showing what has to be built at each seam and why 'actions of others' belongs inside the loop rather than outside it as noise.
Source video ↗Side Effects Are Just Model-External Feedback: Why Your Observability Scope Must Exceed Your Reasoning Scope
Unintended consequences are not a separate category of event but the return path of whatever the agent's representation omitted, which makes the model boundary itself the risk surface and makes boundary instrumentation a design requirement rather than a monitoring afterthought.
AngleDefine side effects operationally as effects outside the mental model that feed back, then derive the rule that whatever the context window, summary, or memory store excludes will re-enter as unexplained failure unless the gap between observability scope and reasoning scope is explicitly instrumented.
Source video ↗Too Clever for Our Own Good: Why Sophisticated Agents Fail Inside Wrong Boundaries
Sophistication applied inside a flawed model boundary produces confidently wrong interventions, so failures occur because of our cleverness rather than despite it; the corrective is not more analytical power but better boundary selection and decision-rule change.
AngleUse the short-term time horizon trap and open-loop project management as the pattern: local optimization against an explicit objective that omits feedback generates long-horizon regressions, and each round of added sophistication makes the resulting intervention more elaborate and more wrong.
Source video ↗The Model-Swap Reflex: Treating Every Agent Failure as a Structure Problem First
Attributing agent failures to the model or the prompt author is the component-replacement reflex and is almost always low-leverage, so root-cause processes should require a structural hypothesis about physics, information availability, or decision rules before a component-level fix is accepted.
AngleWalk the causal checklist (what the environment permits, what the agent can actually see at decision time, which policy it applies) against common agent incidents and show how the model-swap fix leaves the structural cause intact so the failure recurs.
Source video ↗Loop-Closure Bench: open-loop vs closed-loop agent harnesses
beyond-evalsOn long-horizon tasks whose external state changes mid-execution, a closed-loop harness with an explicit state-observation step will show a lower regression rate per task than an open-loop harness with the identical model and tools, and the cost per completed task will not exceed the reliability gain by more than a stated factor.
Proof of conceptBuild two harnesses over the same model, tools, and task suite: (A) plan-once-then-execute, (B) plan plus a measured-state re-planning step and a side-effect channel. Inject mid-episode external state changes and score final outcome correctness.
MeasurementRegression frequency (tasks that succeed at the checkpoint but fail at completion), absolute success rate, and cost/tokens per completed task for each harness, held against the same model and prompt.
Source video ↗Boundary Gap Instrument: telemetry for effects the agent cannot observe
gatehouseFor each agent run, effects outside a declared model boundary that are causally adjacent to the action can be enumerated and instrumented, and detecting them during the run reduces the fraction of failures that surface only as unexplained terminal errors.
Proof of conceptDeclare an explicit model boundary per task, build a telemetry sidecar that records out-of-boundary effects (downstream consumer state, cost, rate limits, other systems' state), and compare incident classification with and without the sidecar on a replay of past failures.
MeasurementShare of failures classified as boundary-crossing feedback versus genuine internal error, and mean time between the causal action and the detection of its out-of-boundary effect.
Source video ↗Substrate-vs-Policy A/B: does infra-only upgrading move agent behavior?
beyond-evalsUpgrading the substrate (more tools, larger context, better retrieval) while holding the decision policy fixed produces behavior drift below a pre-registered threshold on a fixed task suite, while pairing the same upgrade with an explicit decision-policy change produces drift above it.
Proof of conceptRun three arms on the same task suite and model: (A) baseline, (B) substrate upgrade only, (C) identical substrate upgrade plus a changed decision policy. Freeze prompts and tool schemas across arms except for the substrate variable.
MeasurementBehavior drift as change in action-sequence distribution and outcome scores between arms A and B versus A and C, with a pre-registered minimum effect size for 'behavior changed'.
Source video ↗Practice-Traces vs Instructions for tool-use skill acquisition
movement-labFor an agent acquiring a tool-use skill, accumulated practice traces (replay, experience memory) yield higher held-out task success than declarative instruction alone, and the gap widens as the number of dependent sub-steps in the skill increases.
Proof of conceptGive two agent configurations the same tool: one receives a written specification and worked examples, the other receives a practice loop with stored trajectories and replay. Scale task complexity by chaining sub-steps and measure transfer to unseen variants.
MeasurementHeld-out success rate on unseen task variants at each complexity level, plus error-recovery rate after a mid-task tool failure, for the instruction-only versus trace-accumulating configuration.
Source video ↗