Your Agent Isn't Bad at Reasoning, Your Search Is Bad at Retrieving
Most apparent agent reasoning failures in retrieval-augmented systems are downstream symptoms of a weak retrieval harness, and the cheapest way to prove it is to watch tool-call counts and token spend inflate on failing runs before accuracy visibly degrades. Teams that tune prompts while leaving search untouched are optimizing the wrong layer.
AngleDiagnostic playbook: treat tool-call count and token spend as retrieval-quality telemetry, then walk through the fix ordering — lexical baseline first, hybrid second, orchestration third — with the cost savings as the business case.
Source video ↗A Firm Without Paralegals: Why Single-Agent Research Hits a Ceiling
The single-agent research loop is structurally analogous to a professional services firm staffed entirely with senior experts and no associates — it fails not from lack of intelligence but from lack of division of labor. Hierarchical delegation with per-sub-agent context isolation is the correct abstraction when a task exceeds one context window, and the gain shows up as a measurable narrowing of the gap to human experts.
AngleOrganizational-design framing applied to agent architecture: map orchestrator/decomposer/retrieval-specialist/synthesizer onto firm roles, and enumerate the new failure surface each role introduces.
Source video ↗Why Coding Agents Convinced Us We Were Further Along Than We Were
Coding agents succeeded because code is a self-describing substrate with durable identifiers, explicit paths, and exact-match syntax — affordances that non-code knowledge work simply does not have. Treating coding-agent benchmarks as evidence for enterprise document work is a category error, and architectures built on structured-domain assumptions will fail on meaning-driven corpora.
AngleContrarian correction: enumerate the specific affordances code provides, then show what replaces each one (or doesn't) in legal, medical, and financial document work.
Source video ↗Agent Rollouts Plateau Because You Didn't Change the Workflow
Agent deployments that automate a single step inside an unchanged process are optimizing inside a fixed loop and will hit a ceiling. Because knowledge work runs on co-evolving tool and organization loops, the tool and the workflow must be designed jointly — otherwise the deployment captures a fraction of available value regardless of model quality.
AngleDeployment post-mortem framing: identify the signature of a local-minimum rollout, then describe how to instrument where human roles re-form around the agent as the signal for where the workflow must change next.
Source video ↗Oracle Gap Ablation Harness
beyond-evalsHolding the base model and corpus fixed, retrieval-harness and orchestration upgrades together close at least as much of the residual distance to an oracle-retrieval upper bound as a stronger base model does, and the contributions of the three components (baseline lexical tuning, hybrid/multimodal retrieval, hierarchical orchestration) are individually measurable and non-overlapping.
Proof of conceptBuild a four-condition ablation on a public multi-document QA benchmark: (A) lexical-only single agent, (B) tuned lexical + hybrid retrieval single agent, (C) hybrid + orchestrator with specialized retrieval sub-agents, (D) oracle-retrieval upper bound. Hold the model constant across A–C; run a second sweep with a stronger model to compare against the harness axis.
MeasurementEnd-task accuracy per condition, residual gap closed relative to the oracle condition per component, tokens per resolved query, tool calls per resolved query, and latency.
Source video ↗Retrieval Regression Telemetry
gatehouseTool-call count and token spend per resolved task rise measurably before end-task accuracy falls when retrieval quality degrades, giving a leading indicator that detects retrieval regressions at lower evaluation cost and higher frequency than periodic accuracy benchmarks.
Proof of conceptDeploy a fixed agent task suite over a stable corpus, then progressively degrade the retrieval index (drop fields, disable semantic component, remove reranking) in discrete steps. Log tool calls, tokens, and accuracy per step; compute the lead time between telemetry drift and accuracy drift.
MeasurementCorrelation and lead time between per-task tool-call/token drift and accuracy drop; false-positive rate of a threshold-based alert on synthetic degradation events; comparison against cost of running full accuracy evals at the same detection latency.
Source video ↗Sub-Agent Granularity Sweep
movement-labThere is a measurable sub-agent count and specialization depth at which coordination and synthesis error overtake context-budget savings, and accuracy is non-monotonic in sub-agent count with a peak at low single-digit specialization.
Proof of conceptOn a fixed retrieval corpus and fixed model, sweep orchestrator designs from 1 sub-agent (monolithic) through N specialized retrieval sub-agents, with and without a dedicated synthesis stage. Log decomposition errors, delegation errors, and synthesis errors as separate failure classes.
MeasurementAccuracy, tokens, latency, and dollars per resolved query versus sub-agent count; error-class distribution attributable to decomposition vs. delegation vs. synthesis at each point; location and sharpness of the peak.
Source video ↗Baseline-First Transfer Test
newTuning a lexical (BM25) baseline before layering hybrid retrieval outperforms jumping directly to hybrid search on end-task accuracy and cost, but the size of the advantage shrinks as corpora move from homogeneous short documents toward heterogeneous long-form documents.
Proof of conceptRun the same agent against two retrieval-engineering orderings (baseline-first vs. hybrid-first) across a ladder of corpora varying in document type and scale, including at least one long-form, non-technical corpus. Freeze the agent logic and model across all cells.
MeasurementAccuracy delta and token/tool-call delta per corpus per ordering; whether the claimed ~20% cost saving reproduces; effect-size trend as corpus heterogeneity increases.
Source video ↗