The Last 10% Is Not a Prompting Problem
Agent reliability plateaus because the final 10% is an optimization problem; teams should treat RL/search/self-correction as a separate runtime layer, not a prompt-refinement task.
AngleArchitectural separation of prompt/tool harness from outcome-driven optimization.
Source video ↗Your Benchmark Is Already Decaying
Fixed public benchmarks are unsafe as capability signals at the frontier; sustained measurement requires continuously renewed, non-public expert item pools.
AngleEvaluation as living, versioned infrastructure rather than a static leaderboard.
Source video ↗Hallucination Needs a Router, Not a Retry
Splitting hallucinations into context, prompt, and capability failures enables targeted runtime interventions that outperform generic retries.
AngleError taxonomy to runtime dispatch.
Source video ↗Design for the Next Bottleneck
AI platforms should keep the value layer pluggable because the scarce input shifts from data collection to alignment and deployment.
AnglePlatform architecture and business model alignment.
Source video ↗RL Agentic Loop Harness
movement-labAdding a reward-driven agentic loop to a prompt-only agent raises success on verifiable long-horizon tasks from roughly 90% to above 95%, with measurable cost per additional point.
Proof of conceptImplement a benchmark of verifiable multi-step tasks, run a prompt-only baseline, then add an RL/self-correction loop over outcomes; compare success, attempts, and token cost.
MeasurementTask completion rate, variance across runs, attempts to success, tokens/compute per solved task.
Source video ↗Benchmark Half-Life Tracker
beyond-evalsFrontier benchmark scores saturate with a measurable half-life, and a non-public expert-item pool retains discriminating power longer than public benchmarks.
Proof of conceptTrack scores of successive model versions on a fixed public benchmark and a freshly authored private benchmark; estimate time to saturation and discrimination loss.
MeasurementScore trajectory, saturation half-life, discrimination gap between public and private items, contamination rate.
Source video ↗Hallucination Trigger Router
newClassifying agent errors into missing context, incorrect prompting, or capability gap and routing to retrieval, prompt repair, or capability gating reduces hallucination rate more than generic retry.
Proof of conceptInstrument an agent runtime to log and label errors, build a classifier, and A/B test routed versus generic interventions on a task suite.
MeasurementHallucination rate, resolution rate by trigger type, retry count, latency, cost.
Source video ↗Agent Control Plane for Edge-Case Escalation
gatehouseA control plane exposing vision/goal definition, metric configuration, and structured edge-case escalation reduces human intervention time while maintaining or improving task success.
Proof of conceptBuild a minimal control plane for a multi-step agent workflow; compare manager effort and success against a baseline where humans monitor every step.
MeasurementHuman minutes per completed workflow, escalation rate, edge-case resolution time, task success rate.
Source video ↗