AI Engineer · Published 2026-08-20

Your Agent Evolved. Your Evals Didn't. — Ameya Bhatawdekar, Braintrust

privateReview: unreviewedConfidence: high
agentsevalsarchitectureflywheelObservabilityagent-architecturepass-at-kai-harnessworkflow-graphproduction-flywheelmodel-capability
Open on YouTube ↗

Summary

Overview

  • Speaker: Ameya Bhatawdekar
  • Channel: AI Engineer
  • Main topic: How AI agent architecture evolution requires corresponding evolution in evaluation strategies
  • Purpose: To educate AI engineers and architects on how to design evaluation strategies that scale alongside rapidly evolving AI agent architectures. Ameya Bhatawdekar, Field CTO at Braintrust, explains why building AI applications is easy but making them production-quality and evolving them is extremely difficult. As foundational models and agent architectures evolve rapidly—from simple prompt calls to chains, ReAct loops, and workflow graphs—evaluations (evals) must evolve in parallel. Traditional static evals break down under dynamic, multi-step agent workflows. Bhatawdekar presents a six-generation evolution of AI architecture and argues that the eval set is the durable asset that must follow architectural changes, supported by a production feedback flywheel.

Topic Map

The Challenge of AI Evolution (12:00-14:00)

  • Explanation: Introduction to the common pattern where building an AI demo is easy, but making it production-quality and evolving it over time is hard due to rapidly changing models, user requirements, and data.
  • Key claims:
    • Building AI demos is easy, but production quality is hard.
    • System architecture around AI models evolves dynamically and rapidly.
  • Examples:
    • Rewriting chat bots or agent applications after complex surgery.
  • Terminology:
    • field CTO
    • observability platform
  • Why it matters: Highlights why engineering teams struggle to maintain quality as models release every few months.

Why You Can't Just Swap the Model (14:00-24:00)

  • Explanation: Detailed breakdown of why dropping in a new model breaks existing agent systems built on older assumptions, limitations, and tool-calling capabilities.
  • Key claims:
    • Current agents are designed around model constraints that no longer exist.
    • Unlocking new model capabilities requires re-architecting and updating evals.
  • Examples:
    • New models unlocking tool-calling and long-context reasoning.
  • Terminology:
    • re-architecting
    • evals
    • surface area
  • Why it matters: Explains the technical debt and architectural friction when upgrading underlying LLMs.

Six Generations of AI Architecture (24:00-35:00)

  • Explanation: A historical and evolutionary walkthrough of AI architectures from Gen 1 (single prompt) to Gen 6 (the AI harness with memory, skills, tools, and sandboxes).
  • Key claims:
    • Gen 1: Single prompt, single model call, answer.
    • Gen 2: The chain (fixed pipeline: parse, retrieve, reason, answer).
    • Gen 3: The ReAct loop (model controls path through tools).
    • Gen 4: Workflow graphs (explicit graph/state machine for high-stakes systems).
    • Gen 5: Modern loop returns with stronger models.
    • Gen 6: The AI harness wrapping memory, tools, skills, sandboxes, and durable state.
  • Examples:
    • Sentinel: an SRE incident response agent watching microservices.
  • Terminology:
    • ReAct loop
    • workflow graph
    • MCP
    • state machine
    • eval unit
  • Why it matters: Provides a structural taxonomy of how AI agent systems have matured and where failure modes shift.

The AI Flywheel and Evaluation Discipline (35:00-40:00)

  • Explanation: How to build a robust flywheel connecting production observation, development, eval updates, and deployment to handle architectural shifts.
  • Key claims:
    • Architecture follows model capability; evals follow architecture.
    • The implementation changes, but the eval set is the durable asset.
    • Turn production behavior into data, turn data into evals, use evals to ship.
  • Examples:
    • Braintrust Topics cluster analysis for discovering new failure modes from production data.
  • Terminology:
    • AI flywheel
    • pass@k
    • pass_wedge_k
    • cluster analysis
    • guardrails
  • Why it matters: Gives teams a concrete methodology to prevent eval stagnation and maintain system reliability in production.

Key Points

Architecture follows model capability; evals follow architecture.

  • Explanation: As foundational models release updates every few months with new capabilities like long context or robust tool use, system architectures must change, and evaluation metrics must adapt to the new architecture.
  • Evidence: Historical evolution from single prompt calls to complex workflow graphs and AI harnesses.
  • Practical implication: Teams cannot rely on static evaluation benchmarks when upgrading agent architectures.

The eval set is the durable asset.

  • Explanation: While code implementations and model versions constantly change, a comprehensive and growing suite of evaluation test cases remains the long-term asset ensuring quality.
  • Evidence: Discussion on how teams must update evals whenever they re-architect or swap models.
  • Practical implication: Invest heavily in curating and expanding eval sets derived from real production failures.

Shift from single-answer evals to distribution-based metrics.

  • Explanation: In multi-step agent loops with high variance, evaluating a single run is insufficient. Systems require statistical metrics across multiple runs.
  • Evidence: Introduction of metrics like pass@k and pass_wedge_k.
  • Practical implication: Run evals multiple times to measure true system reliability and trajectory coverage.

Frameworks, Models & Processes

The Six Generations of AI Architecture

  • How it works: Categorizes AI application patterns from simple prompts to complex stateful agent harnesses with memory and sandboxes.
  • Components:
    • Gen 1: Prompt
    • Gen 2: Chain
    • Gen 3: ReAct Loop
    • Gen 4: Workflow Graph
    • Gen 5: Modern Loop
    • Gen 6: The AI Harness
  • When to use: To assess the maturity and architectural complexity of an AI agent system.

The AI Flywheel

  • How it works: A continuous operational loop for improving AI systems by turning production telemetry into data, evals, and improvements.
  • Components:
    • Observe (log input, output, trace, failure)
    • Develop (hill-climbing, iterate, repeat)
    • Eval (turn failure patterns into new eval cases)
    • Deploy (ship quickly & confidently)
  • When to use: When managing production AI agents that require continuous adaptation and reliability.

Examples & Case Studies

Building 'Sentinel', an SRE incident response agent that watches microservices, reads metrics and logs, and takes write actions like rollbacks or page escalations.

  • Illustrates: The progression from simple prompt answering to complex agentic workflows with read/write tool access.
  • Lesson: High-stakes agent systems require sophisticated workflow graphs and rigorous node-level evaluations.

Using Braintrust Topics to run cluster analysis on production data.

  • Illustrates: How production telemetry surfaces new failure categories that engineers did not anticipate.
  • Lesson: Automated topic clustering helps teams expand their eval sets with real-world failure modes.

Actionable Takeaways

  • Immediate:
    • Audit your current agent architecture against the six generations.
    • Ensure your evaluation metrics match your system's architectural complexity (moving beyond simple answer checks to trajectory and node-level checks).
  • Strategic:
    • Treat your eval set as the primary durable asset of your AI product.
    • Build a production observation flywheel to automatically capture failure modes and feed them back into your eval suite.
  • Questions to investigate:
    • Are our current evals static, or do they evolve alongside our model and architectural updates?
    • How are we measuring variance and trajectory success across multi-step agent loops?

Claims Worth Verifying

  • Every few months a new model breakthrough resets what is possible in AI engineering. (industry trend)
  • Statistical metrics like pass@k provide a better measure of agent reliability than single-run evaluations. (technical best practice)

Notable Quotes

"Your Agent Evolved. Your Evals Didn't." "The rate at which we change our agent architecture has been astonishing. Every few months a new model breakthrough resets what is possible." "Architecture follows model capability. Evals follow architecture. The implementation keeps changing. The eval set is the durable asset."

Compressed Summary

  • Rapid model advancements force continuous re-architecting of AI agents.
  • Static evaluations fail as architectures evolve from simple prompts to stateful graphs and harnesses.
  • The eval set, not the codebase, is the durable asset of an AI system.
  • A production flywheel (Observe -> Develop -> Eval -> Deploy) is essential for capturing new failure modes.
  • Keywords: agents, evals, architecture, flywheel, observability
  • Core insight: As AI agent architectures rapidly evolve to leverage new model capabilities, static evaluation strategies fail, making a dynamic, production-backed eval flywheel the durable foundation for reliable systems.

Core insights

7
Mental Modelmedium noveltymoderate evidence

Architecture is a dependent variable of model capability, and evals are a dependent variable of architecture: when a new model unlocks previously impossible capabilities (tool-calling, long context), the existing architecture is stale, and the eval suite built for the old architecture becomes invalid. The corollary is that evals must be re-derived, not just re-run, on every architectural change.

Why it matters

It reframes eval maintenance as a consequence of architectural change rather than an independent QA chore, and predicts that teams who treat evals as static artifacts will silently measure an architecture they no longer run.

Generalization

Applies to any system whose behavior depends on a fast-moving external component: the test suite encodes assumptions about that component, so capability shifts invalidate measurements before they invalidate code.

Architecture follows model capability; evals follow architecture.
Open source video
The eval set is the durable asset that must follow architectural changes, supported by a production feedback flywheel.
Open source video
Teams cannot rely on static evaluation benchmarks when upgrading agent architectures.
Open source video
Practicemedium noveltymoderate evidence

Beneath constant model and architecture churn, the durable engineering asset is the curated eval set — a growing corpus of test cases derived from real production failures — while implementations, prompts, and scaffolding are treated as disposable. Investment therefore shifts from code longevity to test-case longevity.

Why it matters

It gives a concrete answer to 'what should we invest in when everything around it gets rewritten every few months,' and implies eval cases should live in a system that outlives the agent code and framework of the day.

Generalization

For any rapidly re-implemented system, port the specification corpus across rewrites rather than the implementation; the specification is the compounding asset.

The implementation changes, but the eval set is the durable asset.
Open source video
Invest heavily in curating and expanding eval sets derived from real production failures.
Open source video
Mechanismhigh noveltymoderate evidence

In multi-step agent loops with high run-to-run variance, single-run pass/fail is an insufficient (and misleading) signal; evaluation must move to distributional metrics across repeated runs, named here as pass@k and pass_wedge_k, which measure reliability and trajectory coverage rather than one sampled answer.

Why it matters

It changes eval harness design from one-shot assertion to repeated sampling with statistical aggregation, with direct cost/latency consequences, and prevents teams from shipping on luck from a single favorable rollout.

Generalization

For any stochastic multi-step pipeline, a single green run is a sample of size one; reliability claims require repeated trials and a distributional statistic.

In multi-step agent loops with high variance, evaluating a single run is insufficient. Systems require statistical metrics across multiple runs.
Open source video
Run evals multiple times to measure true system reliability and trajectory coverage.
Open source video
Predictionhigh noveltymoderate evidence

Agent architecture does not progress monotonically toward more explicit structure: Gen 4 workflow graphs (explicit state machines for high-stakes control) were followed by Gen 5, in which the model-driven modern loop returns because stronger models can carry control that previously had to be externalized. Structure is added when the model is weak and removed when it strengthens.

Why it matters

It implies that today's orchestration scaffolding is partly a compensation for model weakness, so teams should expect to delete graph/state-machine layers as models improve — and should design those layers as removable.

Generalization

Orchestration and guardrail scaffolding tracks the capability gap of the underlying model; as the gap closes, externally imposed control becomes overhead rather than safety.

Gen 4: Workflow graphs (explicit graph/state machine for high-stakes systems).
Open source video
Gen 5: Modern loop returns with stronger models.
Open source video
Current agents are designed around model constraints that no longer exist.
Open source video
Architecturemedium noveltymoderate evidence

Evaluation granularity must match architecture granularity: explicit workflow graphs used for high-stakes agents require node-level evaluations (per-step assertions), not only end-to-end pass/fail, because a graph's failure is localized to a node while an end-to-end signal only reports that the whole run failed.

Why it matters

It dictates where instrumentation and eval hooks belong in a graph runtime (at node boundaries) and determines whether a failure can be attributed to a specific step or only observed as an aggregate error.

Generalization

Evaluation units should be coextensive with the system's composable units; coarser eval units lose failure localization and slow debugging.

High-stakes agent systems require sophisticated workflow graphs and rigorous node-level evaluations.
Open source video
eval unit
Open source video
Mechanismmedium noveltyweak evidence

Production telemetry is the source of new eval cases: cluster analysis over production traces surfaces failure categories engineers did not anticipate, which are then promoted into the eval set — closing the loop so the eval suite grows from observed reality rather than from imagined scenarios.

Why it matters

It makes the eval set self-expanding and ties observability infrastructure directly to test-case generation, giving a repeatable procedure for discovering unknown-unknown failure modes instead of waiting for user complaints.

Generalization

Treat production traffic as a discovery channel for test cases: unsupervised clustering on traces is a mechanism for growing coverage where hand-written cases miss.

Using Braintrust Topics to run cluster analysis on production data.
Open source video
Automated topic clustering helps teams expand their eval sets with real-world failure modes.
Open source video
Turn production behavior into data, turn data into evals, use evals to ship.
Open source video
Architecturemedium noveltymoderate evidence

The Gen 6 'AI harness' — memory, tools, skills, sandboxes, and durable state wrapped around the model — relocates the system boundary outward from the model call, which means the unit under test is the harness-plus-state, not the model response.

Why it matters

It redefines what an eval must instantiate: a harness with persistent state and sandboxed tool access, rather than a stateless prompt-response call, which changes fixture design, isolation, and reproducibility of evaluations.

Generalization

As the system boundary expands to include stateful infrastructure, the evaluation fixture must expand with it or the measurement no longer covers the deployed system.

Gen 6: The AI harness wrapping memory, tools, skills, sandboxes, and durable state.
Open source video
The eval unit
Open source video

Deep dives

5

Calibrating k for distributional agent evals

Research question

What value of k, and what aggregation, actually distinguish a reliable agent from a lucky one under realistic token and latency budgets?

Why

Single-run pass/fail overfits to one favorable rollout in stochastic multi-step loops; teams need a cheapest-k rule to make ship decisions without multiplying eval cost and latency without bound.

In multi-step agent loops with high variance, evaluating a single run is insufficient. Systems require statistical metrics across multiple runs.
Open source video
Run evals multiple times to measure true system reliability and trajectory coverage.
Open source video
Source video

Portable eval corpus across architecture generations

Research question

How are eval cases ported and re-validated when an agent moves from one architecture generation to the next, given that the same case may be valid for a chain but ill-defined for a harness with durable state?

Why

The eval set is claimed as the durable asset carried across rewrites, but without declared architectural assumptions and incompatibility checks, ported cases silently measure the wrong system.

The eval set is the durable asset that must follow architectural changes
Open source video
The implementation changes, but the eval set is the durable asset.
Open source video
Source video

Removable orchestration as model capability changes

Research question

How should orchestration layers be designed so that graph/state-machine control can be withdrawn when model capability improves without losing safety or eval coverage?

Why

Explicit graph control encodes assumptions about model weakness; if models absorb path control, today's orchestration becomes migration tax and rigidity rather than safety.

Gen 4: Workflow graphs (explicit graph/state machine for high-stakes systems).
Open source video
Gen 5: Modern loop returns with stronger models.
Open source video
Current agents are designed around model constraints that no longer exist.
Open source video
Source video

Node-level eval granularity in workflow graphs

Research question

Which failures should be attributed to a node and which to the model, and can node-level evals be written without encoding the very model constraints the architecture is trying to escape?

Why

A graph's failure is localized to a node, but end-to-end pass/fail only reports that the whole run failed; the runtime must expose per-node instrumentation to make failures attributable and debuggable.

High-stakes agent systems require sophisticated workflow graphs and rigorous node-level evaluations.
Open source video
eval unit
Open source video
Source video

Novelty yield of production trace clustering

Research question

Does production-traffic clustering reliably surface novel failure modes, or does it mostly re-surface known categories while remaining blind to rare, high-severity failures?

Why

If clustering only restates known failures, the production feedback flywheel grows the eval set without increasing coverage of unanticipated failure modes, wasting curation effort.

Automated topic clustering helps teams expand their eval sets with real-world failure modes.
Open source video
Turn production behavior into data, turn data into evals, use evals to ship.
Open source video
Source video

Article ideas

4

Your Eval Suite Is a Fossil: Model Upgrades Invalidate Tests Before Code

Static eval benchmarks are architecture-specific; when a model unlocks new capabilities, the existing eval suite measures an abandoned architecture, so teams must re-derive evals, not just re-run them.

Angle

Show that eval invalidation is a leading indicator of architectural obsolescence, then prescribe architecture-aware eval redeployment as the first migration step.

Source video

The Eval Set Is the Only Code You Should Never Rewrite

In a world where prompts, scaffolding, and frameworks are disposable, the curated eval corpus derived from production failures is the compounding engineering asset; invest in portability and case quality, not implementation longevity.

Angle

Argument for treating eval cases as long-lived specifications that outlive any agent framework, with concrete implications for storage, versioning, and migration budgets.

Source video

Your Orchestration Layer Is a Loan Against Model Weakness

Workflow graphs and state machines externalize control because models cannot yet be trusted with the path; as models strengthen, that scaffolding becomes migration tax, so design orchestration to be removable and capability-contingent.

Angle

Reframe orchestration layers as capability-contingent scaffolding, with a checklist for making graph/state-machine control deletable without losing eval coverage.

Source video

One Green Run Is a Sample of Size One: Let pass@k Decide Your Release

Multi-step agents are stochastic pipelines, so single-run pass/fail overfits to luck; release decisions require distributional metrics across repeated runs and explicit cost budgeting.

Angle

Translate variance in agent loops into eval-harness design requirements: repeated sampling, aggregation, and tiered case sets under token and latency budgets.

Source video

Project ideas

4

K-Calibrator

beyond-evals

For a representative multi-step agent task, a minimal k exists beyond which ship/no-ship decisions match a high-k ground truth with high agreement, and this k is small enough to keep eval cost practical.

Proof of concept

Run a ReAct-loop agent on a fixed case set with k=50, compute pass@k and pass_wedge_k stability curves, and compare decisions from k=1..20 against the k=50 reference.

Measurement

Agreement rate between low-k decisions and k=50 decisions, variance reduction per added run, and token/latency cost per reliable ship decision.

Source video

EvalPort

beyond-evals

Eval cases annotated with declared architectural assumptions (generation, state requirements, tool surface) can be automatically classified as portable, needing rewrite, or invalid when migrating across architecture generations, with low false-invalidation.

Proof of concept

Define an eval-case schema with architectural assumptions, build a compatibility checker, and migrate a seed corpus from a Gen 3 loop to a Gen 6 harness while manually auditing the labels.

Measurement

Precision and recall of portability labels against manual audit, number of cases surviving unchanged, and engineering time saved versus a from-scratch rewrite.

Source video

Scaffold Debt Meter

movement-lab

As underlying model capability increases, explicit graph/state-machine orchestration can be removed for a class of high-stakes tasks without reducing task success or safety, because the model absorbs path control.

Proof of concept

Implement the same incident-response agent as a Gen 4 workflow graph and a Gen 5 model-driven loop, run both on a fixed model version and on a stronger model, and compare outcomes.

Measurement

Delta in task success rate, write-action authorization violations, latency, and engineering effort to remove the orchestration layer.

Source video

NodeScope

beyond-evals

Node-level assertions in a workflow graph localize failures to graph nodes with higher precision than end-to-end evals alone, without materially increasing false-positive brittle assertions when assertions are property-based.

Proof of concept

Instrument a graph agent with per-node property assertions, inject faults at selected nodes, and compare failure attribution against end-to-end pass/fail alone.

Measurement

Localization precision/recall, false-positive assertion rate, and debugging time reduction in a controlled fault-injection exercise.

Source video

Architectural implications

5

Eval artifacts are decoupled from the agent implementation they test.

Before

Eval logic lives inside or alongside a specific agent implementation, prompt, or framework version, and dies with it.

After

Eval cases are stored as framework-independent assets (inputs, expected properties, trajectory assertions) that are ported across re-architectures and model swaps.

Consequence

Architecture migrations become 're-run the eval corpus against the new harness' rather than 'rewrite the tests,' making re-architecting economically viable and accelerating model adoption.

Source video

The eval runner becomes a statistical sampling system rather than a pass/fail test runner.

Before

A single deterministic run per case yields a binary pass/fail in a synchronous test suite.

After

Each case is executed k times with aggregated distributional metrics (pass@k, pass_wedge_k), requiring parallel execution, variance tracking, and cost budgeting per eval cycle.

Consequence

Eval latency and token cost multiply by k, pushing eval suites toward scheduled sampling, tiered case sets, and dedicated infrastructure rather than CI-time execution.

Source video

Control placement shifts between explicit graph and model-driven loop as model capability changes.

Before

High-stakes workflows externalize control into explicit graphs and state machines because the model cannot be trusted with the path.

After

Stronger models absorb path control back into the loop, and the explicit graph degrades into optional guardrail scaffolding.

Consequence

Orchestration layers must be designed as removable and capability-contingent; hard-coded control flow becomes the migration tax when models improve.

Source video

Failure attribution requires eval hooks at node boundaries inside the graph execution engine.

Before

Evaluation observes only the final output of the agent run.

After

Each node emits its own inputs, outputs, and assertions, so a failure is localized to a step rather than the whole trajectory.

Consequence

The graph/state-machine runtime must expose per-node instrumentation as a first-class primitive, coupling the execution engine to the observability and eval layers.

Source video

The agentic control surface widens with each generation: from a model call, to tool calls, to write actions with side effects.

Before

Agents read data and return an answer; scope of potential harm is limited to incorrect output.

After

Agents take write actions such as rollbacks or page escalations, making evaluation of action correctness and authorization part of the reliability problem.

Consequence

Evals must assert on actions taken in a sandbox, not just on text produced, which requires sandboxed execution environments and rollback-safe test harnesses.

Source video

Tradeoffs and failure modes

5

Explicit workflow graphs vs model-driven loops

Benefit

Explicit graph/state-machine control gives deterministic path guarantees for high-stakes systems and makes per-node evaluation possible.

Cost or risk

The same control flow encodes assumptions about model weakness; when models improve, the graph becomes rigidity and re-architecture burden rather than safety.

Gen 4: Workflow graphs (explicit graph/state machine for high-stakes systems).
Open source video
Source video

Scaffolding around current model limitations

Benefit

Designing agents around known model constraints (context, tool-calling reliability) delivers production behavior today.

Cost or risk

Those constraints disappear on the next model release, leaving the agent built on assumptions that no longer hold and forcing re-architecting plus eval updates.

Current agents are designed around model constraints that no longer exist.
Open source video
Source video

Distributional evaluation (pass@k, pass_wedge_k)

Benefit

Measures true reliability and trajectory coverage under high variance instead of overfitting to one lucky run.

Cost or risk

Repeated execution multiplies eval cost and latency, making exhaustive case coverage and frequent full eval runs impractical.

Run evals multiple times to measure true system reliability and trajectory coverage.
Open source video
Source video

Promoting production failures into evals via cluster analysis

Benefit

Surfaces unanticipated real-world failure categories that hand-written eval sets omit.

Cost or risk

Eval sets grow continuously from telemetry, creating curation, deduplication, and prioritization burden, and embedding production-specific noise into the suite.

Turn production behavior into data, turn data into evals, use evals to ship.
Open source video
Source video

Rapid shipping on top of a changing model and architecture stack

Benefit

The flywheel (observe, develop, eval, deploy) lets teams ship quickly and confidently despite constant underlying change.

Cost or risk

Shipping confidence depends entirely on the eval set having already caught up to the current architecture; stale evals convert fast shipping into unmeasured regression risk.

Teams cannot rely on static evaluation benchmarks when upgrading agent architectures.
Open source video
Source video

Open questions

5

How are eval cases ported and re-validated when an agent moves from one architecture generation to the next, given that the same case may be valid for a chain but ill-defined for a harness with durable state?

Why unresolved

The summary asserts the eval set is the durable asset that must follow architectural changes but does not describe any migration, versioning, or invalidation procedure for eval cases across generations.

Research direction

Prototype an eval-case schema with declared architectural assumptions and a compatibility check that flags cases invalidated by an architecture or model upgrade.

Source video

What value of k, and what aggregation, actually distinguish a reliable agent from a lucky one under realistic token and latency budgets?

Why unresolved

pass@k and pass_wedge_k are named as the metrics but the summary provides no definition, threshold, cost model, or empirical comparison of k values.

Research direction

Empirically measure run-to-run variance on a multi-step agent workload and plot reliability estimate stability against number of repetitions to find the cheapest k that separates passing and failing systems.

Source video

Which failures should be attributed to a node and which to the model, and can node-level evals be written without encoding the very model constraints the architecture is trying to escape?

Why unresolved

Node-level evaluation is recommended for high-stakes graphs, but the summary does not explain how node-level assertions avoid becoming brittle coupling to current model behavior.

Research direction

Compare node-level vs end-to-end failure detection rates and false-positive rates on a workflow-graph agent to quantify localization benefit versus brittleness.

Source video

Does production-traffic clustering reliably surface novel failure modes, or does it mostly re-surface known categories while remaining blind to rare, high-severity failures?

Why unresolved

Cluster analysis on production data is presented as the discovery mechanism for new failure categories, but only a single vendor example is cited with no measurement of novelty yield or false discovery.

Research direction

Run controlled experiments injecting known novel failure modes into production-like traces and measure whether unsupervised clustering recovers them within a useful candidate set size.

Source video

As control migrates back from explicit graphs to model-driven loops, what replaces the determinism that high-stakes systems relied on?

Why unresolved

The summary notes the modern loop returns with stronger models but does not state how guarantees, authorization, or blast-radius limits for write actions are preserved.

Research direction

Investigate whether guardrails and sandboxing can substitute for graph-level control on write-capable agents without degrading task success rates.

Source video

Key claims

8
causalVerification needed

Agent architecture must change when foundational models release new capabilities, and evaluation metrics must then change to match the new architecture.

Evidence

As foundational models release updates every few months with new capabilities like long context or robust tool use, system architectures must change, and evaluation metrics must adapt to the new architecture.

Question

Do teams that upgrade models without re-architecting and re-tooling evals show measurable quality regressions versus teams that do?

Source video
opinionVerification needed

While implementations and model versions change, the curated evaluation test-case suite remains the durable long-term quality asset.

Evidence

The implementation changes, but the eval set is the durable asset.

Question

Across real re-architecture events, what fraction of eval cases survive unchanged, and does the surviving set actually predict production quality of the new architecture?

Source video
causalVerification needed

Evaluating a single run is insufficient for high-variance multi-step agent loops; statistical metrics across multiple runs are required.

Evidence

In multi-step agent loops with high variance, evaluating a single run is insufficient. Systems require statistical metrics across multiple runs.

Question

How large is run-to-run variance on representative multi-step agent tasks, and at what k does a single-run evaluation start producing wrong ship decisions?

Source video
factualVerification needed

The eval metrics named pass@k and pass_wedge_k are used to measure true system reliability and trajectory coverage.

Evidence

Run evals multiple times to measure true system reliability and trajectory coverage.

Question

What is the precise definition of pass_wedge_k, and how does it differ from pass@k in what it detects about agent trajectories?

Source video
causalVerification needed

Automated topic clustering over production data expands eval sets with real-world failure modes that were not anticipated.

Evidence

Automated topic clustering helps teams expand their eval sets with real-world failure modes.

Question

What proportion of clusters from production traces correspond to genuinely new failure modes versus restatements of known ones?

Source video
factualVerification needed

Different agent architectures have different evaluation units and failure modes, from single prompt (Gen 1) through chain, ReAct loop, workflow graph, and modern loop, to an AI harness wrapping memory, tools, skills, sandboxes, and durable state (Gen 6).

Evidence

Gen 6: The AI harness wrapping memory, tools, skills, sandboxes, and durable state.

Question

Is the six-generation progression a recognized taxonomy, and do real production systems cluster into these categories or occupy hybrids?

Source video
opinionVerification not requested

High-stakes agent systems require workflow graphs with rigorous node-level evaluations rather than end-to-end evaluation alone.

Evidence

High-stakes agent systems require sophisticated workflow graphs and rigorous node-level evaluations.

Source video
causalVerification needed

The production flywheel converts observed behavior into data, data into evals, and evals into confident deployments.

Evidence

Turn production behavior into data, turn data into evals, use evals to ship.

Question

Does closing this loop measurably reduce production incident rates or ship-decision error rates compared with periodic manual eval curation?

Source video

Connections

5