Sequoia Capital · Published 2026-08-13

Continual Learning: How AI Agents Get Better With Every Use | Arjun Karanam, Trajectory

privateReview: unreviewedConfidence: high
continual learningAI agentstraceabilityReinforcement learningHarnesscontinual-learningagent-tracesrolloutable-evalsharness-engineeringopen-weightsrl-credit-assignmentagent-observability
Open on YouTube ↗

Summary

Overview

  • Speaker: Arjun Karanam
  • Channel: Sequoia Capital
  • Main topic: Continual learning platforms for AI agents
  • Purpose: To introduce Trajectory's platform for continual learning and explain how companies can build AI agents that improve with use over time. Arjun Karanam, co-founder of Trajectory, discusses the concept of continual learning for AI agents. He explores why current frontier models optimize purely for IQ without experience, the economic value of experience, and how Trajectory is building a platform that turns every interaction into continually improving systems through traceability, evaluations, model behavior specs, and iterative RL training.

Topic Map

The Experience Gap in Frontier AI Models

  • Explanation: Frontier AI models are optimized for high IQ (PhD level) with zero experience, resembling a brilliant new hire on day one. However, the economy values experience deeply, creating an 'Experience Gap' that companies must close.
  • Key claims:
    • Frontier labs optimize solely for IQ.
    • Today's agents are slow, expensive, and error-prone.
    • The economy values 30-year domain experience much more than raw intelligence alone.
  • Examples:
    • Terence Tao arriving day one at an accounting firm vs. a 30-year expert accountant.
  • Terminology:
    • frontier models
    • IQ axis
    • experience gap
    • orthogonal axis of experience
  • Why it matters: Models must gain real-world experience through continual learning rather than remaining static and expensive.

Unlocking Real-World Interaction Data

  • Explanation: Billions of tokens are generated daily by AI agents in production, but most of this real-world signal goes to waste. Trajectory aims to treat real-world interactions as the primary signal for continual learning.
  • Key claims:
    • 100 billion tokens per day are generated by production AI.
    • Most interaction signals currently go to waste.
    • Learning from real-world interactions mimics how humans get smarter.
  • Examples:
    • Weekly usage of models across OpenRouter growing rapidly.
  • Terminology:
    • tokens/day
    • product telemetry
    • real-world signal
  • Why it matters: Real-world interactions provide the training data needed to make models efficient, reliable, and continuously improving.

The Trajectory Platform Solution

  • Explanation: Trajectory's platform bridges the gap through four main pillars: traceability, model behavior specs, model optimization, and harness co-evolution.
  • Key claims:
    • Traceability requires capturing full traces plus product telemetry including subagents.
    • Evaluations must represent the full distribution of tasks and outcomes.
    • Models should be better, cheaper, and faster.
    • Harnesses must co-evolve with models and contain domain knowledge.
  • Examples:
    • Harvey lab benchmark integration with 1,760 tasks.
  • Terminology:
    • traceability
    • SDPO
    • model behavior spec
    • harness
    • rolloutable evals
  • Why it matters: Providing a structured product layer allows companies to own their own experience layer and continuously improve agents.

Key Points

The Economy Values Experience

  • Explanation: Companies pay most for personnel with decades of domain experience, not just raw intelligence.
  • Evidence: Comparison between Terence Tao on day one at an accounting firm vs. an experienced accountant.
  • Practical implication: AI agents must transition from static tools to continuous learners that compound in value with use.

Capturing Full Traces and Subagents

  • Explanation: Traceability must capture entire execution trees including subagents, rather than throwing away intermediate tool calls.
  • Evidence: Coding agents frequently break after a few commits if subagent failures are discarded.
  • Practical implication: Build error-correction, edits, and retries natively into product interfaces to capture clean training signals.

Making Evals Rolloutable

  • Explanation: Evaluations must reflect real production traffic and allow full replay of user starting states.
  • Evidence: Static benchmarks do not represent actual user workflows and requests.
  • Practical implication: Sample real distribution of tasks and ensure evaluation harness matches production.

Frameworks, Models & Processes

Trajectory Continual Learning Framework

  • How it works: Captures user interactions, defines model behavior specs, optimizes models and harnesses jointly, and deploys updates instantly.
  • Components:
    • Traceability
    • Model Behavior 'Spec'
    • Evaluations
    • Models
    • Harness
  • When to use: When deploying AI agents in production that need to improve autonomously over time.

Examples & Case Studies

Harvey integrated with Trajectory to manage benchmark testing across 1,760 tasks.

  • Illustrates: Easy import, model training, evaluation comparison, and one-click deployment of fine-tuned models.
  • Lesson: Continual learning infrastructure can reduce post-training and deployment workflows to minutes.

Actionable Takeaways

  • Immediate:
    • Capture full execution traces including subagents in your AI product.
    • Design products with native affordances for edits, undos, and retries to generate clean supervision labels.
    • Build evaluation harnesses that match production traffic distributions.
  • Strategic:
    • Every company should own its own experience layer that never stops evolving.
    • Move away from hard-coded workflows toward light, open, and informative harnesses.
    • Transition from closed hosted models to training on open-weight models to unlock autonomous improvement.
  • Questions to investigate:
    • How do we handle differential privacy and customer data isolation during continual learning?
    • What role does episodic memory play in updating agent behavior after corrections?
    • How do we weight human feedback versus automated task completion signals in RL credit assignment?

Claims Worth Verifying

  • Every company should own its own experience layer that never stops evolving. (strategic vision)
  • Training models on domain expertise matches large general models at a fraction of the cost. (economic claim)

Notable Quotes

"Frontier models feel like a brilliant new hire, on their day one of the job." (at 0:56) "Trajectory turns every interaction into continually improving systems." (at 5:05) "You can only improve what you can see." (at 10:11) "Every company should own its own experience layer that never stops evolving." (at 23:36)

Compressed Summary

  • Frontier AI models suffer from an Experience Gap, optimizing for IQ without real-world experience.
  • Trajectory provides infrastructure for continual learning via traceability, evals, model specs, and harnesses.
  • Capturing full traces and subagent interactions is essential for extracting learning signals.
  • Running on open weights allows companies to continuously train and deploy improved agent models.
  • Keywords: continual learning, ai agents, traceability, reinforcement learning, harness
  • Core insight: AI agents must transition from static, pre-trained tools into systems that continuously learn and compound in value from every real-world user interaction.

Core insights

7
Mechanismhigh noveltymoderate evidence

Trace fidelity is a training-signal constraint, not just an observability concern: learning signal is destroyed if intermediate tool calls and subagent failures are discarded rather than captured as part of the full execution tree.

Why it matters

It defines the minimum data contract for any continual-improvement loop: agent success/failure is often caused at the subagent/step level, so outcome-only logging produces data that cannot be credited back to the responsible action.

Generalization

For any agent that learns from its own usage, the capture schema (execution tree + product telemetry) constrains what behaviors can ever be improved; incomplete logging silently caps the ceiling of the learning system.

Traceability requires capturing full traces plus product telemetry including subagents.
Open source video
Coding agents frequently break after a few commits if subagent failures are discarded.
Open source video
Architecturehigh noveltymoderate evidence

Product interface affordances (edits, undos, retries, error correction) are treated as native label-generation mechanisms for RL, rather than as mere UX conveniences.

Why it matters

It relocates data-labeling from a separate offline annotation effort into the product interaction loop, where user corrections become clean supervision signals at near-zero marginal cost.

Generalization

Interaction primitives that expose user intent (revert, edit, retry, reject) can be designed deliberately as supervision APIs for any agent product that wants to keep training on production usage.

Design products with native affordances for edits, undos, and retries to generate clean supervision labels.
Open source video
Build error-correction, edits, and retries natively into product interfaces to capture clean training signals.
Open source video
Practicemedium noveltymoderate evidence

Evaluations must be 'rolloutable': sampled from the real distribution of production tasks and able to replay actual user starting states, rather than being static benchmarks.

Why it matters

Static benchmarks do not predict production behavior, so a training-loop that gates deployment on them can improve the benchmark while degrading real workflows; replayability makes eval an executable, environment-matched artifact.

Generalization

Any iterative RL or fine-tuning loop needs an eval harness that mirrors production state and task distribution, otherwise model updates and harness updates cannot be compared on the axis that matters.

Evaluations must represent the full distribution of tasks and outcomes.
Open source video
Static benchmarks do not represent actual user workflows and requests.
Open source video
Sample real distribution of tasks and ensure evaluation harness matches production.
Open source video
Mental Modelmedium noveltymoderate evidence

The harness is a trained artifact that must co-evolve with the model and carry domain knowledge, with the design direction moving away from hard-coded workflows toward light, open, informative harnesses.

Why it matters

It breaks the assumption that the model is the only thing that improves; scaffolding, prompts, tool wiring, and control flow are part of the same optimization loop, which changes what belongs in code versus in weights.

Generalization

When capability can be moved between weights and orchestration code, the boundary becomes an optimization decision (cost, latency, retrainability) rather than a fixed architectural boundary.

Harnesses must co-evolve with models and contain domain knowledge.
Open source video
Move away from hard-coded workflows toward light, open, and informative harnesses.
Open source video
Tradeoffmedium noveltyweak evidence

Open-weight models are positioned as a structural prerequisite for continual learning: closed hosted models block the autonomous train-and-deploy loop.

Why it matters

It makes model hosting choice a first-class architectural decision tied to whether a company can ever own its improvement loop, not just a cost/quality comparison.

Generalization

Any system whose value depends on updating its own weights must treat model licensing/serving as part of the control plane, because the improvement loop's existence depends on it.

Transition from closed hosted models to training on open-weight models to unlock autonomous improvement.
Open source video
Every company should own its own experience layer that never stops evolving.
Open source video
Predictionmedium noveltyweak evidence

Production agents emit enormous interaction volume (claimed ~100B tokens/day) that is mostly discarded, making an owned, continuously-updated 'experience layer' the compounding asset rather than the base model itself.

Why it matters

It reframes competitive advantage as the retained, curated interaction data plus the pipeline that converts it into weights and harness updates, which is not obtainable off the shelf.

Generalization

Wherever an agent runs in production, the byproduct telemetry is a strategic asset whose value is realized only if it is captured, labeled, and fed back into training on a short cycle.

100 billion tokens per day are generated by production AI.
Open source video
Most interaction signals currently go to waste.
Open source video
Every company should own its own experience layer that never stops evolving.
Open source video
Empirical Resultmedium noveltymoderate evidence

A packaged continual-learning pipeline can collapse post-training and deployment workflows to minutes, evidenced by importing a 1,760-task benchmark, training, comparing evals, and one-click deploying fine-tuned models.

Why it matters

If the train-eval-deploy cycle is minutes rather than weeks, agent teams can iterate per-customer or per-domain on a cadence that changes release and reliability practices.

Generalization

Loop latency (not just model quality) determines whether continual learning is practically usable; infrastructure that shortens the cycle is what makes the paradigm viable.

Harvey lab benchmark integration with 1,760 tasks.
Open source video
Continual learning infrastructure can reduce post-training and deployment workflows to minutes.
Open source video

Deep dives

5

Trace granularity as the hard ceiling on continual learning

Research question

How does the granularity of captured execution traces (outcome-only vs. tool-call level vs. full subagent tree) change the achievable improvement rate and the regression rate of a continually trained agent?

Why

If the learning loop's ceiling is set by the capture schema rather than by the optimizer, then storage/retention/schema design sits on the critical path of model quality; teams that log only final outcomes are silently capped and will misattribute failure to model capacity. Establishing the granularity/regression curve converts an asserted failure mode into an engineering budget line.

Traceability requires capturing full traces plus product telemetry including subagents.
Open source video
You can only improve what you can see.
Open source video
Source video

Rolloutable evals as production-state replay

Research question

Does gating model and harness updates on distribution-sampled, state-replayable evals predict production behavior better than static benchmark suites, and by how much does the gap widen as the agent accumulates domain experience?

Why

A training loop gated on static benchmarks can raise the benchmark while degrading real workflows, because benchmark tasks are not drawn from the production distribution. If evals must replay real user starting states, eval infrastructure becomes coupled to product state management — a concrete architectural dependency most teams have not budgeted for.

Evaluations must represent the full distribution of tasks and outcomes.
Open source video
Static benchmarks do not represent actual user workflows and requests.
Open source video
Source video

Harness/model co-evolution and the weights-versus-code boundary

Research question

When capability can be placed either in weights or in orchestration code, what cost/latency/retrainability tradeoffs determine the optimal split, and does a lighter, more informative harness outperform a hard-coded workflow harness over successive training rounds?

Why

Treating the harness as a trainable artifact breaks the assumption that the model is the only thing that improves, which changes what belongs in code versus in weights. This turns a fixed architectural boundary into a per-iteration optimization decision — a directly actionable design lever for agent teams.

Harnesses must co-evolve with models and contain domain knowledge.
Open source video
Models should be better, cheaper, and faster.
Open source video
Source video

Product affordances as a supervision channel (edit/undo/retry)

Research question

Do natively instrumented edit/undo/retry affordances yield training signal of sufficient quality and volume to replace a meaningful fraction of offline human annotation, and do they cause regressions when the action space has irreversible side effects?

Why

This relocates labeling from an offline annotation effort into the interaction loop at near-zero marginal cost, which changes the economics of continual learning. It also forces UI/UX and data-pipeline design to be co-designed and requires reversible state, which is nontrivial for agents with side effects.

Providing a structured product layer allows companies to own their own experience layer and continuously improve agents.
Open source video
Most interaction signals currently go to waste.
Open source video
Source video

Multi-tenant trace retention under privacy and isolation constraints

Research question

What is the learning-signal cost of enforcing per-customer trace isolation and differential-privacy noise budgets when training on multi-tenant production traces, and can isolation be enforced at the trace level without collapsing per-customer improvement?

Why

The same capture completeness that makes the learning loop work is what creates the privacy and isolation obligation, so the two constraints are in direct tension. Without a measured tradeoff curve, teams will either over-retain (compliance risk) or under-retain (capped learning), and the correct point is currently unspecified.

Traceability requires capturing full traces plus product telemetry including subagents.
Open source video
Most interaction signals currently go to waste.
Open source video
Source video

Article ideas

4

You Can Only Improve What You Can See: Trace Fidelity Is the Real Ceiling on Agent Continual Learning

The limiting factor in continually improving agents is not the optimizer or the model but the capture schema: if intermediate tool calls and subagent failures are not retained as first-class training inputs, the learning loop silently plateaus, and teams will misdiagnose that plateau as a model-capability problem.

Angle

Move observability out of the debugging-tooling category and into the training-data-contract category, showing how retention policy, schema design, and privacy controls become model-quality decisions with a measurable ceiling.

Source video

Benchmarks Are Not Evals: Why Rolloutable, State-Replayable Evaluation Is the Gate for Agent Training

Any iterative RL or fine-tuning loop gated on static benchmarks can improve the benchmark while degrading production workflows, so evaluations must be sampled from the live task distribution and able to replay real user starting states — which makes eval infrastructure a dependency of product state management, not a standalone test harness.

Angle

Contrast benchmark-shaped evals with executable, environment-matched replay, and enumerate the state-reconstruction requirements this imposes on the product and data layers.

Source video

The Harness Is a Weight: Rethinking What Belongs in Code Versus in Parameters

Once the harness is understood as a trained artifact that carries domain knowledge and must co-evolve with the model, the boundary between orchestration code and weights stops being a fixed architectural line and becomes an optimization decision driven by cost, latency, and retrainability — and hard-coded workflows become a liability rather than a safety net.

Angle

Argue for light, open, informative harnesses by showing how each hard-coded step is a capability locked out of the training loop.

Source video

Your Product UI Is a Labeling Pipeline

Edit, undo, retry, and error correction should be designed deliberately as supervision APIs — instrumented to emit clean training signals — because that relocates labeling from a scarce offline annotation effort into production interactions at near-zero marginal cost.

Angle

Reframe ordinary interaction primitives as data-collection surfaces, and confront the cost: reversible state for agents with side effects, plus a second responsibility for every product surface.

Source video

Project ideas

4

Trace-Granularity Ablation Bench

new

Increasing trace capture granularity from outcome-only to tool-call level to full subagent execution trees monotonically reduces multi-commit regression rate in a coding agent after successive training rounds, with a measurable threshold below which added trace noise reverses the gain.

Proof of concept

Instrument one coding agent to emit three parallel trace streams at increasing granularity over the same task distribution; run identical continual-learning cycles per stream and replay the same task set after each commit sequence; plot regression rate against capture depth and against added-noise volume.

Measurement

Regression rate per commit index per granularity level, plus improvement in rolloutable-eval pass rate and the cost/latency overhead of each capture depth.

Source video

Replay-Eval Harness

beyond-evals

An eval harness that reconstructs production starting states and samples the live task distribution predicts post-update production performance significantly better than a static benchmark suite, and the predictive gap widens as domain-specific experience accumulates.

Proof of concept

Build a replayer that snapshots and restores real user starting states, sample tasks from captured production traces, and run the same set of model/harness updates through both the replay harness and a static benchmark; compare predicted versus observed production outcomes.

Measurement

Rank correlation between gate decision and post-deploy production outcome, false-ship and false-block rates, and the divergence between benchmark delta and production delta over successive training rounds.

Source video

Weights-vs-Workflow Splitter

movement-lab

Moving a defined capability from hard-coded orchestration into model weights (or vice versa) produces a predictable cost/latency/retrainability outcome, and a light, open harness outperforms a hard-coded workflow harness after N co-evolution rounds.

Proof of concept

Take one agent task with a workflow-style harness, enumerate steps, and produce paired variants where each step lives in code versus weights; train both variants on the same trace corpus across several rounds and compare on rolloutable evals.

Measurement

Task success and regression rate per round, inference cost and latency per task, and the size of the training delta needed to absorb each migrated capability into weights.

Source video

Isolated Trace Trainer

gatehouse

Per-customer trace isolation plus a bounded privacy budget can be enforced without collapsing per-customer improvement, and there exists a measurable isolation granularity (trace-level vs. tenant-level) where learning signal per customer remains positive while cross-tenant leakage is eliminated.

Proof of concept

Split a multi-tenant trace corpus into isolated shards with configurable privacy noise budgets; train per-tenant adapters alongside a shared baseline, then measure per-customer improvement plus a leakage-attack probe against the shared components.

Measurement

Per-tenant eval delta versus a non-isolated control, privacy budget consumed per unit of learning signal, and leakage-probe success rate across isolation granularities.

Source video

Architectural implications

4

Observability data becomes training data.

Before

Traces and telemetry are retained briefly for debugging and final outputs are the unit of record.

After

Full execution trees including subagents plus product telemetry are retained and queryable as first-class training inputs.

Consequence

Storage, retention policy, schema design, and privacy controls move onto the critical path of model quality; incomplete capture permanently caps achievable improvement.

Source video

The model layer changes ownership.

Before

A closed hosted API model is the system's intelligence and is treated as fixed.

After

The model becomes a trainable artifact the company owns, with open weights and a deployable fine-tune pipeline.

Consequence

Companies need training, eval, and deployment infrastructure with minute-scale cycles, and lose the automatic capability upgrades of closed frontier APIs — a new operational surface.

Source video

Evaluation boundary shifts from benchmark to replayable production environment.

Before

Static benchmark suites are the gate for shipping a model or prompt change.

After

Rolloutable evals sample production task distributions and replay real user starting states.

Consequence

Eval infra must be able to reconstruct environment state, which couples eval design to product state management.

Source video

User correction interactions become a supervision interface.

Before

Edit, undo, and retry are interaction polish, and labels are collected separately offline.

After

They are deliberately instrumented as native affordances that emit clean training signals.

Consequence

UI/UX design and data pipeline design must be co-designed; product surfaces gain a second responsibility as a labeling channel.

Source video

Tradeoffs and failure modes

4

Full trace and telemetry capture vs. privacy and customer isolation

Benefit

Capturing complete execution trees including subagents yields the signal needed to credit and fix failures.

Cost or risk

Retaining user interaction data for continual training raises differential-privacy and per-customer data-isolation obligations that the summary explicitly leaves unresolved.

How do we handle differential privacy and customer data isolation during continual learning?
Open source video
Source video

Open-weight training vs. closed hosted frontier models

Benefit

Open weights unlock continual training and instant deployment of improved models.

Cost or risk

Forfeits the automatic capability gains and reliability of closed hosted frontier models, in exchange for operating your own training and deployment stack.

Transition from closed hosted models to training on open-weight models to unlock autonomous improvement.
Open source video
Source video

Human feedback vs. automated completion signals in credit assignment

Benefit

Both signals are available from production usage and can drive RL updates.

Cost or risk

Human feedback is high-quality but scarce and expensive; automated completion signals are abundant but noisy, and the correct weighting is unspecified.

How do we weight human feedback versus automated task completion signals in RL credit assignment?
Open source video
Source video

Native edit/undo/retry affordances vs. product surface complexity

Benefit

Produces clean supervision labels from genuine user corrections.

Cost or risk

Adds interaction complexity and requires the product to support reversible state, which is nontrivial for agents with side effects.

Design products with native affordances for edits, undos, and retries to generate clean supervision labels.
Open source video
Source video

Open questions

5

How should differential privacy and per-customer data isolation be enforced when training on multi-tenant production traces?

Why unresolved

The summary raises this as an open question investigating continual learning and provides no mechanism or demonstrated solution.

Research direction

Prototype trace-level isolation and DP noise budgets across tenants, measuring the tradeoff between learning signal per customer and privacy guarantees.

Source video

What role does episodic memory play in updating agent behavior after user corrections?

Why unresolved

Corrections can be absorbed either as weights updates or as retrieved episodic memory, and the summary does not specify which is used when or how they interact.

Research direction

Compare correction-persistence via fine-tuning versus retrieval-based episodic memory on the same production task distribution, measuring retention and interference.

Source video

How should human feedback be weighted against automated task-completion signals in RL credit assignment?

Why unresolved

Both signal sources exist in production traces but no weighting scheme or evidence is given.

Research direction

Build a hybrid reward model and ablate the human/automated weighting to see which produces fewer regressions on rolloutable evals.

Source video

Do domain-specialized models trained on company experience actually match large general models at a fraction of the cost, and at what data scale?

Why unresolved

Stated as a claim worth verifying, with no benchmark, model size, or data volume reported in the summary.

Research direction

Replicate the claim on a domain task distribution with matched evals, sweeping training data volume to find where specialization overtakes general capability.

Source video

How complete must trace capture be before the learning loop stops producing regressions (e.g., the coding-agent brittleness after a few commits)?

Why unresolved

The failure mode is asserted but no threshold, capture granularity, or measurement protocol is provided.

Research direction

Ablate trace granularity (final outcome only vs. tool calls vs. subagent trees) and measure downstream regression rate after successive commits.

Source video

Key claims

7
factualVerification needed

Production AI generates on the order of 100 billion tokens per day, and most of that interaction signal goes to waste.

Evidence

100 billion tokens per day are generated by production AI.

Question

What is the measured volume of production agent tokens, and what fraction is retained or used for training today?

Source video
causalVerification needed

Discarding subagent failures causes coding agents to break after a few commits.

Evidence

Coding agents frequently break after a few commits if subagent failures are discarded.

Question

Does trace granularity measurably change multi-commit regression rates in coding agents?

Source video
comparativeVerification needed

Training models on domain expertise matches large general models at a fraction of the cost.

Evidence

Training models on domain expertise matches large general models at a fraction of the cost.

Question

On which evals and at what training data scale does a domain-specialized model match a large general model, and at what cost delta?

Source video
opinionVerification needed

Frontier labs optimize for IQ only, leaving an orthogonal experience axis unaddressed.

Evidence

Frontier labs optimize solely for IQ.

Question

Is the IQ/experience separation empirically observable, e.g., do frontier models fail on experience-dependent production tasks that smaller trained models pass?

Source video
predictionVerification needed

Every company should own its own experience layer that never stops evolving.

Evidence

Every company should own its own experience layer that never stops evolving.

Question

Does owning an experience layer produce durable advantage versus renting capability from closed model providers?

Source video
factualVerification needed

Continual learning infrastructure can reduce post-training and deployment workflows to minutes, demonstrated by importing a 1,760-task benchmark, training, comparing evals, and one-click deployment.

Evidence

Continual learning infrastructure can reduce post-training and deployment workflows to minutes.

Question

What is the wall-clock time and cost of a full train-eval-deploy cycle under this pipeline, and how much human intervention is required?

Source video
causalVerification needed

You can only improve what you can see.

Evidence

You can only improve what you can see.

Question

Does increasing trace coverage translate monotonically into measurable agent improvement, or does added noise degrade learning?

Source video

Connections

5