AI Engineer · Published 2026-09-26

Beating RL With Reflection: GEPA and Optimize Anything — Lakshya A. Agrawal, GEPA

privateReview: unreviewedConfidence: high
GEPAoptimizationreflectionparetoagentsreflective-optimizationpareto-selectionprompt-optimizationagent-architecturesample-efficiencytext-space-optimizationevolutionary-search
Open on YouTube ↗

Summary

Overview

  • Speaker: Lakshya A. Agrawal
  • Channel: AI Engineer
  • Main topic: Reflective Optimization (GEPA and optimize_anything) for prompts, agents, and models from text feedback
  • Purpose: To introduce GEPA and optimize_anything frameworks as sample-efficient, text-space optimization alternatives to gradient-based RL/SFT. Lakshya A. Agrawal presents GEPA (Reflective Prompt Optimization for Agents) and optimize_anything, a universal API for optimizing any text parameter using textual feedback and Pareto-based candidate selection. The talk covers how traditional Reinforcement Learning (RL) and supervised fine-tuning (SFT) suffer from sample inefficiency and expensive rollouts, whereas optimizing in text space via reflection on intermediate traces and tool outputs provides dramatic performance gains, cost reductions, and state-of-the-art results across tasks like multi-hop QA, coding, math, and agent architecture discovery.

Topic Map

The Limitations of Standard AI Task Teaching

  • Explanation: Standard methods like pretraining, SFT, and RL require massive compute, trillions of tokens, 10,000s of labeled examples, or 100,000s of rollouts. These approaches suffer from bottleneck sample efficiency, low availability of domain-specific knowledge, and expensive, slow agent rollouts.
  • Key claims:
    • Weight updates with gradient descent require massive data and compute.
    • Modern agentic workflows have expensive, slow rollouts.
    • Online learning algorithms applied to long agent traces are not feasible due to sample inefficiency.
  • Examples:
    • Trillions of tokens for pretraining
    • 10,000s of labeled examples for SFT
    • 100,000s of rollouts for RL in math and coding
  • Terminology:
    • gradient descent
    • pretraining
    • supervised fine-tuning (SFT)
    • reinforcement learning (RL)
    • sample efficiency
  • Why it matters: Most teams lack the compute or data required for traditional gradient-based fine-tuning on domain-specific agent tasks.

The Core Idea: Reflective Optimization in Text Space

  • Explanation: Instead of using sparse 0/1 reward signals and gradient descent, use a language model to inspect entire rollout traces (including intermediate outputs, tool calls, and error messages) and reflect on what worked and what did not, updating prompts via natural language.
  • Key claims:
    • Reflective optimization uses all intermediate outputs and tool traces.
    • Updating a prompt via natural language updates can induce large behavior changes instantly.
    • Text space optimization avoids thousands of tiny sequential gradient updates.
  • Examples:
    • Changing a summary prompt from 'generate a 1-line summary' to 'generate a 10-line summary' instantly alters behavior.
  • Terminology:
    • reflective optimization
    • text space
    • natural language updates
    • intermediate outputs
  • Why it matters: Enables rapid model and agent adaptation using rich domain feedback without weight updates.

GEPA: Reflective Prompt Optimization for Agents

  • Explanation: GEPA uses an evolutionary loop combined with a Pareto-based candidate selection strategy. It performs RL in text space, obtaining scores alongside domain-specific textual feedback.
  • Key claims:
    • GEPA achieves twice the performance gains of GRPO using significantly fewer rollouts.
    • GEPA discovers test-case-specific solutions first and aggregates them later via a Pareto pool.
    • GEPA optimizes Qwen3 8B and GPT-4-mini without external expert teachers.
  • Examples:
    • HotpotQA and HoVer multi-hop QA benchmarks
    • Programming AMD accelerators (NPUExt2 API)
  • Terminology:
    • GEPA
    • GRPO
    • Pareto pool
    • evolutionary loop
    • candidate selection
  • Why it matters: Allows open-source and proprietary models to self-improve significantly on complex reasoning tasks autonomously.

optimize_anything: A Universal API for Optimizing Any Text Parameter

  • Explanation: A universal API that takes a set of problems to be solved and an evaluator/fitness function returning a score and side information (e.g., error messages, profiler outputs), and iteratively optimizes any text parameter (code, prompts, configurations, policies).
  • Key claims:
    • If you can write it as text and score it, GEPA and optimize_anything can optimize it.
    • Applicable to code optimization, numeric optimization, agent harness optimization, and policy optimization.
    • Achieved 90x cost reduction for Databricks agent pipelines and cut cloud scheduling costs by 40%.
  • Examples:
    • CUDA kernel code optimization
    • 3D modeling with Blender and Pyrender
    • Agent architecture discovery on ARC-AGI (improving from 32.5% to 89.5%)
  • Terminology:
    • optimize_anything
    • fitness function
    • actionable side information
    • evaluator
    • Pareto pool
  • Why it matters: Unifies prompt optimization, code generation, and agent design under a single simple Python API.

Key Points

Text-Space Reflection Beats Reward-Only Gradient Updates

  • Explanation: Standard RL uses 0/1 reward signals, discarding rich diagnostic data like tool error messages. Reflective optimization leverages all intermediate outputs to guide prompt/agent evolution.
  • Evidence: GEPA achieves GRPO-level performance gains in just 3 data points / reflection rounds compared to 25,000 rollouts.
  • Practical implication: Teams can optimize complex agents using natural language feedback loops instead of expensive RL pipelines.

Pareto-Based Candidate Selection Prevents Local Optima

  • Explanation: Standard optimization loops keep only the single best candidate and get stuck in local optima. A Pareto pool retains every candidate that wins on even one training example.
  • Evidence: Over four benchmarks, Pareto-based candidate selection accounts for more than half of GEPA's performance gains, doubling standard loop performance.
  • Practical implication: Always maintain a diverse pool of performing candidates during evolutionary search.

optimize_anything Generalizes Beyond Prompts

  • Explanation: Because prompts, code, configurations, and policies are all text artifacts, any system component that can be represented as text and evaluated can be optimized.
  • Evidence: Successfully applied to code optimization, cloud scheduling policies, math reasoning prompts, and ARC-AGI agent architectures.
  • Practical implication: You can optimize entire agent harnesses, scripts, and scheduling policies with a single Python function call.

Frameworks, Models & Processes

GEPA (Reflective Prompt Optimization for Agents)

  • How it works: Runs the system on a few examples to collect feedback, uses an LLM/agent to reflect on feedback and propose a better prompt, and maintains a Pareto pool of successful candidates.
  • Components:
    • Run (system execution and feedback collection)
    • Reflect (LLM-based prompt improvement)
    • Keep a Pareto pool (candidate retention strategy)
  • When to use: When optimizing prompts, agent instructions, or components where domain feedback and traces are available.

optimize_anything

  • How it works: A universal API where you provide a dataset (problems), an evaluator function (returning score and actionable side information), and a seed candidate, and it iteratively optimizes the text parameter via an LLM.
  • Components:
    • Set of problems to be solved
    • Fitness function (returns score & side info)
    • Seed candidate
  • When to use: When optimizing code kernels, numeric parameters, cloud policies, agent skills, or prompts.

Examples & Case Studies

Optimizing a coding agent to program a new AMD accelerator (NPUExtDNa2) with zero web documentation.

  • Illustrates: GEPA's ability to inject domain-specific instructions into agent prompts.
  • Lesson: Push vector utilization from 4.25% to 30.52% in one optimization step by eliminating bad API assumptions.

Discovering agent architecture for ARC-AGI starting from a simple 4-line Python program.

  • Illustrates: Agent architecture discovery using optimize_anything.
  • Lesson: Improved Gemini Flash accuracy on ARC-AGI from 32.5% to 89.5% in 16 rounds of reflection.

Optimizing SKILL.md for Mini-SWE-Agent across repositories.

  • Illustrates: Learning agent skills from execution trajectories.
  • Lesson: Increased issue resolution rate on Claude Sonnet 4.5 to 100% while cutting execution time and token usage by 50%.

Actionable Takeaways

  • Immediate:
    • Install and test the open-source library at github.com/gepa-ai/gepa.
    • Use optimize_anything by defining a dataset, fitness function, and seed candidate.
  • Strategic:
    • Shift from heavy weight updates (RL/SFT) to text-space reflection and evolutionary prompt/code optimization.
    • Leverage actionable side info (compiler errors, traces, profiler logs) in your evaluator functions.
  • Questions to investigate:
    • How does text-space optimization scale with extremely large codebases?
    • What are the compute cost tradeoffs of LLM reflection vs traditional gradient steps?

Claims Worth Verifying

  • GEPA achieves in 3 data points the performance gains that GRPO got after 25,000 rollouts. (performance benchmark)
  • Databricks achieved 90x cost reduction using automated prompt optimization. (industry result)
  • Cloud scheduling policy optimization cut costs by 40% compared to expert heuristics. (system optimization)

Notable Quotes

"If you can write it as text and score it, GEPA can optimize it." "Don't be afraid to optimize in the text space."

Compressed Summary

  • GEPA performs reflective prompt and agent optimization using text feedback and Pareto candidate pools.
  • optimize_anything is a universal API for optimizing any text parameter (code, prompts, policies, configs).
  • Outperforms RL/SFT approaches by leveraging rich intermediate traces, tool outputs, and error messages.
  • Demonstrated massive gains across math, coding, ARC-AGI, cloud scheduling, and 3D modeling.
  • Keywords: gepa, optimization, reflection, pareto, agents
  • Core insight: Reflective optimization in text space using Pareto candidate pools achieves state-of-the-art agent and code performance with extreme sample efficiency compared to traditional reinforcement learning.

Core insights

6
Mechanismhigh noveltystrong evidence

Maintaining a Pareto pool that retains every candidate which wins on even a single training example — rather than keeping only the single best candidate — is responsible for more than half of GEPA's performance gains, doubling standard optimization-loop performance.

Why it matters

Standard evolutionary/optimization loops collapse to the current best and stall in local optima; a Pareto retention rule preserves specialists that later aggregate into a stronger general solution, which is a cheap and general fix to a ubiquitous failure mode.

Generalization

Any iterative search over candidates (prompts, code, configs, agent topologies) evaluated on a heterogeneous example set should retain Pareto-non-dominated candidates rather than a single global best.

A Pareto pool retains every candidate that wins on even one training example.
Open source video
Over four benchmarks, Pareto-based candidate selection accounts for more than half of GEPA's performance gains, doubling standard loop performance.
Open source video
Mechanismhigh noveltystrong evidence

Reflective optimization replaces sparse 0/1 reward with full rollout traces — intermediate outputs, tool calls, and error messages — as the supervision signal fed back to an LLM that rewrites the artifact in natural language.

Why it matters

Reward-only RL discards exactly the diagnostic data (tool errors, intermediate reasoning) that a language model could act on; feeding rich traces into a reflection step collapses the sample budget needed to improve a system.

Generalization

When an evaluator can surface structured diagnostics alongside a scalar score, prefer consuming those diagnostics in a text-space optimizer over training on the scalar alone.

Standard RL uses 0/1 reward signals, discarding rich diagnostic data like tool error messages.
Open source video
Reflective optimization uses all intermediate outputs and tool traces.
Open source video
GEPA achieves GRPO-level performance gains in just 3 data points / reflection rounds compared to 25,000 rollouts.
Open source video
Architecturehigh noveltymoderate evidence

The evaluator/fitness function is designed to return not just a score but 'actionable side information' (error messages, profiler outputs) that directly drives the reflection step — making the evaluator's return shape a first-class architectural contract.

Why it matters

It shifts the evaluator from a pure scoring gate into the primary source of optimization signal; the richness of its side information bounds how much the optimizer can improve, so evaluator design becomes a core engineering task.

Generalization

Harness and eval infrastructure should be designed to emit diagnostic payloads, not just pass/fail, whenever an automated optimizer will consume them.

takes a set of problems to be solved and an evaluator/fitness function returning a score and side information (e.g., error messages, profiler outputs)
Open source video
If you can write it as text and score it, GEPA and optimize_anything can optimize it.
Open source video
Mental Modelhigh noveltymoderate evidence

Because prompts, code, configurations, and policies are all text artifacts, a single optimization API can target any of them — unifying prompt engineering, code generation, agent-harness design, and scheduling-policy tuning under one interface.

Why it matters

It reframes 'prompt optimization' as a general text-parameter optimization problem, which changes where teams should invest: one optimizer plus good evaluators instead of bespoke per-artifact tuning tooling.

Generalization

Any component expressible as text and amenable to automated scoring is a candidate for the same reflective optimization loop.

Applicable to code optimization, numeric optimization, agent harness optimization, and policy optimization.
Open source video
Achieved 90x cost reduction for Databricks agent pipelines and cut cloud scheduling costs by 40%.
Open source video
Empirical Resultmedium noveltymoderate evidence

Optimization can target agent architecture itself, not just its instructions: GEPA's discovery on ARC-AGI raised performance from 32.5% to 89.5%, and the loop also optimized CUDA kernels, 3D modeling scripts, and cloud scheduling policies.

Why it matters

It demonstrates that the search space can include structural/topological decisions and non-ML code, expanding the scope of what 'optimization' covers in an agent stack.

Generalization

Where a structured artifact has an automated evaluator, architecture-level search over that artifact is viable without gradient access.

Agent architecture discovery on ARC-AGI (improving from 32.5% to 89.5%)
Open source video
CUDA kernel code optimization
Open source video
3D modeling with Blender and Pyrender
Open source video
Practicehigh noveltymoderate evidence

GEPA performs 'RL in text space' by running the system on a few examples, reflecting on feedback to propose a better prompt, and retaining Pareto-successful candidates — without updating model weights or needing an external expert teacher.

Why it matters

It offers a weight-free improvement path for open-source and proprietary models, sidestepping the compute/data barriers of SFT and RL rollouts for domain-specific agent tasks.

Generalization

Teams lacking large labeled datasets or rollout budgets can still raise agent performance by iterating on text artifacts with an LLM reflector.

GEPA optimizes Qwen3 8B and GPT-4-mini without external expert teachers.
Open source video
GEPA achieves twice the performance gains of GRPO using significantly fewer rollouts.
Open source video

Deep dives

5

Pareto pool scaling and aggregation cost

Research question

At what candidate population size and example-set size does maintaining a Pareto pool stop being compute-cheap and become the throughput bottleneck of reflective optimization?

Why

The Pareto retention rule is claimed to drive more than half of GEPA's gains, but its storage, evaluation, and aggregation cost determines whether it is practical for long agent traces.

A Pareto pool retains every candidate that wins on even one training example.
Open source video
Over four benchmarks, Pareto-based candidate selection accounts for more than half of GEPA's performance gains, doubling standard loop performance.
Open source video
Source video

Held-out generalization of Pareto-aggregated candidates

Research question

Do reflectively optimized artifacts that specialize per training example retain performance on held-out examples after Pareto aggregation, or do they overfit?

Why

If test-case-specific solutions drive gains, the method risks benchmark overfitting and its production value is unclear.

GEPA discovers test-case-specific solutions first and aggregates them later via a Pareto pool.
Open source video
Over four benchmarks, Pareto-based candidate selection accounts for more than half of GEPA's performance gains, doubling standard loop performance.
Open source video
Source video

Evaluator diagnostic richness as the optimization budget

Research question

How much of GEPA's reported performance and cost gains are attributable to evaluator side-information quality versus the Pareto retention rule itself?

Why

The evaluator is a first-class architecture contract; if diagnostic richness is the limiting factor, teams must invest in eval payloads before optimizers.

takes a set of problems to be solved and an evaluator/fitness function returning a score and side information (e.g., error messages, profiler outputs)
Open source video
Standard RL uses 0/1 reward signals, discarding rich diagnostic data like tool error messages.
Open source video
Source video

Reflector error propagation under noisy side information

Research question

When the LLM reflector receives misleading or incomplete diagnostics, how quickly does optimized artifact quality degrade, and can Pareto retention buffer it?

Why

Reflection is treated as a component, but production traces contain noisy errors; robustness determines whether text-space optimization is reliable.

Reflective optimization uses all intermediate outputs and tool traces.
Open source video
Updating a prompt via natural language updates can induce large behavior changes instantly.
Open source video
Source video

Boundary conditions for write-it-as-text-and-score-it

Research question

Which artifact classes have text representations too lossy or evaluators too sparse for reflective optimization to outperform random or gradient baselines?

Why

The universal API claim needs falsifiable scope limits to guide adoption.

If you can write it as text and score it, GEPA and optimize_anything can optimize it.
Open source video
Applicable to code optimization, numeric optimization, agent harness optimization, and policy optimization.
Open source video
Source video

Article ideas

4

Pareto Retention Is the Real Optimizer: Why GEPA's Gains Aren't Just Reflection

The evidence that Pareto-based candidate selection accounts for more than half of GEPA's gains means reflective text-space optimization is primarily a search-diversity problem, so teams should adopt Pareto retention before investing in more elaborate reflectors.

Angle

Ablation-driven argument that separates the reflector from the retention strategy.

Source video

Design Your Evaluator as a Teacher, Not a Judge

Because GEPA consumes score plus actionable side information, eval harnesses that return only pass/fail are leaving most optimization signal on the table and will underperform on the same tasks.

Angle

Architecture contract for evaluator return shapes and diagnostic payloads.

Source video

Text-Space Optimization Collapses the Prompt/Code/Config Boundary

If prompts, code, configs, and policies are all text artifacts scored by evaluators, then one reflective optimizer plus good evaluators should replace bespoke per-artifact tuning pipelines.

Angle

Platform architecture argument with caveats from scope and generalization risks.

Source video

The Overfitting Trap in Reflective Optimization

Pareto aggregation of test-case-specific solutions is a strength and a risk: without held-out splits, reported gains may reflect benchmark memorization rather than transferable capability.

Angle

Methodological critique and protocol for evaluating reflective optimizers.

Source video

Project ideas

4

SideInfo Ablation Harness

beyond-evals

At equal rollout or evaluation budget, reflective optimization with score plus diagnostic side information yields higher held-out performance than score-only feedback, and the gap grows with task diagnostic complexity.

Proof of concept

Wrap one evaluator to emit either scalar-only or scalar plus error/profiler messages. Run the same optimizer on a multi-hop QA or coding task. Compare final artifacts.

Measurement

Delta in held-out accuracy and number of reflection rounds to reach baseline score.

Source video

ParetoPoolBench

movement-lab

For a fixed evaluation budget, Pareto-pool retention improves held-out score over single-best retention up to a crossover candidate population, after which aggregation overhead reverses the gain.

Proof of concept

Implement a minimal reflective loop over a small prompt or code task with two modes: single-best versus Pareto pool. Sweep example-set size and candidate population. Log pool size, eval calls, aggregation time, train score, and held-out score.

Measurement

Held-out score delta per 1k evaluator calls and wall-clock time until aggregation exceeds 50 percent of loop time.

Source video

Pareto Generalization Splitter

new

Pareto aggregation of test-case specialists overfits training examples; on held-out examples it underperforms a single-best candidate selected by validation score when per-example specialization is high.

Proof of concept

Run reflective optimization with train, validation, and test splits. Compare Pareto-aggregated candidates versus single-best candidates on held-out examples. Vary example heterogeneity.

Measurement

Train-test gap in score and variance across splits.

Source video

Reflector Noise Injector

new

Misleading side information degrades optimized artifact quality monotonically with feedback error rate, but Pareto retention reduces the degradation slope versus single-best retention.

Proof of concept

Inject controlled false error messages into evaluator feedback. Run reflective optimization with Pareto versus single-best retention. Measure score across noise levels.

Measurement

Slope of held-out score decline per 10 percent misleading feedback, with variance across seeds.

Source video

Architectural implications

3

The optimization loop sits outside the model weights, operating over externalized text artifacts rather than parameters.

Before

Improvement meant collecting data and fine-tuning or running RL on weights.

After

Improvement means running a reflect-and-evaluate loop over prompts, code, configs, and policies with a Pareto pool of candidates.

Consequence

Model weights and harness artifacts decouple; the harness becomes the primary site of iteration and must be versioned/evaluated independently.

Source video

The evaluator must emit a score plus diagnostic side information because that payload is the optimizer's learning signal.

Before

Evaluation returned a scalar reward or pass/fail.

After

Evaluation returns score plus error messages, profiler output, and traces consumable by an LLM reflector.

Consequence

Eval harness design becomes an optimization-quality bottleneck; richer diagnostics yield better candidates.

Source video

Search state is a persistent Pareto pool of candidates rather than a single incumbent.

Before

The loop kept the single best candidate and discarded the rest.

After

The loop retains all candidates non-dominated on at least one example, aggregating them later.

Consequence

Optimizer state and storage grow, but local-optima traps are mitigated — diversity becomes a managed resource.

Source video

Tradeoffs and failure modes

3

Pareto pool retention

Benefit

Prevents local optima and accounts for more than half of performance gains by preserving specialists.

Cost or risk

Keeping every non-dominated candidate increases state, evaluation, and aggregation cost, and may delay convergence as the pool grows.

A Pareto pool retains every candidate that wins on even one training example.
Open source video
Source video

Actionable side information dependency

Benefit

Rich textual feedback (tool errors, profiler output) enables sample-efficient reflection.

Cost or risk

Optimization quality is bounded by the quality of textual feedback; sparse or misleading side information degrades the reflector's updates, and the summary does not characterize how much.

Standard RL uses 0/1 reward signals, discarding rich diagnostic data like tool error messages.
Open source video
Source video

Test-case-specific solutions aggregated later

Benefit

Different candidates can specialize on particular examples before being combined.

Cost or risk

Optimizing per-example solutions risks overfitting to the training set; generalization to held-out cases is not established in the summary.

GEPA discovers test-case-specific solutions first and aggregates them later via a Pareto pool.
Open source video
Source video

Open questions

4

How does the Pareto pool scale with candidate count and problem size, and when does aggregation become the throughput bottleneck?

Why unresolved

The summary reports the retention rule's benefit but not its storage/evaluation cost or scaling behavior.

Research direction

Measure pool growth and aggregation overhead across benchmark sizes and candidate populations.

Source video

Do reflectively optimized artifacts generalize beyond the examples used for optimization, or do they overfit test-case-specific solutions?

Why unresolved

The summary notes test-case-specific solutions aggregated via the pool but provides no held-out generalization evidence.

Research direction

Run train/test splits on the reported benchmarks and compare generalization of Pareto-aggregated vs single-best candidates.

Source video

How reliable is the LLM reflector, and what happens when reflection produces incorrect hypotheses about why a rollout failed?

Why unresolved

The summary treats the reflect step as a component without discussing reflector error or its effect on optimization.

Research direction

Inject noisy or misleading feedback and measure degradation in optimized artifact quality.

Source video

Which of the reported gains (90x cost reduction, 40% scheduling cost cut, ARC-AGI 32.5%→89.5%) are attributable to reflective optimization versus the availability of domain-specific feedback signals?

Why unresolved

The summary reports outcomes without ablating the contribution of side-information quality versus the optimization loop itself.

Research direction

Ablate evaluator diagnostic richness to isolate its contribution to end results.

Source video

Key claims

7
comparativeVerification needed

GEPA achieves GRPO-level performance gains with roughly three data points/reflection rounds versus 25,000 rollouts.

Evidence

GEPA achieves GRPO-level performance gains in just 3 data points / reflection rounds compared to 25,000 rollouts.

Question

On which benchmarks and models was this comparison run, and how were GRPO and GEPA budgets normalized?

Source video
causalVerification needed

Pareto-based candidate selection accounts for more than half of GEPA's performance gains across four benchmarks.

Evidence

Over four benchmarks, Pareto-based candidate selection accounts for more than half of GEPA's performance gains, doubling standard loop performance.

Question

What is the ablation setup that isolates the Pareto retention rule from other GEPA components?

Source video
comparativeVerification needed

GEPA achieves twice the performance gains of GRPO using significantly fewer rollouts.

Evidence

GEPA achieves twice the performance gains of GRPO using significantly fewer rollouts.

Question

What metric defines 'performance gains' and on what task set?

Source video
predictionVerification needed

Any artifact that can be written as text and scored can be optimized by GEPA/optimize_anything.

Evidence

If you can write it as text and score it, GEPA and optimize_anything can optimize it.

Question

What artifact classes fail this criterion in practice, and where does text representation become a limiting factor?

Source video
factualVerification needed

Reflective optimization improved ARC-AGI agent architecture performance from 32.5% to 89.5%.

Evidence

Agent architecture discovery on ARC-AGI (improving from 32.5% to 89.5%)

Question

On which ARC-AGI split was this measured and with what optimization budget?

Source video
factualVerification needed

Applying the optimizer to Databricks agent pipelines yielded a 90x cost reduction and cut cloud scheduling costs by 40%.

Evidence

Achieved 90x cost reduction for Databricks agent pipelines and cut cloud scheduling costs by 40%.

Question

What baseline and cost accounting underlie the 90x and 40% figures?

Source video
causalVerification needed

Modern agentic workflows have rollouts so expensive and slow that online learning on long traces is not feasible.

Evidence

Online learning algorithms applied to long agent traces are not feasible due to sample inefficiency.

Question

At what trace lengths and costs does online learning become infeasible relative to reflective optimization?

Source video

Connections

5

GEPAGRPO

Direct comparison: reflective text-space optimization claimed to match/exceed RL gains at far lower rollout counts.

Source video