AI Engineer · Published 2026-10-11

Hill-Climbing Skills: Improve Agents Without Changing the Model — Shubhankar Srivastava, Browserbase

Open on YouTube ↗

Summary

Overview

  • Speaker: Shubhankar Srivastava
  • Channel: AI Engineer
  • Main topic: Improving browser agents via hill-climbing self-improving loops without changing model weights
  • Purpose: To demonstrate how browser agents can self-improve and become more reliable through iterative loops and skill extraction without fine-tuning underlying LLMs. Shubhankar Srivastava, sales engineer at Browserbase, discusses how to improve browser agents in production without modifying model weights. He introduces the concept of hill-climbing through an auto-research loop inspired by Andrej Karpathy's auto-research concept, utilizing a doer-teacher agent architecture that generates production playbooks (skill.md files) containing workflow steps and heuristics learned across iterations.

Topic Map

Introduction to Browser Agents and Reliability

  • Explanation: Defines what browser agents and Browserbase are, and addresses reliability as the biggest engineering problem facing agents today.
  • Key claims:
    • Browser agents are not reliable due to getting stuck, captcha issues, or incomplete tasks.
    • Reliability is an engineering problem rather than solely a model capability problem.
    • Every run starts from zero without built-in learning from past execution traces.
  • Examples:
    • A browser agent failing to drag the time filter slider on Google Flights.
    • An agent encountering an 'Access Denied' anti-bot page on OpenTable.
  • Terminology:
    • browser agent
    • Browserbase
    • browse CLI
    • reliability
  • Why it matters: Establishes the core problem of agent fragility in production environments.

The Auto-Research Loop (Doer & Teacher Architecture)

  • Explanation: Presents an automated loop inspired by Andrej Karpathy's autoresearch concept, combining a doer agent and a teacher agent to iteratively refine execution strategies.
  • Key claims:
    • The auto-research loop uses two agents: a doer that browses and a teacher that analyzes traces.
    • A strategy.md file serves as the persistent artifact across runs, accumulating learnings.
    • The output of the converged loop is a production playbook (skill.md) containing exact workflows and heuristics.
  • Examples:
    • Iterating on Google Flights search to discover that pre-filled URLs and JS injection are superior to calendar clicking.
  • Terminology:
    • doer agent
    • teacher agent
    • strategy.md
    • skill.md
  • Why it matters: Provides a scalable alternative to costly fine-tuning or manual prompt engineering for every new task.

browse.sh Product and Ecosystem

  • Explanation: Introduces browse.sh as muscle memory for the web, offering free read-made skills, prompt-to-skill conversion, and suggested selectors.
  • Key claims:
    • Browse.sh provides over 400 ready-made skills that can be imported into any agent.
    • Users can submit a prompt to automatically build a skill for a site.
    • Suggested selectors cut token costs significantly.
  • Examples:
    • Using browse CLI commands like browse skills add alltrails.com.
  • Terminology:
    • browse.sh
    • browser CLI
    • skill catalog
    • token cost reduction
  • Why it matters: Demonstrates practical tooling for developers to deploy reliable browser automation at scale.

Generalizability Beyond Browsers

  • Explanation: Explores how the train-skill loop applies to coding agents, research agents, and internal operations.
  • Key claims:
    • Any agent working a live system re-learns from zero unless equipped with a train-skill loop.
    • Applying the loop requires two conditions: a replayable environment and a verifiable outcome.
    • Replayable environments and verifiable metrics enable cheap, compounding improvement.
  • Examples:
    • Coding agents re-learning repo conventions and research agents rediscovering which sources work.
  • Terminology:
    • replayable environment
    • verifiable outcome
    • coding agents
    • research agents
  • Why it matters: Broadens the application of hill-climbing loops to any domain requiring agent reliability.

Key Points

Browser agents suffer from lack of cumulative learning

  • Explanation: Every agent execution starts from scratch, repeating mistakes because there is no built-in mechanism to learn from production traces.
  • Evidence: Failures on Google Flights slider dragging and OpenTable anti-bot screens.
  • Practical implication: Developers must build harness infrastructure around models to capture and persist learnings.

The Doer-Teacher loop enables autonomous skill creation

  • Explanation: A doer agent executes tasks using browser tools while a teacher agent analyzes screenshots and logs to update a strategy file after every iteration.
  • Evidence: Live demo showing cost and latency decreasing across iterations while success rate increases.
  • Practical implication: Reduces engineering hours spent manually writing and updating scraping scripts.

Two conditions are required for domain hill-climbing

  • Explanation: To successfully run a self-improving loop, you need a replayable environment and a verifiable outcome.
  • Evidence: Comparison between coding agents with deterministic testing and open-ended research agents.
  • Practical implication: Without determinism and verification, agents cannot reliably evaluate whether a strategy iteration succeeded.

Frameworks, Models & Processes

Auto-Research Loop (Doer-Teacher)

  • How it works: A prompt goes into the loop; the doer executes the task using browser tools, and the teacher evaluates traces and screenshots to fix strategy.md. When a run passes, the skill graduates to skill.md.
  • Components:
    • Doer agent
    • Teacher agent
    • strategy.md artifact
    • skill.md output
  • When to use: When building browser agents or autonomous workflows that need to handle tricky, non-deterministic websites reliably.

Examples & Case Studies

A browser agent tried to drag the time filter slider on Google Flights and got stuck.

  • Illustrates: The fragility of general-purpose models when interacting with complex UI elements like drag sliders.
  • Lesson: Agents should use pre-filled URLs and query parameters instead of complex manual UI interactions.

An agent hit an 'Access Denied' anti-bot page on OpenTable.

  • Illustrates: Site-specific anti-bot and captcha failure modes in browser automation.
  • Lesson: Avoid navigating via homepage roots; use direct navigation URLs with lat-long metro pins.

Actionable Takeaways

  • Immediate:
    • Use pre-filled URLs instead of navigating homepages from scratch.
    • Leverage browser CLI tools like Browserbase for robust infrastructure.
    • Inject strategy markdown files into system prompts for iterative learning.
  • Strategic:
    • Reliability is an engineering problem solved by harnesses and loops, not just better foundation models.
    • Build replayable environments and verifiable outcomes to enable agent self-improvement.
    • Shift from manual script writing to automated skill generation via doer-teacher loops.
  • Questions to investigate:
    • How to optimize for convergence to reduce iteration costs in high-cost tasks?
    • How to handle rapidly changing website DOM structures without breaking existing skills?
    • Can we use a verifier as a service for browsing tasks involving mock side effects like purchases?

Claims Worth Verifying

  • Auto-research loops can reduce task costs by up to 16x across iterations. (Performance metric)
  • browse.sh offers over 400 ready-made skills for free use. (Product offering)

Notable Quotes

"Browser agents are not reliable. But reliability is an engineering problem." "Browser agents are geniuses that never learn."

Compressed Summary

  • Browser agents struggle with reliability in production due to lack of memory and learning.
  • The auto-research loop pairs a doer agent with a teacher agent to iteratively refine strategies.
  • strategy.md captures learnings across runs and graduates into reusable skill.md production playbooks.
  • Successful domain self-improvement requires a replayable environment and a verifiable outcome.
  • Keywords: browserbase, browser agents, auto-research, hill-climbing, automation
  • Core insight: Reliability in browser agents is achieved not by fine-tuning models, but by engineering self-improving doer-teacher loops that accumulate execution heuristics into reusable production playbooks.

Core insights

6
Architecturemedium noveltymoderate evidence

A two-agent doer-teacher loop converts execution traces into a persistent strategy.md artifact that accumulates learnings across iterations, then graduates to a skill.md playbook once a run passes — improving agent reliability without touching model weights.

Why it matters

Locates the improvement surface in an external, versionable text artifact rather than in fine-tuning or ad-hoc prompt edits, meaning the loop, the artifact, and the model can evolve independently and the artifact can be injected into any agent's system prompt.

Generalization

Any agent operating a live, stateful system can be hill-climbed by separating execution (doer) from after-the-fact trace analysis (teacher) and persisting the diff into a durable artifact.

The auto-research loop uses two agents: a doer that browses and a teacher that analyzes traces.
Open source video
A strategy.md file serves as the persistent artifact across runs, accumulating learnings.
Open source video
the teacher evaluates traces and screenshots to fix strategy.md. When a run passes, the skill graduates to skill.md.
Open source video
Mental Modelhigh noveltymoderate evidence

Self-improving loops only work when two preconditions hold: the environment is replayable and the outcome is verifiable. Without both, the agent cannot reliably judge whether an iteration succeeded.

Why it matters

Turns 'should I build a hill-climbing loop?' into a testable environment-design question — teams should invest in replayability and outcome verification before investing in the loop itself.

Generalization

The replayable + verifiable criterion predicts where self-improvement will work (coding agents with deterministic tests) and where it will stall (open-ended research agents).

Applying the loop requires two conditions: a replayable environment and a verifiable outcome.
Open source video
Without determinism and verification, agents cannot reliably evaluate whether a strategy iteration succeeded.
Open source video
Replayable environments and verifiable metrics enable cheap, compounding improvement.
Open source video
Mechanismmedium noveltymoderate evidence

The doer-teacher loop autonomously discovers counterintuitive strategy heuristics — e.g., preferring pre-filled URLs and JS injection over clicking calendar UI, and avoiding homepage navigation in favor of direct URLs with lat-long metro pins.

Why it matters

Shows the loop surfaces operational tactics a human engineer would likely not hand-author, shifting authoring effort from script-writing to environment/verifier construction.

Generalization

Automated strategy search can find path-of-least-resistance routes through brittle UI/anti-bot surfaces, but those heuristics are site-state-specific and may decay when the target changes.

Iterating on Google Flights search to discover that pre-filled URLs and JS injection are superior to calendar clicking.
Open source video
Agents should use pre-filled URLs and query parameters instead of complex manual UI interactions.
Open source video
Avoid navigating via homepage roots; use direct navigation URLs with lat-long metro pins.
Open source video
Practicemedium noveltymoderate evidence

Skills become a distributable, cache-like artifact: browse.sh hosts 400+ ready-made skills, supports prompt-to-skill generation, and its suggested selectors materially reduce token cost — skill.md acts as portable 'muscle memory for the web' injected into any agent.

Why it matters

If per-site reliability knowledge is shareable and reusable, the unit of engineering shifts from per-project scraping scripts to a catalog of versioned skills consumed across agents, with token cost as a first-class economic lever.

Generalization

Skill registries could emerge as a standard layer for agent reliability, analogous to package registries for code, with token-cost reduction as the adoption driver.

Browse.sh provides over 400 ready-made skills that can be imported into any agent.
Open source video
Users can submit a prompt to automatically build a skill for a site.
Open source video
Suggested selectors cut token costs significantly.
Open source video
Mental Modelmedium noveltymoderate evidence

Reliability is framed as a harness engineering problem, not a model capability ceiling — because every run starts from zero and no mechanism persists learnings from production traces.

Why it matters

Redirects investment from waiting for better foundation models toward building trace-capture, persistence, and replay infrastructure around existing models.

Generalization

Any agent working a live system re-learns from zero unless equipped with a train-skill loop, so harness investment compounds while model upgrades alone do not.

Reliability is an engineering problem rather than solely a model capability problem.
Open source video
Every run starts from zero without built-in learning from past execution traces.
Open source video
Developers must build harness infrastructure around models to capture and persist learnings.
Open source video
Empirical Resultmedium noveltyweak evidence

Demonstrated empirical curve: across auto-research iterations cost and latency decrease while success rate increases, suggesting reliability gains can be compounding rather than one-off prompt tuning.

Why it matters

Gives a measurable optimization target (success rate up, token cost and latency down per iteration) that can be instrumented as a benchmark for any self-improving loop.

Generalization

If the loop reliably improves these three metrics jointly, teams can A/B the loop itself rather than individual prompts.

Live demo showing cost and latency decreasing across iterations while success rate increases.
Open source video
Iterating on Google Flights search to discover that pre-filled URLs and JS injection are superior to calendar clicking.
Open source video

Deep dives

5

Convergence control in doer-teacher hill-climbing loops

Research question

What early-stopping and iteration-budget heuristics maximize success-rate delta per iteration before a strategy.md graduates to skill.md, and at what iteration count does marginal gain collapse?

Why

The loop's economics hinge on how many doer/teacher cycles are needed before a playbook is production-ready; if convergence is uncontrolled, teacher calls and trace storage may cost more than the reliability gain, making the whole approach uneconomic for long-tail sites.

Live demo showing cost and latency decreasing across iterations while success rate increases.
Open source video
How to optimize for convergence to reduce iteration
Open source video
Source video

Replay substrates for live, anti-bot web environments

Research question

Can recorded-trace replay, DOM snapshot fixtures, or request-level mocking reproduce a live site's failure surface (sliders, captchas, anti-bot walls) faithfully enough for the teacher agent to evaluate a strategy iteration offline?

Why

Replayability is one of the two stated preconditions for the loop, yet browser targets are the least deterministic environments available; without a replay substrate, hill-climbing must run against live sites at full cost and cannot isolate regressions from site-side drift.

Applying the loop requires two conditions: a replayable environment and a verifiable outcome.
Open source video
A browser agent failing to drag the time filter slider on Google Flights.
Open source video
An agent encountering an 'Access Denied' anti-bot page on OpenTable.
Open source video
Source video

Proxy verifiers for non-deterministic agent domains

Research question

Do proxy verifiers (source-grounding checks, rubric grading, selector-level assertions) correlate strongly enough with true task success to serve as the reward signal for open-ended research agents?

Why

The replayable+verifiable criterion predicts self-improvement stalls in open-ended domains; if a cheap proxy verifier can be shown to track ground truth, the addressable surface for hill-climbing loops expands well beyond deterministic coding tasks.

Without determinism and verification, agents cannot reliably evaluate whether a strategy iteration succeeded.
Open source video
Replayable environments and verifiable metrics enable cheap, compounding improvement.
Open source video
Source video

Skill decay detection and automatic re-entry into the loop

Research question

Can runtime success-rate monitoring reliably detect when a graduated skill.md has gone stale, and does auto-demotion plus re-entry into the doer-teacher loop recover reliability faster than a full re-derivation?

Why

Discovered heuristics are site-state-specific, so a catalog of graduated skills silently rots; without a drift trigger, skill reuse becomes a liability that degrades agents rather than improving them.

Agents should use pre-filled URLs and query parameters instead of complex manual UI interactions.
Open source video
Avoid navigating via homepage roots; use direct navigation URLs with lat-long metro pins.
Open source video
Source video

Teacher update rule: diff vs rewrite vs append on strategy.md

Research question

Which update semantics (incremental diff, full rewrite, append-only log) on strategy.md produce stable convergence without context bloat or loss of earlier-learned heuristics?

Why

The artifact is injected into the system prompt, so its growth directly trades against context budget and token cost; the update rule is a concrete, testable engineering choice that determines whether skill.md stays small and portable.

A strategy.md file serves as the persistent artifact across runs, accumulating learnings.
Open source video
Suggested selectors cut token costs significantly.
Open source video
Source video

Article ideas

3

Reliability Is a Harness Problem, Not a Model Capability Problem

Teams waiting on stronger foundation models are optimizing the wrong variable: because every browser agent run starts from zero with no persistence of execution traces, harness investment (trace capture, replay, and a train-skill loop) compounds run over run while model upgrades alone reset to the same baseline.

Angle

Reframe the reliability debate from a model-capability question to an infrastructure question, using the doer-teacher loop as the worked example and the cost/latency/success-rate curve as the argument for where the next engineering dollar goes.

Source video

The Two Preconditions: Where Self-Improving Agents Work and Where They Stall

A self-improving loop is only as good as its environment, and the replayable-plus-verifiable test cleanly partitions the agent landscape: browser and coding agents with deterministic tests can hill-climb cheaply, while open-ended research agents cannot judge their own iterations and will silently optimize noise.

Angle

Turn an architectural observation into a decision checklist teams can apply before building any self-improvement loop, with the open-ended-research case as the falsification test.

Source video

Skills Are a Package Registry: The Coming Supply Chain of Agent Reliability

Once per-site reliability knowledge is shareable as skill.md artifacts, the unit of engineering shifts from per-project scraping scripts to a versioned catalog, and the token-cost savings from suggested selectors will drive adoption faster than any correctness guarantee — importing third-party skills therefore propagates unknown, unvalidated heuristics into production agents.

Angle

Argue that the registry layer needs versioning, validation against current site state, and provenance metadata before adoption outpaces the ability to audit what a skill actually does.

Source video

Project ideas

4

ConvergenceBench

beyond-evals

Marginal success-rate gain per doer-teacher iteration collapses sharply after a small number of cycles, and an early-stopping rule keyed to strategy.md change-magnitude will reach a production-grade skill.md in far fewer iterations than a fixed-budget loop, at no loss of final success rate.

Proof of concept

Run the same task set (e.g., flight search, restaurant booking) through the loop with per-iteration instrumentation of success rate, token cost, latency, and a structural diff size on strategy.md; A/B a fixed 20-iteration budget against a diff-magnitude early-stopping rule.

Measurement

Iterations-to-graduation, final success rate, and total token cost per task; success criterion is equal-or-better success rate at a materially lower iteration count and lower cumulative cost.

Source video

Replay Fixture Harness

movement-lab

DOM snapshot fixtures plus request-level mocking can reproduce the specific failure surfaces that make browser agents unreliable (slider drag, anti-bot page) faithfully enough that a teacher agent's strategy verdict on replay matches its verdict on the live site in the majority of cases.

Proof of concept

Capture traces and DOM snapshots from real failing runs on two sites, build a replay substrate, then run the teacher agent on both live and replayed traces and compare the resulting strategy.md edits and pass/fail verdicts.

Measurement

Agreement rate between live and replayed verdicts, replay fidelity per failure-mode class, and cost reduction per evaluated iteration versus live execution.

Source video

Skill Drift Sentinel

gatehouse

Runtime success-rate monitoring on a graduated skill.md can detect staleness before agent reliability visibly degrades in production, and auto-demotion plus re-entry into the doer-teacher loop recovers reliability faster and cheaper than full re-derivation from scratch.

Proof of concept

Deploy graduated skills against a set of sites, inject controlled UI/DOM mutations, and compare three responses: no action, full re-derivation, and drift-triggered re-entry with the existing skill.md as the teacher's starting point.

Measurement

Detection latency (runs after drift before alert), recovery time and token cost per recovery path, and post-recovery success rate relative to pre-drift baseline.

Source video

Skill Registry Auditor

new

Third-party skills can be statically decomposed into their selector and navigation heuristics, and skills whose selectors no longer resolve against current site state measurably inflate token cost and lower success rate when imported — so validation gates plus selector-cost scoring will predict skill quality without executing them.

Proof of concept

Import a sample of catalog skills into a fixed agent, measure token cost and success rate against a selector-free baseline, and correlate those outcomes with a static audit score (selector resolution, staleness, instruction length).

Measurement

Token reduction and success-rate delta per imported skill, plus rank correlation between the static audit score and measured performance; a strong correlation validates pre-execution quality screening.

Source video

Architectural implications

3

Learning state lives in a text artifact (strategy.md → skill.md) injected into the system prompt, not in model weights.

Before

Improvement required fine-tuning the model or hand-editing prompts per task/site.

After

The improvement loop writes and revises a versioned playbook that any agent can load at runtime.

Consequence

Decouples model choice from reliability tuning, enables skill sharing (browse.sh) and per-site versioning, but makes prompt-context budget and artifact staleness primary concerns.

Source video

A dedicated teacher agent consumes traces and screenshots separately from the executing doer agent.

Before

A single agent both executed and self-corrected within a run, with no cross-run memory.

After

Execution and post-hoc analysis are split into distinct roles with the trace as the handoff interface.

Consequence

Introduces a clean observability boundary (traces/screenshots) and a natural place to add reward/eval logic, at the cost of extra agent calls and trace storage.

Source video

Replayability and verifiability are treated as environment properties rather than agent capabilities.

Before

Reliability debugging happened ad hoc against live production sites.

After

Environments must be provisioned to be replayable with verifiable outcomes before the loop is viable.

Consequence

Shifts engineering effort upstream into environment/harness construction and implies that domains lacking determinism (open-ended research) cannot use the loop as-is.

Source video

Tradeoffs and failure modes

4

Self-improving loop vs precondition costs

Benefit

Cheap, compounding improvement without fine-tuning and reduced manual script-writing.

Cost or risk

Requires a replayable environment AND a verifiable outcome; domains missing either (open-ended research) cannot reliably evaluate iterations.

Without determinism and verification, agents cannot reliably evaluate whether a strategy iteration succeeded.
Open source video
Source video

Discovered heuristics vs generality

Benefit

Loop finds superior tactics like pre-filled URLs and JS injection.

Cost or risk

These are site-state-specific; browser fragility and anti-bot/captcha failure modes can invalidate a graduated skill when the target site changes.

A browser agent failing to drag the time filter slider on Google Flights.
Open source video
Source video

Skill sharing/token savings vs correctness risk

Benefit

400+ ready-made skills and suggested selectors cut token costs significantly.

Cost or risk

Importing third-party skills into any agent propagates whatever heuristics (and staleness) those skills encode, creating supply-chain-like trust concerns.

Browse.sh provides over 400 ready-made skills that can be imported into any agent.
Open source video
Source video

Automated skill generation vs convergence

Benefit

Reduces engineering hours spent manually writing and updating scripts.

Cost or risk

Loop may require many iterations and it is unresolved how to optimize convergence, so iteration cost could dominate before a skill graduates.

How to optimize for convergence to reduce iteration
Open source video
Source video

Open questions

4

How can convergence be optimized to reduce the number of iterations before a skill graduates?

Why unresolved

The summary lists this explicitly as an open question and provides no convergence criterion or iteration budget.

Research direction

Instrument iteration count vs success-rate delta and test early-stopping heuristics on strategy.md changes.

Source video

How do you make inherently non-deterministic live websites replayable for the loop?

Why unresolved

The loop is stated to require a replayable environment, but the summary offers no mechanism for replaying a live site with anti-bot and captcha behavior.

Research direction

Explore recorded-trace replay, DOM snapshot fixtures, and request-level mocking as replay substrates.

Source video

How is a 'verifiable outcome' defined for domains like open-ended research agents?

Why unresolved

The summary contrasts coding agents with deterministic testing against open-ended research agents but does not resolve how verification is achieved for the latter.

Research direction

Design proxy verifiers (source-grounding checks, rubric grading) and measure their correlation with true task success.

Source video

How quickly do graduated skills decay when the target site changes, and can the loop detect and re-trigger?

Why unresolved

Skills are site-specific heuristics; the summary shows anti-bot/UI fragility but no drift-detection or re-learning trigger.

Research direction

Add runtime success monitoring to auto-demote skills and re-enter the doer-teacher loop on failure-rate spikes.

Source video

Key claims

8
opinionVerification needed

Reliability of browser agents is an engineering problem rather than solely a model capability problem.

Evidence

Reliability is an engineering problem rather than solely a model capability problem.

Question

Do harness/loop improvements outperform model upgrades on the same browser task benchmark?

Source video
factualVerification needed

Every agent execution starts from scratch with no built-in mechanism to learn from past traces.

Evidence

Every run starts from zero without built-in learning from past execution traces.

Question

Does the underlying model/provider retain any cross-session learning for the same task today?

Source video
factualVerification needed

The auto-research loop uses a doer agent that browses and a teacher agent that analyzes traces, updating a persistent strategy.md.

Evidence

The auto-research loop uses two agents: a doer that browses and a teacher that analyzes traces.

Question

What is the exact update rule the teacher applies to strategy.md (diff, rewrite, or append)?

Source video
causalVerification needed

The converged loop outputs a production playbook (skill.md) of exact workflows and heuristics.

Evidence

The output of the converged loop is a production playbook (skill.md) containing exact workflows and heuristics.

Question

What defines convergence and how many iterations does it typically take?

Source video
causalVerification needed

Applying the loop requires both a replayable environment and a verifiable outcome.

Evidence

Applying the loop requires two conditions: a replayable environment and a verifiable outcome.

Question

Do loops with only one of the two conditions measurably fail to improve?

Source video
causalVerification needed

Across iterations cost and latency decreased while success rate increased.

Evidence

Live demo showing cost and latency decreasing across iterations while success rate increases.

Question

What were the per-iteration numbers, task set size, and variance across tasks?

Source video
comparativeVerification needed

Suggested selectors from browse.sh cut token costs significantly.

Evidence

Suggested selectors cut token costs significantly.

Question

What is the measured token reduction vs baseline selector-free browsing?

Source video
factualVerification needed

Browse.sh provides over 400 ready-made skills importable into any agent.

Evidence

Browse.sh provides over 400 ready-made skills that can be imported into any agent.

Question

Are these skills versioned and validated against current site states?

Source video

Connections

4