AI Engineer · Published 2026-09-14

Agents Without Code: Skills, YAML, and Filesystems Replaced Python — Philipp Schmid, Google DeepMind

privateReview: unreviewedConfidence: high
agentsgeminiinteractions-apisandboxesmarkdownagent-orchestrationdeclarative-configcredential-isolation
Open on YouTube ↗

Summary

Overview

  • Speaker: Philipp Schmid
  • Channel: AI Engineer
  • Main topic: How markdown files, YAML, and filesystems replace traditional Python code in building LLM agents
  • Purpose: To demonstrate the shift in agent engineering from hardcoded Python orchestration loops to file-driven, framework-managed, and cloud-sandboxed managed agents. Philipp Schmid from Google DeepMind explores the evolution of building LLM agents, demonstrating how raw Python code loops, JSON schema definitions, and custom tooling are being replaced by declarative configuration, filesystem-based agent definitions (AGENTS.md, SKILL.md), and managed remote sandboxes via the Interactions API.

Topic Map

Introduction & Definition of an LLM Agent

  • Explanation: Philipp Schmid introduces the session as the final one before keynotes on day four, defining an LLM agent using Simon Willson's definition: an LLM agent runs tools in a loop to achieve a goal.
  • Key claims:
    • An LLM agent runs tools in a loop until it reaches a goal.
    • We are moving from raw Python code to file-driven agent definitions.
  • Examples:
    • Building a GitHub PR Review Agent in three different ways with decreasing code complexity.
  • Terminology:
    • LLM Agent
    • Tool Call Loop
  • Why it matters: Sets the foundational definition for what constitutes agent execution.

The Interactions API & Raw SDK Approach

  • Explanation: Introduction to the Gemini Interactions API as a unified interface for models and agents with server-side state management and background execution.
  • Key claims:
    • Interactions API provides a unified interface for model calls and agent interactions.
    • Traditional agent building in Python required manual JSON schema definitions, state tracking, and error handling loops.
  • Examples:
    • Building a GitHub PR reviewer using raw Python code, manual tool definitions, and API loops.
  • Terminology:
    • Interactions API
    • Turn-based vs Steps Data Model
    • Function Calling
  • Why it matters: Highlights the high boilerplate overhead of early agent engineering.

Agent Frameworks & The Rise of Agent Development Kits (ADK)

  • Explanation: Frameworks like ADK abstract away tool loops, function dispatching, and retries, allowing developers to focus on domain instructions and tool signatures.
  • Key claims:
    • Frameworks automate JSON schema generation from docstrings and handle execution loops.
    • Developers still have to own Python plumbing code for every tool and host runtime environments.
  • Examples:
    • Using Agent Development Kits to manage PR review agents without manual JSON schemas.
  • Terminology:
    • Agent Development Kit
    • Tool Routing
    • Function Signatures
  • Why it matters: Shows how frameworks reduced boilerplate but still left infrastructure and plumbing ownership with the developer.

Managed Agents & Remote Sandboxes (Antigravity)

  • Explanation: Google's Antigravity remote agent powered by the Gemini API uses AGENTS.md, SKILL.md files, and secure isolated remote Linux sandboxes.
  • Key claims:
    • Agents are just files: Markdown files define system instructions and capabilities.
    • Remote sandboxes provide isolated execution environments with network proxy credential security.
  • Examples:
    • Running a managed agent using only AGENTS.md and SKILL.md files inside a remote cloud sandbox.
  • Terminology:
    • Managed Agents
    • AGENTS.md
    • SKILL.md
    • Remote Sandbox
    • Interactions API
  • Why it matters: Represents the modern paradigm where files replace Python code for agent orchestration.

Key Points

The Bitter Lesson of Agent Engineering

  • Explanation: General computation beats hard-coded knowledge as model capabilities improve.
  • Evidence: Cursor removed 12,000 lines of TypeScript code and replaced it with a 200-line Markdown skill file; Manus and Vercel refactored their harnesses significantly.
  • Practical implication: Stop over-engineering agent orchestration harnesses as foundation models become more capable.

Agents Are Just Files

  • Explanation: Lifecycle management and deployment of agents can be simplified using standard filesystems and markdown files.
  • Evidence: Using AGENTS.md for instructions and SKILL.md for tools and context.
  • Practical implication:

Frameworks, Models & Processes

Interactions API

  • How it works: Provides server-side state management, native tool use, multimodal input/image generation, and background execution via a unified client interface.
  • Components:
    • Unified interface for models and agents
    • Server-side state management
    • Background execution for long-running workflows
  • When to use: When building scalable agent applications without managing custom state loops.

Managed Agents Architecture

  • How it works: Decouples execution sandboxes configured dynamically or saved by ID, utilizing markdown files for instructions and skills.
  • Components:
    • AGENTS.md
    • SKILL.md
    • Remote Linux Sandbox
    • Network Proxy Credential Injection
  • When to use: When deploying general-purpose agents into secure cloud environments.

Examples & Case Studies

Cursor replaced 12,000-15,000 lines of TypeScript code with a 200-line Markdown skill file.

  • Illustrates: The bitter lesson of agent engineering where better models reduce the need for hardcoded orchestration code.
  • Lesson: Build to delete code as model capabilities increase.

Actionable Takeaways

  • Immediate:
    • Visit Google AI Studio (ai.studio) to get an API key.
    • Start building agents with files and skills instead of Python loops.
  • Strategic:
    • Do not fight the model; provide general tools and let models reason and discover solutions.
    • Own what is yours: focus on domain instructions, workflows, and evaluations.
  • Questions to investigate:
    • How do sandbox network proxies securely handle credential injection without exposing secrets to the agent?

Claims Worth Verifying

  • Cursor replaced 12,000 to 15,000 lines of TypeScript with 200 lines of Markdown skill file. (empirical)

Notable Quotes

"An LLM agent runs tools in a loop to achieve a goal."

Compressed Summary

  • Agents are shifting from Python code loops to filesystem-based Markdown files (AGENTS.md, SKILL.md).
  • The Gemini Interactions API provides native tool use, server-side state management, and background execution.
  • Managed remote sandboxes provide secure, isolated execution environments with network proxy credential injection.
  • The bitter lesson of agent engineering proves that as models improve, hardcoded orchestration code should be deleted.
  • Keywords: agents, gemini, interactions-api, sandboxes, markdown
  • Core insight: As foundation models improve, hardcoded Python agent orchestration loops and tool plumbing are being replaced by declarative filesystems, Markdown instructions, and managed remote sandboxes.

Core insights

7
Mental Modelhigh noveltymoderate evidence

The 'bitter lesson' applied to agent harnesses: as foundation models improve, hard-coded orchestration should be deleted rather than maintained. Cursor replaced 12,000-15,000 lines of TypeScript with a 200-line Markdown skill file, and Manus/Vercel refactored their harnesses.

Why it matters

It reframes harness engineering as a depreciating asset: elaborate tool-routing and retry loops are likely to be obsoleted by model capability, so minimizing harness code reduces future rewrite cost and lets the model's own reasoning handle orchestration.

Generalization

Design agent infrastructure so capability from the model can substitute for your code; treat orchestration code as something to remove over time, not accumulate.

Cursor removed 12,000 lines of TypeScript code and replaced it with a 200-line Markdown skill file
Open source video
Stop over-engineering agent orchestration harnesses as foundation models become more capable.
Open source video
Build to delete code as model capabilities increase.
Open source video
Architecturehigh noveltymoderate evidence

Agent lifecycle and definitions move into the filesystem: AGENTS.md carries system instructions and SKILL.md carries tools and context, so an agent becomes a set of ordinary files rather than Python code.

Why it matters

Instruction, capability, and deployment surface collapse into versionable, diffable, portable files, which changes how agents are authored, reviewed, shipped, and shared — closer to config-as-code than application code.

Generalization

When the model can interpret prose, declarative artifacts (markdown/YAML) can replace imperative glue code for defining behavior and capabilities.

Agents are just files: Markdown files define system instructions and capabilities.
Open source video
Using AGENTS.md for instructions and SKILL.md for tools and context.
Open source video
Mechanismmedium noveltymoderate evidence

The Interactions API moves state management and long-running execution server-side, offering a unified interface for both model calls and agents with background execution, replacing client-side manual state tracking and error-handling loops.

Why it matters

Offloading the state/turn loop to the service removes a whole class of client plumbing (state serialization, retries, loop control) and enables long-running workflows that outlive a single request, changing where reliability and orchestration logic live.

Generalization

Hosted agent runtimes shift orchestration responsibility from application code to the platform, trading control for reduced boilerplate.

Unified interface for models and agents
Open source video
Server-side state management
Open source video
Background execution for long-running workflows
Open source video
Architecturemedium noveltyweak evidence

Managed remote Linux sandboxes with network proxy credential injection run agents in isolated environments where secrets are injected at the network proxy rather than exposed to the agent process.

Why it matters

It creates a security boundary for untrusted, model-driven code execution: the agent can call authenticated services without holding credentials, which is a prerequisite for letting agents autonomously run code and reach external APIs.

Generalization

Credential isolation via an out-of-band proxy is a reusable pattern for any system executing model-generated or untrusted actions.

Remote sandboxes provide isolated execution environments with network proxy credential security.
Open source video
Network Proxy Credential Injection
Open source video
Tradeoffmedium noveltymoderate evidence

Frameworks such as ADK auto-generate JSON schemas from docstrings and absorb tool loops, dispatching, and retries — but developers still own per-tool plumbing and the runtime environment.

Why it matters

The abstraction boundary is asymmetric: control-flow boilerplate is removed, but the tool implementations and hosting remain the developer's responsibility, meaning framework adoption does not eliminate infrastructure ownership.

Generalization

Each layer of agent abstraction removes one category of code (schema, loop, dispatch) while preserving others (tool bodies, runtime), so teams should plan for residual ownership explicitly.

Frameworks automate JSON schema generation from docstrings and handle execution loops.
Open source video
Developers still have to own Python plumbing code for every tool and host runtime environments.
Open source video
Empirical Resultmedium noveltyweak evidence

The same PR-review agent can be built three ways with decreasing code complexity (raw SDK loop, ADK framework, managed agent with files), framing agent evolution as a monotonic step-function of code removal.

Why it matters

Gives a concrete axis for evaluating agent stacks: how much code and infrastructure the developer must own versus how much the model and platform absorb, which is a more useful comparison than feature lists.

Generalization

Comparative implementation of one task across abstraction tiers is a good method for exposing hidden ownership costs in agent tooling.

Building a GitHub PR Review Agent in three different ways with decreasing code complexity.
Open source video
Practicelow noveltyweak evidence

Strategic guidance is to give models general tools and let them reason and discover solutions rather than fight the model with narrow, hardcoded workflows; teams should own domain instructions, workflows, and evaluations.

Why it matters

It redefines the engineer's value-add as domain framing and evaluation rather than tool plumbing, which changes what gets invested in and measured.

Generalization

As models absorb mechanics, the durable engineering surface is problem specification, domain knowledge, and evaluation harnesses.

Do not fight the model; provide general tools and let models reason and discover solutions.
Open source video
Own what is yours: focus on domain instructions, workflows, and evaluations.
Open source video

Deep dives

5

Credential isolation in sandbox network proxies for autonomous agents

Research question

How can a sandbox network proxy inject scoped credentials for authenticated service calls without exposing secrets to the agent process or allowing credential exfiltration through proxied requests?

Why

Credential isolation is the enabling security boundary for letting model-driven agents execute code and reach external APIs. Without a concrete mechanism, autonomous tool use in managed sandboxes remains unsafe or limited.

Remote sandboxes provide isolated execution environments with network proxy credential security.
Open source video
Network Proxy Credential Injection
Open source video
Source video

What is lost when agents are defined purely as markdown files

Research question

What concrete engineering properties are lost when AGENTS.md and SKILL.md files replace imperative agent code, and which failures become harder to diagnose, test, or prevent?

Why

File-based agent definitions improve reviewability and portability, but if they cannot express deterministic constraints or type contracts, teams may trade boilerplate for hidden failure modes.

Agents are just files: Markdown files define system instructions and capabilities.
Open source video
Using AGENTS.md for instructions and SKILL.md for tools and context.
Open source video
Source video

Verifying harness deletion without capability loss

Research question

Can large hardcoded agent orchestration harnesses be replaced by markdown skill definitions while preserving task success, latency, cost, and safety characteristics?

Why

The bitter-lesson argument for agent engineering depends on whether deleting orchestration code actually preserves reliability. If it only shifts failure modes, the recommended direction is incomplete.

Stop over-engineering agent orchestration harnesses as foundation models become more capable.
Open source video
Build to delete code as model capabilities increase.
Open source video
Source video

Runtime state semantics for hosted agent APIs

Research question

What state granularity, resumption behavior, and observability are provided by turn-based versus steps data models in hosted agent runtimes, and which long-running workflows require which model?

Why

Moving state and execution server-side removes client plumbing but introduces platform semantics as a dependency. Choosing the wrong data model can affect durability, debugging, and workflow resumption.

Server-side state management
Open source video
Background execution for long-running workflows
Open source video
Turn-based vs Steps Data Model
Open source video
Source video

Residual ownership after agent framework abstraction

Research question

After adopting frameworks like ADK, what per-tool and runtime responsibilities remain with the developer, and how should teams budget for that residual infrastructure ownership?

Why

Framework adoption is often framed as eliminating boilerplate, but asymmetric abstraction can leave tool bodies and hosting behind. Teams need explicit cost models before standardizing on a framework.

Frameworks automate JSON schema generation from docstrings and handle execution loops.
Open source video
Developers still have to own Python plumbing code for every tool and host runtime environments.
Open source video
Source video

Article ideas

4

Your Agent Harness Is Technical Debt: Build to Delete It

Hardcoded agent orchestration should be treated as a depreciating asset. As foundation models improve, teams should measure harness code by how little of it remains and design for deletion rather than accumulation.

Angle

Reframe agent engineering as continual subtraction: identify orchestration code that model capability can absorb, and replace it with declarative files while instrumenting reliability to prove nothing important was lost.

Source video

Agents Are Just Files—And That's the Problem

File-driven agent definitions improve reviewability and portability, but they sacrifice determinism, type safety, testability, and failure diagnosis. Teams should use markdown for instructions and capabilities while keeping code for enforcing critical constraints.

Angle

Argue against the totalizing version of the 'agents are just files' claim by cataloging what markdown and YAML cannot express, and propose a hybrid model where files define intent and code enforces invariants.

Source video

The Abstraction Boundary: What Agent Frameworks Don't Remove

Agent development kits remove schema generation, tool loops, dispatching, and retries, but they leave per-tool implementation and runtime hosting with the developer. Adoption plans must budget for this residual ownership instead of assuming framework use eliminates infrastructure.

Angle

Compare raw SDK, ADK-style framework, and managed file-based agent implementations to expose the exact categories of code each tier does and does not remove.

Source video

Server-Side State Is Not Free: Turn-Based vs Steps Data Models

Offloading agent state and execution to a hosted API removes client-side serialization, retry, and loop control, but introduces platform-specific data models that determine resumption granularity, observability, and long-running workflow design.

Angle

Argue that teams should select a hosted agent runtime based on the data model's fit with their durability and debugging needs, not only on boilerplate reduction.

Source video

Project ideas

4

Proxy Credential Isolation Testbed

gatehouse

A sandbox network proxy can inject scoped credentials for agent-initiated authenticated requests such that no credential material is readable by the agent process and credential exfiltration through proxied requests is blocked.

Proof of concept

Run an agent in an isolated remote Linux sandbox behind a credential-injecting network proxy. The agent calls an authenticated API without receiving secrets. Attempt to read environment variables, process memory, and filesystem; attempt to exfiltrate secrets through proxied requests and local logging.

Measurement

Count secrets present in agent-readable surfaces; record unauthorized exfiltration attempts and whether they are blocked; measure successful authenticated API call rate and latency overhead introduced by the proxy.

Source video

Harness Deletion Benchmark

movement-lab

Replacing a hardcoded agent orchestration harness with declarative markdown skill definitions preserves task success and reduces maintained code by more than 80 percent without increasing failure rate or operational cost.

Proof of concept

Implement the same PR-review agent twice: once as a raw Python harness with explicit tool loops and JSON schemas, and once as AGENTS.md and SKILL.md files run in a managed runtime. Run both against a fixed set of historical pull requests.

Measurement

Lines of maintained code and configuration; task success rate on review benchmarks; latency; token cost; categories of failures and whether they are diagnosable from available logs.

Source video

File-Defined Agent Capability Auditor

beyond-evals

File-only agent definitions cannot express certain deterministic constraints such as typed tool contracts, retry budgets, and security policies, and these gaps predict specific failure modes under fault injection.

Proof of concept

Build a constraint coverage matrix for agent definitions across categories: instruction, capability, tool contract, retry policy, security boundary, and observability hook. Implement a file-driven agent and a code-driven agent, then inject faults such as malformed tool outputs, API timeouts, and prompt-injection attempts.

Measurement

Percentage of constraint categories expressible in files versus code; number and severity of injected faults that produce unhandled or undiagnosable failures; reproducibility of failure diagnosis from available artifacts.

Source video

Residual Ownership Instrumenter

new

After adopting an agent framework, teams still own measurable per-tool and runtime work, and instrumenting this residual reveals the true cost of abstraction tiers better than lines-of-code counts alone.

Proof of concept

Implement the same agent task three ways: raw SDK loop, framework-based agent, and managed file-driven agent. Track developer ownership across tool implementation, schema maintenance, runtime hosting, deployment, and operational response.

Measurement

Lines of code and configuration owned per tier; number of runtime resources hosted; setup and deployment time; number and type of operational incidents during a fixed test period.

Source video

Architectural implications

3

Agent state and execution loop move from application code to a hosted API with server-side state and background execution.

Before

Client-side Python loop tracks turns, serializes state, retries failures, and ties agent lifetime to the process.

After

Server holds state and runs long-lived workflows; client issues interactions and receives results.

Consequence

Reliability, retry semantics, and observability become platform concerns; applications gain durability but lose fine-grained control and gain a dependency on runtime semantics like turn-based vs steps data models.

Source video

Agent definitions become filesystem artifacts (AGENTS.md, SKILL.md) rather than application source.

Before

Agent behavior encoded in Python modules, JSON schemas, and bespoke tool code.

After

Behavior and capabilities declared in markdown files loaded by a managed runtime.

Consequence

Agents become reviewable, diffable, and portable artifacts; authorship shifts toward prompt/instruction engineering and away from programming, and tooling must support file-based agent lifecycle management.

Source video

Code execution and credential access are separated into an isolated sandbox plus an injecting network proxy.

Before

Agent process runs with ambient credentials or receives secrets directly.

After

Agent runs in an isolated remote Linux sandbox; the proxy injects credentials so the agent never holds secrets.

Consequence

Enables safer autonomous tool use and third-party access without secret exposure, but introduces a proxy as a critical path component that must itself be secured and audited.

Source video

Tradeoffs and failure modes

3

Framework abstraction vs residual ownership

Benefit

Frameworks like ADK remove boilerplate: automatic JSON schema generation from docstrings, managed tool loops, dispatching, and retries.

Cost or risk

Developers still must own Python plumbing for every tool and the runtime hosting environment, so adopting a framework does not remove infrastructure burden.

Developers still have to own Python plumbing code for every tool and host runtime environments.
Open source video
Source video

Minimizing harness code as models improve

Benefit

Deleting orchestration code (e.g., 12,000-15,000 lines of TypeScript replaced by a 200-line markdown skill) reduces maintenance and lets model capability drive behavior.

Cost or risk

Reducing hardcoded control flow may sacrifice determinism and explicit guardrails, making behavior depend on model reasoning quality.

Stop over-engineering agent orchestration harnesses as foundation models become more capable.
Open source video
Source video

Remote sandbox with proxy credential injection

Benefit

Provides isolated execution and lets agents use authenticated services without being given secrets.

Cost or risk

The mechanism for keeping secrets away from the agent while still authenticating its requests is not explained, leaving an unresolved security question.

How do sandbox network proxies securely handle credential injection without exposing secrets to the agent?
Open source video
Source video

Open questions

5

How does a sandbox network proxy inject credentials without exposing secrets to the agent?

Why unresolved

The summary states proxy-based credential security as a property but gives no mechanism for policy, scope, or leakage prevention.

Research direction

Threat-model sandbox network proxies: scope of injected credentials, request-level authorization, and whether an agent can exfiltrate via proxied requests.

Source video

What concretely is lost when agents are defined purely by markdown files instead of code?

Why unresolved

The summary asserts the file-based paradigm but lists only benefits and leaves the 'Practical implication' of 'Agents Are Just Files' incomplete.

Research direction

Compare file-driven versus code-driven agents on determinism, type safety, testability, versioning, and failure diagnosis.

Source video

How reproducible and general is the 12,000-line-to-200-line markdown reduction observed at Cursor?

Why unresolved

It is a single anecdotal empirical claim with a wide range (12,000-15,000 lines) and no measurement methodology.

Research direction

Audit before/after capability, latency, cost, and error rates to test whether capability was preserved while code shrank.

Source video

Which runtime semantics (turn-based vs steps data model) should applications depend on for server-side state?

Why unresolved

The summary names both data models but does not explain the distinction or when each applies.

Research direction

Prototype long-running and background agent workflows against both models and compare state granularity, resumption, and observability.

Source video

How much of the orchestration burden actually shifts to the platform versus the developer across abstraction tiers?

Why unresolved

The three-tier comparison of the same PR-review agent is described qualitatively as 'decreasing code complexity' with no measured breakdown.

Research direction

Instrument a single agent task built three ways and quantify lines of code, infra owned, and operational surface at each tier.

Source video

Key claims

8
factualVerification not requested

An LLM agent runs tools in a loop to achieve a goal.

Evidence

An LLM agent runs tools in a loop to achieve a goal.

Source video
factualVerification needed

Cursor replaced 12,000 to 15,000 lines of TypeScript with a 200-line Markdown skill file.

Evidence

Cursor removed 12,000 lines of TypeScript code and replaced it with a 200-line Markdown skill file

Question

Can this code reduction be independently verified, and were capabilities preserved at parity?

Source video
causalVerification needed

Reducing hardcoded orchestration as models improve is the correct engineering direction (the bitter lesson of agent engineering).

Evidence

Stop over-engineering agent orchestration harnesses as foundation models become more capable.

Question

Does reducing harness code preserve reliability and safety as model capability increases, or merely shift failure modes?

Source video
factualVerification needed

Managed agents use AGENTS.md and SKILL.md files inside remote sandboxes, meaning agents are defined by files rather than code.

Evidence

Agents are just files: Markdown files define system instructions and capabilities.

Question

Do file-only agent definitions cover all cases currently handled by imperative agent code?

Source video
factualVerification needed

The Interactions API provides server-side state management and background execution for long-running workflows via a unified interface.

Evidence

Server-side state management

Question

What are the durability, resumption, and cost characteristics of server-side state versus client-managed state?

Source video
comparativeVerification needed

Frameworks reduce boilerplate but developers still own Python plumbing and runtime hosting.

Evidence

Developers still have to own Python plumbing code for every tool and host runtime environments.

Question

How much per-tool and runtime work remains after framework adoption across real projects?

Source video
factualVerification needed

Remote sandboxes provide isolated execution with network proxy credential security.

Evidence

Remote sandboxes provide isolated execution environments with network proxy credential security.

Question

How are credentials scoped and prevented from leaking to the agent or through proxied requests?

Source video
opinionVerification needed

Engineers should provide general tools and let models reason and discover solutions rather than encoding narrow workflows.

Evidence

Do not fight the model; provide general tools and let models reason and discover solutions.

Question

Do general tools outperform task-specific tools on reliability and cost for well-scoped production tasks?

Source video

Connections

5