Agent Frameworks Considered Harmful — Rémi Louf, .txt

Overview

• Speaker: Rémi Louf
• Channel: AI Engineer
• Main topic: The architectural limitations of agent frameworks and why event-driven, kernel-like primitives without graph abstractions are superior for reliable agent engineering.
• Purpose: To critique existing agent frameworks and propose an event-driven, OS-inspired runtime architecture for building reliable, auditable AI agents. Rémi Louf, CEO of .txt, shares insights from building infrastructure for reliable AI agents. He argues that traditional agent frameworks and graph-based abstractions are cumbersome, forcing developers to fiddle with YAML inside code and trapping them in chat-box paradigms. Instead, he advocates for an operating system mindset: treating agents as independent processes, using immutable event logs as system memory, and relying on content-addressed artifacts and typed tool calls to make bad actions impossible.

Topic Map

The Problem with Traditional Agent Frameworks

• Explanation: Agent frameworks often force developers to write YAML frontmatter inside code files and manage complex execution graphs, which leads to maintenance overhead and brittle prompts.
• Key claims:
  • Frameworks spend too much time editing prompts within code.
  • YAML frontmatter inside code is cumbersome and hard to version.
  • Graph-based abstractions add unnecessary complexity.
• Examples:
  • Editing system prompts embedded directly in Python or TypeScript files.
• Terminology:
  • agent frameworks
  • YAML frontmatter
  • execution graphs
• Why it matters: Understanding why code-centric agent orchestration fails helps developers adopt cleaner architectures.

Event-Driven Agent Architecture

• Explanation: Moving away from cron jobs and graphs toward an event log where agents subscribe to specific events, enabling fan-in and fan-out without managing edges.
• Key claims:
  • Cron jobs are too rigid because they only cover fixed points in time.
  • Agents should subscribe to typed events emitted by the system.
  • No explicit graph topology is needed; just drop a file or emit an event and the topology emerges.
• Examples:
  • A voice note recorded while walking emits a voice-note.ready event, triggering a voice note processor agent.
• Terminology:
  • event log
  • cron jobs
  • fan-in
  • fan-out
  • typed events
• Why it matters: Event-driven systems provide clean decoupling, easy debugging, and natural asynchronous processing for AI workflows.

The Runtime and Content-Addressed Memory

• Explanation: Treating the immutable log as the system memory, using content-addressing (like git or nix) for prompts, system prompts, skills, and model answers.
• Key claims:
  • The log is the system's memory, ensuring nothing is lost and everything is queryable.
  • Content-addressing makes caching, diffing, and replays trivial.
  • The chat box is a lie because what you see in UI chats does not match the actual hidden prompt state sent to models.
• Examples:
  • Using sha256 hashes to uniquely identify system prompts, tool descriptors, user messages, and model responses.
• Terminology:
  • content-addressed artifacts
  • immutable log
  • replays
  • diffs
• Why it matters: Content-addressing and immutable logs enable complete auditability and deterministic debugging of agent failures.

Kernel-Like Agent Systems and Safety

• Explanation: Thinking of agent runtimes like operating system kernels: their job is to make bad actions impossible, not just unlikely.
• Key claims:
  • A kernel's job is to make bad actions impossible.
  • Typed tool calls and typed events are non-negotiable for robust agents.
  • Open-source models are mature enough to replace proprietary APIs for local development and production.
• Examples:
  • Using strict structured outputs to eliminate 20% failure rates from malformed LLM responses.
• Terminology:
  • kernel
  • typed tool calls
  • structured outputs
  • isolation
• Why it matters: Enforcing strict type boundaries at the kernel level prevents cascading failures in multi-agent systems.

Key Points

Agent frameworks vs. operating systems

• Explanation: Instead of using heavy libraries with graph abstractions, treat agents as independent processes run by a kernel that isolates them and journals their actions.
• Evidence: Deploying ~20 agents successfully within a company using a simple file-drop and event-log architecture.
• Practical implication: Developers should build lightweight event runtimes instead of adopting complex orchestration frameworks.

The chat box is a lie

• Explanation: UI chat interfaces hide the true prompt state, compaction steps, and model internals, making debugging difficult.
• Evidence: Model providers do not share thinking traces or exact prompt assembly history.
• Practical implication: Systems must log exact content-addressed prompts and responses to ensure full auditability.

Structured outputs are non-negotiable

• Explanation: Without strict type enforcement on tool calls and events, LLM outputs fail frequently and break agent pipelines.
• Evidence: Using .txt's internal tooling where untyped outputs caused high failure rates.
• Practical implication: Always enforce typed tool calls and typed events between agents.

Frameworks, Models & Processes

Zeta Event-Driven Agent Runtime

• How it works: Agents are defined as markdown files with YAML frontmatter and a prompt body. They register interest in specific event types, read from an immutable event log, and emit new events upon completion.
• Components:
  • agent markdown definition
  • immutable event log
  • typed event queue
  • content-addressed storage
  • replay engine
• When to use: When building multi-agent background automation pipelines that require high reliability and auditability.

Examples & Case Studies

Rémi built a morning briefing workflow where a voice note recorded on a walk is transcribed and summarized into a Slack message via background agents.

• Illustrates: How event-driven agent pipelines replace manual tool triggering and cron jobs.
• Lesson: Background agents working asynchronously via events feel magical and require zero manual intervention.

Initial cron-based implementations resulted in posted duplicates, vanished voice notes, and corrupted market briefs due to untracked prompt changes.

• Illustrates: The brittleness of naive agent scripting without proper runtime guarantees.
• Lesson: Failures in agent systems force the creation of robust runtimes, immutable logs, and content-addressed diffs.

Actionable Takeaways

• Immediate:
  • Stop relying solely on heavy third-party agent frameworks.
  • Implement typed tool calls and structured outputs for all LLM interactions.
  • Use content-addressing for prompts and agent configurations to enable easy diffing and replays.
• Strategic:
  • Think kernel, not libraries: design agent systems around process isolation and event logging.
  • Build before you buy while the infrastructure category remains unsettled.
  • Eat your own dog food by deploying agents internally for real workflows.
• Questions to investigate:
  • How do content-addressed prompt systems scale with very large context windows?
  • What are the latency overheads of immutable event logging in high-frequency agent systems?

Claims Worth Verifying

• Structured outputs reduce agent event failure rates from 20% down to near zero. (empirical)
• Content-addressed prompt logging makes Kv cache management significantly easier. (architectural)

Notable Quotes

"Agents frameworks considered harmful." (at 0:25) "The chat box is a lie. What you see is not what the model sees." (at 18:36) "A kernel's job is to make bad actions impossible, not unlikely." (at 27:28)

Compressed Summary

• traditional agent frameworks and yaml-in-code are cumbersome and hard to maintain
• event-driven architectures with immutable logs eliminate the need for complex graph abstractions
• content-addressing prompts and agent states enables trivial diffing, caching, and replays
• kernel-like runtimes must enforce typed tool calls and structured outputs to prevent bad agent actions
• Keywords: agents, runtime, events, kernel, auditability
• Core insight: Reliable AI agents require operating system principles—immutable event logs, content-addressing, and typed boundaries—rather than complex graph frameworks.