Brains vs Hands: How to Run AI Agents Safely in Production — Viren Baraiya

Overview

• Speaker: Viren Baraiya
• Channel: AI Engineer
• Main topic: Running AI agents reliably and safely in production using durable runtimes (harnesses) to handle deterministic and non-deterministic tasks.
• Purpose: To provide engineers with architectural patterns and runtime principles for orchestrating AI agents in production environments safely, reliably, and durably. In this talk, Viren Baraiya discusses the operational realities of running AI agents in production. He argues that building agents is not just about LLM chatbots or simple tool-calling loops, but requires a clear separation of responsibilities: the agent (the brain) plans and reasons non-deterministically, while the harness (the execution engine/application) handles deterministic execution, state, routing, and persistence. Using Conductor as an example of a durable workflow engine, Baraiya demonstrates how late-bound sagas enable agents to safely plan actions at runtime within a bounded, deterministic vocabulary.

Topic Map

The Expanding Scope of AI Agents

• Explanation: Agents are moving beyond basic chatbots into background workers, scheduled agents, event-driven listeners, long-running coordinators, and multi-agent systems where humans are rarely watching every step.
• Key claims:
  • Most real agents are not simple chatbots.
  • Production agents operate as background workers, scheduled tasks, or event-driven actors without constant human supervision.
• Examples:
  • An agent running every hour to check system schedules or listening to infrastructure alert logs.
• Terminology:
  • Background workers
  • Scheduled agents
  • Event-driven agents
  • Multi-agent systems
• Why it matters: Understanding production agents requires moving past the simple request-response chat interface paradigm.

Component vs Application: The Harness

• Explanation: An agent is a component, not a full application. Just like microservices are orchestrated by an application layer, agents must be wrapped in a harness (an orchestration layer) that handles state, routing, policy, and failure handling.
• Key claims:
  • You never ship a single microservice without an orchestration layer; the same applies to AI agents.
  • The harness is the application layer around agents where agent behavior becomes product behavior.
• Examples:
  • An SRE harness orchestrating multiple agents to monitor logs, query metrics, and trigger rollbacks.
• Terminology:
  • Harness
  • Orchestration layer
  • Single-service responsibility
• Why it matters: Without a harness, agents cannot reliably interact with external enterprise systems or databases.

Deterministic vs Non-Deterministic Work

• Explanation: Production workflows require combining non-deterministic LLM reasoning (planning, reasoning, classifying, summarizing) with deterministic operations (payments, provisioning, approvals, compliance checks, rollbacks) that must never be improvised.
• Key claims:
  • A harness runs more than just an agent loop; it combines non-deterministic planning with deterministic execution.
  • The agent has no business owning the deterministic half of the application.
• Examples:
  • Executing payments or rolling back a Kubernetes cluster deployment must be deterministic.
• Terminology:
  • Deterministic work
  • Non-deterministic work
  • Idempotency
  • LLM reasoning
• Why it matters: Allowing LLMs to improvise critical infrastructure or financial operations leads to hallucinations and catastrophic failures.

Durability Across Time

• Explanation: Harnesses run across time—seconds, days, or weeks—waiting on humans, reacting to events, firing on schedules, and resuming after crashes. Durability is a core requirement, not an optional feature.
• Key claims:
  • These applications don't run inside a request; they run for hours, days, or weeks.
  • Durability is the cost of admission for long-running agentic workflows.
• Examples:
  • Order management systems waiting on third-party shipments or human approvals.
• Terminology:
  • Durability
  • State management
  • Long-running processes
• Why it matters: Network failures, server restarts, and infrastructure partitions require systems to recover seamlessly without losing state.

Separation of Responsibilities: Brains and Hands

• Explanation: The agent plans (decides what to do, why, and how to adapt as the world changes), while the harness executes (decides what is allowed, what already happened, what can be retried, what needs approval, and what runs next).
• Key claims:
  • The agent plans. The harness executes.
  • The agent proposes; the harness operationalizes.
• Examples:
  • An agent identifies an unhealthy cluster; the harness executes the approved, deterministic restart procedure.
• Terminology:
  • Planning vs execution
  • Human-in-the-loop gates
  • Idempotent operations
• Why it matters: Clear boundaries prevent LLMs from executing dangerous unauthorized actions directly.

Late-Bound Sagas

• Explanation: Instead of pre-defining all workflow branches in a closed world, agentic workflows use late-bound sagas where the agent proposes and builds workflows at runtime from a pre-registered vocabulary of deterministic tools and operations defined by the harness.
• Key claims:
  • Bound late, over operations whose guarantees were bound early.
  • The agent owns the plan; the harness owns the guarantees.
• Examples:
  • A production remediation workflow where the agent decides at runtime to query metrics, trip circuit breakers, and notify teams.
• Terminology:
  • Late-bound sagas
  • Bounded vocabulary
  • DAG (Directed Acyclic Graph)
  • Closed world vs open world
• Why it matters: It combines the flexibility of runtime LLM planning with the safety and determinism of traditional enterprise workflow engines.

Key Points

An agent is a component, not an application

• Explanation: Just as microservices require an orchestration layer for routing, state, and failure handling, AI agents need a harness to coordinate successfully.
• Evidence: Comparison to microservice architecture where services communicate through choreography and orchestration.
• Practical implication: Do not build monolithic agent applications; encapsulate agents within structured runtime harnesses.

Deterministic execution must be separated from LLM reasoning

• Explanation: LLMs should handle reasoning, planning, and classification, while deterministic processes like payments and provisioning must be strictly controlled by code.
• Evidence: List of deterministic tasks (payments, provisioning, rollbacks) versus non-deterministic tasks (planning, reasoning, summarizing).
• Practical implication: Eliminate LLM hallucinations in critical operational paths by enforcing hardcoded deterministic procedures.

Durability is essential for long-running workflows

• Explanation: Agent harnesses must persist state across crashes, restarts, and extended timeframes ranging from seconds to weeks.
• Evidence: Examples of order management and long-running SRE monitoring tasks.
• Practical implication: Use durable workflow orchestration engines to back agentic pipelines.

Late-bound sagas solve the open-world planning problem

• Explanation: Allowing agents to select steps from a pre-registered vocabulary at runtime provides flexibility while maintaining strict execution guarantees.
• Evidence: Demonstration of runtime workflow compilation and execution DAGs in Conductor.
• Practical implication: Define strict tool vocabularies in your harness while letting agents compose them dynamically.

Frameworks, Models & Processes

The Agent Harness Framework

• How it works: Wraps non-deterministic agent loops inside a durable runtime that manages state, system tools, human-in-the-loop approvals, and deterministic execution steps.
• Components:
  • Agent loop (LLM planning)
  • Orchestration layer (harness)
  • Tools and MCP (Model Context Protocol)
  • Human in the loop gates
  • Enterprise systems and databases
• When to use: When deploying AI agents into production systems that require high reliability, safety, and auditability.

Late-Bound Sagas

• How it works: Combines early-binding of operations (pre-registered safe tools with strict guarantees) with late-binding of workflows (agent constructs the execution plan at runtime).
• Components:
  • Registry (deploy time, early binding of operations)
  • Plan (run time, agent creates execution DAG)
  • Execution engine (durable workflow execution like Conductor)
  • Compensation and rollbacks
• When to use: When building flexible agentic workflows that must execute mission-critical or high-risk tasks safely.

Examples & Case Studies

An SRE remediation agent detects an unhealthy Kubernetes cluster, analyzes logs, determines root cause, and executes a rollback workflow.

• Illustrates: Separation of agent planning (analyzing logs, planning steps) and harness execution (performing cluster restarts, sending Slack notifications).
• Lesson: Agents should propose remediation steps, but the harness must execute them deterministically with proper error handling and approvals.

Actionable Takeaways

• Immediate:
  • Treat AI agents as components rather than standalone monolithic applications.
  • Separate non-deterministic LLM planning from deterministic execution logic.
  • Implement durable execution runtimes for any agentic workflow running longer than a single request.
• Strategic:
  • Adopt late-bound sagas to give agents runtime flexibility while enforcing early-bound safety guarantees.
  • Leverage existing open-source workflow orchestration engines (such as Conductor) to manage agent state and orchestration.
  • Design systems with clear separation of brains (LLMs) and hands (harness execution).
• Questions to investigate:
  • How do we handle complex multi-agent negotiations under strict deterministic safety constraints?
  • What are the best patterns for managing state corruption when an agent's plan fails mid-execution?
  • How can Model Context Protocol (MCP) standards integrate more tightly with durable execution runtimes?

Claims Worth Verifying

• The agent owns the plan; the harness owns the guarantees. (Architectural principle)
• Durability is the cost of admission for production agent workflows. (System design claim)

Notable Quotes

"The agent owns the plan. The harness owns the guarantees." "Durability is the cost of admission." "The agent proposes. The harness operationalizes."

Compressed Summary

• Agents are components requiring durable runtime harnesses, not standalone apps.
• Separate non-deterministic LLM planning from deterministic execution operations.
• Late-bound sagas allow agents to build plans at runtime using pre-registered safe tools.
• Durable workflow engines like Conductor ensure long-running agent workflows survive failures and crashes.
• Keywords: agents, harness, orchestration, durability, sagas
• Core insight: AI agents need durable runtime harnesses that separate non-deterministic LLM planning from deterministic execution to safely operate in production.