Agents That Write Their Own Tools at Runtime — Sandhya Subramani, AWS

Overview

• Speaker: Sandhya Subramani
• Channel: AI Engineer
• Main topic: Meta-tooling and self-extending AI agents that write and load custom tools at runtime
• Purpose: To demonstrate how developers can build self-extending AI agents using meta-tooling frameworks like Strands Agents, enabling runtime tool creation, multi-agent pattern orchestration, and rigorous evaluation and guardrails. Sandhya Subramani from AWS presents 'Agents That Forge Their Own Tools' at AI Engineer World's Fair 2026. She introduces meta-tooling and Strands Agents, an open-source Agent Harness SDK. The talk demonstrates how agents with limited initial tools can dynamically recognize capability gaps, write new tool files at runtime, and instantly load and use them without human intervention or restart. She also covers multi-agent design patterns (swarms, graphs, workflows), comprehensive built-in evaluation levels, and crucial guardrails including sandbox execution environments and tool allowlists.

Topic Map

Introduction to Meta-Tooling and Runtime Self-Extension

• Explanation: Discussing the limitation of traditional static tool sets in AI agents and introducing agents that can recognize errors or missing capabilities, write their own tools, and load them at runtime.
• Key claims:
  • Traditional agents are restricted by pre-written static tools.
  • Meta-tooling enables agents to identify when they lack a capability and write their own tools.
  • Runtime tool creation allows agents to fix failures and extend their capabilities without restarting.
• Examples:
  • An agent asked to do complex math creates a math calculator tool on the fly.
  • An agent asked to count characters in a word writes a character counter tool at runtime.
• Terminology:
  • meta-tooling
  • runtime tool creation
  • system prompt
  • LLM agent
• Why it matters: It transforms agents from static software wrappers into autonomous systems capable of adapting and expanding their own operational scope.

Strands Agents Framework and Minimal Configuration

• Explanation: Introducing Strands Agents as an open-source, model-agnostic Agent Harness SDK by AWS that supports Model Context Protocol (MCP) and agent-to-agent (A2A) communication.
• Key claims:
  • Strands Agents is a model-agnostic agent harness SDK built by AWS.
  • Only a few lines of code and three core tools (editor, shell, load tool) are needed for meta-tooling.
  • Model agnosticism allows developers to swap LLM backends without changing agent architecture.
• Examples:
  • Setting up a basic agent with five lines of Python code importing Agent and shell from strands.
  • Configuring meta-tooling with system prompt, editor, shell, and load_tools_from_directory=True.
• Terminology:
  • Strands Agents
  • Agent Harness SDK
  • Model Context Protocol (MCP)
  • agent-to-agent (A2A)
  • shell tool
  • editor tool
  • load tool
• Why it matters: It lowers the barrier to entry for building robust, extensible agentic systems that work across multiple LLM providers.

Multi-Agent Patterns: Swarms, Graphs, and Workflows

• Explanation: Explaining architectural patterns for multi-agent systems, including swarms, directed graphs, and deterministic workflows.
• Key claims:
  • Swarms involve specialized sub-agents working collaboratively in parallel on complex tasks.
  • Graphs use specialized agents as nodes and edges to manage dependencies and information flow.
  • Workflows combine parallel sub-agents, graphs, and swarms in a deterministic pipeline.
• Examples:
  • A travel planning agent breaking down a request into a flight agent, activities agent, and itinerary agent.
  • A workflow combining create_subagent nodes with downstream swarm and graph patterns.
• Terminology:
  • swarms
  • graphs
  • workflows
  • sub-agents
  • hand-off
• Why it matters: Complex real-world tasks require structured multi-agent collaboration rather than single monolithic prompt loops.

Built-In Evaluations and Guardrails

• Explanation: Detailing four-level evaluations (session, trace, tool, output level) and four security guardrails for production agentic systems.
• Key claims:
  • Evaluations span session level (end-goal), trace level (faithfulness, accuracy), tool level, and output level.
  • Guardrails must constrain the environment via sandbox execution, constrain tools via allowlists, enforce permissions, and ensure observability.
  • Code execution must be sandboxed to prevent unintended modification of codebases.
• Examples:
  • Evaluating whether an agent successfully booked flight tickets at the session level.
  • Using sandbox environments to isolate tool code execution from the main application codebase.
• Terminology:
  • evaluations
  • session level
  • trace level
  • tool level
  • output level
  • guardrails
  • sandbox
  • allowlist
  • observability
• Why it matters: Autonomous agents with self-writing capabilities introduce security and reliability risks that require strict governance and evaluation frameworks.

Key Points

Agents can autonomously write their own tools at runtime

• Explanation: Instead of pre-defining every tool, an agent given an editor, shell, and load tool can inspect its environment, write custom Python tool files, and immediately load them.
• Evidence: Live demo showing the agent creating math_calculator.py and character_counter.py dynamically when faced with equation and character counting queries.
• Practical implication: Developers do not need to anticipate every tool an agent might need during pre-deployment planning.

Strands Agents provides a model-agnostic harness

• Explanation: AWS-built Strands Agents allows developers to use any LLM backend without rewriting agent prompts or architectural harness logic.
• Evidence: Presentation slides and code snippets illustrating python imports and agent initialization with Strands SDK.
• Practical implication: Easy experimentation with state-of-the-art models as they are released without infrastructure refactoring.

Sandbox environments are mandatory for meta-tooling agents

• Explanation: Giving an agent the ability to write code and execute shell commands requires strict sandbox isolation to protect production systems.
• Evidence: Discussion on guardrails covering environment sandboxing, network egress controls, and tool allowlists.
• Practical implication: Prevents catastrophic errors or prompt injection attacks from corrupting production applications.

Frameworks, Models & Processes

Strands Agents Meta-Tooling Framework

• How it works: An agent is initialized with a system prompt defining tool specifications and file templates, along with three core tools (editor, shell, load_tool). When a task requires a missing capability, the agent writes a new tool file and loads it dynamically.
• Components:
  • System Prompt (Tool specification and template rules)
  • Editor Tool (for writing tool code)
  • Shell Tool (for executing commands)
  • Load Tool (load_tools_from_directory=True for runtime loading)
• When to use: When building autonomous agents that need to dynamically extend their capabilities based on user requests without redeployment.

Multi-Agent Evaluation Framework

• How it works: Evaluates agent performance across four distinct tiers: session level (goal achievement), trace level (faithfulness and tool selection), tool level (parameter accuracy), and output level (correctness and tone).
• Components:
  • Session Level
  • Trace Level
  • Tool Level
  • Output Level
• When to use: When deploying production multi-agent systems that require rigorous trust, observability, and regression testing.

Examples & Case Studies

An agent with no initial math calculator tool received a complex math query, realized it lacked the capability, wrote math_calculator.py, loaded it, and computed the answer.

• Illustrates: Runtime meta-tooling and self-extending agent capability.
• Lesson: Agents can identify missing tools and forge them autonomously when given file editing and loading primitives.

A travel planning agent decomposed a request for a Hawaii trip into three specialized sub-agents: flight, activities, and itinerary agents.

• Illustrates: Multi-agent pattern orchestration (Swarms / Workflows).
• Lesson: Complex tasks are effectively handled by delegating sub-tasks to specialized parallel sub-agents.

Actionable Takeaways

• Immediate:
  • Explore Strands Agents SDK on GitHub for building model-agnostic agent harnesses.
  • Implement meta-tooling by equipping agents with editor, shell, and load tools.
  • Ensure sandbox isolation before enabling autonomous code generation in agents.
• Strategic:
  • Move away from static tool registries toward self-extending agent architectures.
  • Adopt multi-agent design patterns (swarms, graphs, workflows) for complex domain problems.
  • Build comprehensive evaluation pipelines covering session, trace, tool, and output levels.
• Questions to investigate:
  • How do we prevent malicious code generation when agents write their own tools?
  • What are the latency tradeoffs of runtime tool compilation and loading?
  • How can evaluation frameworks automatically score open-ended agentic tool creation?

Claims Worth Verifying

• Strands Agents is an open-source Agent Harness SDK built by AWS. (factual)
• The Python version of Strands automatically wrote its own TypeScript version. (anecdotal)

Notable Quotes

"Agents that forge their own tools." "How many agents do you know that can fix code at runtime?"

Compressed Summary

• Agents can dynamically write and load custom tools at runtime using meta-tooling.
• Strands Agents is an open-source, model-agnostic AWS SDK supporting MCP and A2A.
• Multi-agent patterns include swarms, directed graphs, and deterministic workflows.
• Robust evaluations cover session, trace, tool, and output levels.
• Guardrails like sandboxing and tool allowlists are essential for secure agentic execution.
• Keywords: meta-tooling, strands-agents, runtime-tools, multi-agent, guardrails
• Core insight: Equipping AI agents with editor, shell, and loader primitives enables them to autonomously write, load, and execute custom tools at runtime, drastically increasing autonomy and adaptability.