The Pragmatic Engineer · Published 2026-09-09

Building Codex with Tibo Sottiaux

privateReview: unreviewedConfidence: high
codexrustopenaiagentsopen-sourceagent-harnesshostile-simulationfuzz-testingcode-reviewlocal-vs-cloud
Open on YouTube ↗

Summary

Overview

  • Speaker: Gergely Orosz and Tibo Sottiaux
  • Channel: The Pragmatic Engineer
  • Main topic: The engineering history, architecture, language choice, and integration of OpenAI Codex
  • Purpose: To provide a deep dive into the engineering practices, technical decisions, and organizational culture behind OpenAI Codex and AI coding agents. Gergely Orosz interviews Tibo Sottiaux, Head of Core Products and Platform at OpenAI, discussing how Codex started, why it was built in Rust, how code reviews changed within the Codex team, and what it meant to merge Codex into ChatGPT.

Topic Map

Origins of Codex and Rust

  • Explanation: Discussion on how Codex started as an internal research project and why it was built in Rust despite Rust not being widely distributed for models at the time.
  • Key claims:
    • Codex was initially built as an internal research project in Rust.
    • Rust proved to be very good for agents fairly quickly when effort was put into it.
    • Building was the fun part; maintenance and re-architecting were the painful parts.
  • Examples:
    • Building Codex in Rust before it was widely used for model distribution.
  • Terminology:
    • Codex
    • Rust
    • Agents
  • Why it matters: Shows how early architectural choices in systems programming language can impact agent performance and maintainability.

Code Reviews and Maintenance

  • Explanation: How the role of code reviews is changing inside AI engineering teams where maintenance and re-architecting are becoming automated.
  • Key claims:
    • The role of code reviews is changing.
    • Maintenance is becoming automated, changing from a slow tax over time to automated execution.
  • Examples:
    • Asking Codex anything during onboarding to get quick answers.
  • Terminology:
    • Code Review
    • Maintenance
    • Automation
  • Why it matters: Demonstrates how AI coding assistants alter traditional engineering workflows and team onboarding.

Open Source vs Closed Source in AI Coding Tools

  • Explanation: Why OpenAI decided to make Codex CLI, SDK, and app server open source, and the trade-offs of open-sourcing developer tools.
  • Key claims:
    • Open-sourcing the CLI, SDK, and app server builds community and contributor engagement.
    • Open source forces companies to compete at the model and harness layers.
    • There are downsides to open source, such as handling random contributions and copycat tools.
  • Examples:
    • Open-sourcing Codex CLI and SDK.
  • Terminology:
    • Open Source
    • CLI
    • SDK
    • Harness
  • Why it matters: Highlights community-building strategies for developer tools in the AI era.

Key Points

Rust for AI Agents

  • Explanation: Building Codex in Rust provided type safety, performance, and compile-time verification which benefited agent execution.
  • Evidence: Tibo noted that Rust proved quite good for agents when effort was applied.
  • Practical implication: Consider systems programming languages for building robust agent harnesses.

Merging Codex into ChatGPT

  • Explanation: Merging a locally running coding agent into a managed cloud-based stack presented significant architectural and stack differences.
  • Evidence: Codex was fully local initially, whereas ChatGPT runs on a managed cloud-based stack.
  • Practical implication: Bridging local execution tools with cloud infrastructure requires careful architectural alignment.

Frameworks, Models & Processes

Hostile Simulation (Antithesis)

  • How it works: Runs the entire system in a hostile simulation with targeted and fuzz testing to find bugs before users do.
  • Components:
    • Hostile simulation
    • Targeted testing
    • Fuzz testing
    • Full reproduction steps
  • When to use: When building complex distributed systems and agentic software where manual code review is insufficient.

Examples & Case Studies

Building Codex in Rust before it was on distribution for Rust.

  • Illustrates: Choosing the right tool for performance and safety early on.
  • Lesson: Language choice matters for agent performance and reliability.

Actionable Takeaways

  • Immediate:
    • Automate maintenance tasks using AI agents.
    • Leverage Rust or robust type systems for agent harnesses.
  • Strategic:
    • Open-source developer toolings to build community and trust.
    • Focus on co-designing models and harnesses together.
  • Questions to investigate:
    • How do code reviews evolve when AI generates the majority of code?
    • What are the best architectures for bridging local execution with cloud models?

Claims Worth Verifying

  • Codex was built in Rust one year before ChatGPT came out. (historical)
  • Entire handles 418 pushes per second, 89x faster than competitors. (performance metric)

Notable Quotes

"Codex is one of the most popular AI coding harnesses today. But how did it all start?" (at 1:41) "Building was the fun part, but maintenance was the painful part." (at 0:41)

Compressed Summary

  • Codex was initially built in Rust as an internal research project.
  • The role of code review is evolving as maintenance becomes automated.
  • Open-sourcing Codex components built a strong community despite trade-offs.
  • Merging local coding agents into cloud platforms requires bridging disparate stacks.
  • Keywords: codex, rust, openai, agents, open-source
  • Core insight: OpenAI Codex was built in Rust to ensure high performance and reliability for AI coding agents, pioneering new paradigms in developer workflows and automated code maintenance.

Core insights

6
Mechanismmedium noveltymoderate evidence

Hostile simulation testing — running the entire system inside a deliberately hostile simulated environment with targeted and fuzz testing, producing full reproduction steps — is used as a substitute/complement for manual code review when shipping agentic software.

Why it matters

If manual review is the bottleneck for AI-generated code volume, then automated adversarial simulation becomes a load-bearing reliability layer rather than a nice-to-have test suite. It changes what CI, reproducibility, and bug-triage infrastructure must provide (full reproduction steps, not just pass/fail).

Generalization

Any system where code volume outpaces human review capacity needs automated adversarial environments that generate reproducible failures as a first-class artifact.

Runs the entire system in a hostile simulation with targeted and fuzz testing to find bugs before users do.
Open source video
When building complex distributed systems and agentic software where manual code review is insufficient.
Open source video
Mental Modelhigh noveltymoderate evidence

Open-sourcing the harness (CLI, SDK, app server) pushes competitive differentiation to the model and harness layers rather than the client tooling layer.

Why it matters

It inverts the usual assumption that client tooling is the moat. If the CLI/SDK is commoditized, engineering investment must concentrate on the harness (context assembly, tool orchestration, execution loop) and the model — otherwise the product is replicable.

Generalization

For AI developer tools, open-sourcing the interface layer is a deliberate move to compete where switching costs actually live: the reasoning/execution stack.

Open source forces companies to compete at the model and harness layers.
Open source video
Open-sourcing the CLI, SDK, and app server builds community and contributor engagement.
Open source video
Architecturemedium noveltymoderate evidence

A locally-executing coding agent and a managed cloud stack are architecturally divergent enough that merging them is a genuine re-architecture, not a deployment change.

Why it matters

Local agents own filesystem, process, credentials, and latency budget; cloud stacks own scaling, state, and policy. The boundary determines where state lives, how tools execute, and what can be sandboxed — a decision that is hard to reverse later.

Generalization

Where an agent's execution boundary sits (local vs managed) dictates the tool surface, security model, and state design more than the model choice does.

Codex was fully local initially, whereas ChatGPT runs on a managed cloud-based stack.
Open source video
Bridging local execution tools with cloud infrastructure requires careful architectural alignment.
Open source video
Practicemedium noveltyweak evidence

Automated maintenance converts what was a slow, compounding tax on engineering time into executable, on-demand operations — which in turn changes what code review is for.

Why it matters

If re-architecting and maintenance work can be executed by agents, human review shifts from line-by-line correctness checking toward intent, risk, and boundary validation. Team structure and review tooling should follow.

Generalization

When a class of work moves from human-latency to machine-latency, the review gate must re-target from syntax/correctness to judgment.

The role of code reviews is changing.
Open source video
Maintenance is becoming automated, changing from a slow tax over time to automated execution.
Open source video
Empirical Resultmedium noveltyweak evidence

Choosing Rust for the agent harness was claimed to pay off for agent behavior specifically — attributed to type safety, performance, and compile-time verification — even before Rust was common in model-adjacent distribution.

Why it matters

Suggests harness language is not a neutral choice: compile-time guarantees may reduce a class of runtime failures in long-running, tool-calling loops where failure recovery is expensive.

Generalization

Systems-language invariants can be a reliability mechanism for agent runtimes, not just a performance optimization.

Rust proved to be very good for agents fairly quickly when effort was put into it.
Open source video
Looking toward systems programming languages for building robust agent harnesses.
Open source video
Mental Modelmedium noveltyweak evidence

Models and harnesses should be co-designed rather than layered independently.

Why it matters

Co-design implies harness changes can require model-side changes (prompt/tool schema, context format) and vice versa, breaking the assumption of a stable API contract between model and orchestration layer.

Generalization

Treating the harness as a fixed consumer of a model API constrains performance; co-evolution allows tighter context and tool affordances.

Focus on co-designing models and harnesses together.
Open source video

Deep dives

5

Hostile simulation as a replacement review gate for agent-generated code

Research question

Can a deliberately hostile simulated environment with targeted plus fuzz testing, emitting full reproduction steps, achieve a lower defect escape rate than human line-level review on high-volume agent-generated code — and at what cost per defect?

Why

If manual review is the bottleneck for AI-generated code volume, automated adversarial simulation becomes a load-bearing reliability layer, which changes what CI, reproducibility, and bug-triage infrastructure must produce: reproducible failures as a first-class artifact rather than pass/fail signals.

Runs the entire system in a hostile simulation with targeted and fuzz testing to find bugs before users do.
Open source video
When building complex distributed systems and agentic software where manual code review is insufficient.
Open source video
The role of code reviews is changing.
Open source video
Source video

Execution boundary design: local-first agent versus managed cloud stack

Research question

Which boundary architecture — thin local shim to a cloud runtime, dual execution with a shared tool schema, or fully remote sandbox — preserves state consistency and security without duplicating the execution loop when a local agent is merged into a managed stack?

Why

Local agents own filesystem, process, credentials and latency budget; cloud stacks own scaling, state and policy. Where that boundary sits determines the tool surface, security model and state design more than the model choice does, and it is hard to reverse later.

Codex was fully local initially, whereas ChatGPT runs on a managed cloud-based stack.
Open source video
Bridging local execution tools with cloud infrastructure requires careful architectural alignment.
Open source video
Codex was fully local initially, whereas ChatGPT runs on a managed cloud-based stack.
Open source video
Source video

Harness language as a reliability mechanism: Rust versus a GC'd language for agent runtimes

Research question

With identical tool semantics and identical model calls, does a Rust agent harness produce measurably fewer runtime failures and better recovery under tool-call faults than a GC'd-language harness, or is the observed quality gain explained by harness design alone?

Why

If compile-time and type guarantees suppress a class of runtime failures in long-running tool-calling loops where recovery is expensive, then harness language is not a neutral choice and should be evaluated as a reliability variable, not a performance one.

Rust proved to be very good for agents fairly quickly when effort was put into it.
Open source video
Building Codex in Rust provided type safety, performance, and compile-time verification which benefited agent execution.
Open source video
Tibo noted that Rust proved quite good for agents when effort was applied.
Open source video
Source video

Model-harness co-design versus a stable model API contract

Research question

When harness-specific context formats and tool schemas are allowed to deviate from a stable model API contract, do joint model+harness evaluations improve enough to justify the loss of interface stability?

Why

Co-design implies harness changes can force model-side changes and vice versa, breaking the assumption of a stable contract between model and orchestration layer. Without a joint scoring harness, teams cannot tell whether co-evolution gains exceed the integration cost they impose.

Focus on co-designing models and harnesses together.
Open source video
Open source forces companies to compete at the model and harness layers.
Open source video
Source video

What open-sourcing the client layer actually commoditizes

Research question

Does open-sourcing a CLI/SDK/app server measurably erode differentiation at the client layer and shift engineering investment toward the model and harness layers, or does it mainly convert client tooling into an unowned maintenance burden?

Why

It inverts the usual assumption that client tooling is the moat. If the interface layer is commoditized, investment must concentrate where switching costs actually live — context assembly, tool orchestration and execution loop — otherwise the product is replicable.

Open source forces companies to compete at the model and harness layers.
Open source video
Open-sourcing the CLI, SDK, and app server builds community and contributor engagement.
Open source video
There are downsides to open source, such as handling random contributions and copycat tools.
Open source video
Source video

Article ideas

4

Code Review Was Never About the Diff

Automating maintenance and generating code at machine volume does not reduce the need for review — it relocates the gate from line-level correctness to intent, risk and system-boundary judgment, and teams that keep reviewing diffs will silently lose their only real control point.

Angle

Argue that the review gate must be re-targeted rather than relaxed, and specify what a boundary-and-intent review artifact looks like when maintenance is executed by agents instead of humans.

Source video

Your Agent's Architecture Is Its Execution Boundary, Not Its Model

The local-versus-managed decision dictates the tool surface, security model and state design of an agent more than the model choice does, so merging a local-first agent into a managed cloud stack is a re-architecture whose cost is fixed at design time and cannot be paid down later with a deployment change.

Angle

Frame the boundary as the primary architectural decision and enumerate the components that must be rewritten rather than adapted when a local agent is consolidated into a managed stack.

Source video

Open-Sourcing Your CLI Is a Bet on Where Your Moat Lives

Open-sourcing the client layer is not a community gesture but a strategic wager: it is only correct if your defensibility genuinely sits in the model and harness, and teams that make the bet without harness investment have simply donated their differentiation to copycat tools.

Angle

Treat the CLI/SDK release as a forcing function that must be preceded by a named harness-level advantage, and identify the specific maintenance burden of random contributions.

Source video

Stop Treating the Harness as a Client of the Model

A stable API contract between model and orchestration layer feels like good engineering hygiene but caps performance; co-design means harness changes may legitimately require prompt, tool-schema and context-format changes, and only a joint evaluation harness can prove the trade is worth the integration cost.

Angle

Argue for joint model+harness scoring over interface stability, and describe what breaks in a versioned-contract world when either side moves.

Source video

Project ideas

4

Hostile Harness Gate

gatehouse

A CI gate that runs the full agent system inside a deliberately hostile simulated environment with targeted and fuzz testing, and that fails builds only with full reproduction steps attached, catches a strictly larger set of real defects per engineer-hour than human line-level review of the same changes.

Proof of concept

Instrument an agentic codebase with a simulation driver that injects hostile conditions, run targeted plus fuzz suites in CI, and force every failure artifact to carry an executable reproduction. Run in parallel with standard human review for one quarter and compare defect escape rates and time-to-triage.

Measurement

Defects escaped to production per 100 merged changes; median time from failure to confirmed reproduction; review hours displaced per defect caught.

Source video

Boundary-Agnostic Agent Runtime

movement-lab

An agent runtime whose tool and state interfaces are defined as boundary-agnostic will migrate from local execution to a managed cloud stack with no rewrite of the execution loop, whereas a runtime that assumes local filesystem and implicit trust will require one.

Proof of concept

Implement one minimal tool-calling agent twice: (a) with filesystem, process and credentials abstracted behind a portable interface, and (b) with direct local assumptions. Then move both behind a remote sandbox and measure the diff surface of the execution loop.

Measurement

Lines changed in the execution loop during migration; state-consistency violations under concurrent local and remote execution; added tool-call latency.

Source video

Harness Language Fault Study

new

Given identical tool semantics and identical model calls, a Rust agent harness exhibits a lower rate of unrecovered runtime failures and lower tail latency under injected tool-call faults than an equivalent harness in a garbage-collected language.

Proof of concept

Build the same minimal tool-calling loop in Rust and in one GC'd language with a shared tool schema and fault-injection driver. Subject both to identical tool-call faults — timeouts, malformed responses, partial writes — and compare failure and recovery behavior.

Measurement

Unrecovered failure rate per 1,000 tool calls; p99 latency under fault injection; recovery success rate; development effort to reach parity.

Source video

Joint Model-Harness Scoring Bench

beyond-evals

Scores produced by an evaluation harness that rates model and harness configurations jointly will rank configurations differently than scores from a fixed stable model API, and the divergence will be large enough to change at least one shipping decision.

Proof of concept

Define a joint scoring harness over a fixed task set, evaluate the same model under a stable API contract versus a harness-specific context format and tool schema, and compare rankings and per-task deltas.

Measurement

Rank correlation between joint scores and fixed-contract scores; per-task score delta attributable to harness-specific formatting; number of shipping decisions the divergence would reverse.

Source video

Architectural implications

2

An agent runtime that begins local and later must run inside a managed cloud stack faces a boundary mismatch.

Before

Execution, filesystem access, and tool invocation live on the developer's machine with implicit trust.

After

Execution must be relocated or split, with state, credentials, and tool sandboxing handled by managed infrastructure.

Consequence

Tool interfaces and state models need to be designed as boundary-agnostic from the start, or the merge becomes a rewrite of the execution loop.

Source video

Adversarial simulation replaces line-by-line human review as the primary defect-discovery gate for high-volume agent-generated code.

Before

Review is a human step in the PR pipeline, and CI runs deterministic unit/integration tests.

After

CI includes a hostile simulated environment with targeted and fuzz testing that must emit full reproduction steps.

Consequence

Reproducibility tooling and failure-artifact quality become part of the review surface; reviewers consume generated failure reports instead of diffs.

Source video

Tradeoffs and failure modes

3

Open-sourcing the agent client (CLI, SDK, app server)

Benefit

Builds community and contributor engagement, and shifts competition to the model and harness layers.

Cost or risk

Handling random contributions and defending against copycat tools.

There are downsides to open source, such as handling random contributions and copycat tools.
Open source video
Source video

Automating maintenance with agents

Benefit

Maintenance changes from a slow tax over time to automated execution.

Cost or risk

Code review's role must be redefined; the human gate that previously caught maintenance-introduced regressions is weakened.

Maintenance is becoming automated, changing from a slow tax over time to automated execution.
Open source video
Source video

Building the harness in Rust

Benefit

Type safety, performance, and compile-time verification benefit agent execution.

Cost or risk

The stated upside came only 'when effort was applied,' implying a nontrivial investment cost and re-architecting pain.

Building was the fun part, but maintenance was the painful part.
Open source video
Source video

Open questions

4

How should code review be redesigned when AI generates the majority of code and maintenance is automated?

Why unresolved

The summary asserts the role of review is changing but gives no replacement procedure, gate criteria, or measured outcome.

Research direction

Prototype review gates that evaluate intent, risk, and system-boundary impact rather than line-level diffs; measure defect escape rate against baseline human review.

Source video

What architecture best bridges a local-execution agent with a managed cloud stack without duplicating the execution loop?

Why unresolved

The summary identifies the stack mismatch as significant but does not describe the resolution.

Research direction

Compare boundary designs — thin local shim to cloud runtime, dual-execution with shared tool schema, or fully remote sandbox — on latency, security, and state consistency.

Source video

How much of the claimed agent-quality gain comes from Rust versus from the surrounding harness design?

Why unresolved

The claim is qualitative ('quite good for agents') with no benchmark or comparison against other languages.

Research direction

Build the same minimal agent harness in Rust and a GC'd language with identical tool semantics and compare failure rates, tail latency, and recovery behavior under tool-call faults.

Source video

How should model and harness co-design be organized so changes on one side do not invalidate the other?

Why unresolved

Stated as a strategic recommendation with no described interface, versioning scheme, or evaluation loop.

Research direction

Define an evaluation harness that scores model+harness configurations jointly, and test whether harness-specific context formats improve results over a stable model API contract.

Source video

Key claims

7
factualVerification needed

Codex was initially built as an internal research project in Rust.

Evidence

Codex was initially built as an internal research project in Rust.

Question

What was the timeline and was Rust used from the first commit?

Source video
opinionVerification needed

Rust proved to be very good for agents fairly quickly when effort was put into it.

Evidence

Rust proved to be very good for agents fairly quickly when effort was put into it.

Question

What measurable agent properties improved, and relative to what baseline?

Source video
causalVerification needed

Open source forces companies to compete at the model and harness layers.

Evidence

Open source forces companies to compete at the model and harness layers.

Question

Does open-sourcing the client layer measurably erode client-layer differentiation and shift spend to harness/model work?

Source video
factualVerification needed

Codex was built in Rust one year before ChatGPT came out.

Evidence

Codex was built in Rust one year before ChatGPT came out.

Question

Do public release dates and internal commit history support this timeline?

Source video
comparativeVerification needed

Entire handles 418 pushes per second, 89x faster than competitors.

Evidence

Entire handles 418 pushes per second, 89x faster than competitors.

Question

Which competitors define the baseline, and under what benchmark conditions was 418 pushes/sec measured?

Source video
causalVerification needed

Merging a locally running coding agent into ChatGPT required bridging fundamentally different stacks.

Evidence

Codex was fully local initially, whereas ChatGPT runs on a managed cloud-based stack.

Question

Which specific components had to be rewritten versus adapted during the merge?

Source video
predictionVerification needed

Automating maintenance turns a slow, compounding maintenance tax into automated execution.

Evidence

Maintenance is becoming automated, changing from a slow tax over time to automated execution.

Question

Does automated maintenance reduce total cost of ownership, or shift it to review and validation?

Source video

Connections

4