The TWIML AI Podcast with Sam Charrington · Published 2026-10-06

Why Jev Is Changing How We Build With AI

privateReview: unreviewedConfidence: high
machine-nativetypesafejevRLHFtransformersCalibrationoptimizationagentic-systemsdiscrete-decisions
Open on YouTube ↗

Summary

Overview

  • Speaker: Sam Charrington and Diogo Almeida
  • Channel: The TWIML AI Podcast with Sam Charrington
  • Main topic: Machine-Native Intelligence and Jev
  • Purpose: To explore the philosophy behind Jev, machine-native intelligence, the gap between AI capabilities and software utility, and the future of optimization in machine learning. In this episode of the TWIML AI Podcast, Sam Charrington interviews Diogo Almeida, co-founder and CEO of TypeSafe and former OpenAI researcher who worked on InstructGPT and RLHF. They discuss Jev, a new model from TypeSafe designed around machine-native intelligence rather than human-facing string generation, focusing on reliable decision-making for software execution.

Topic Map

Introduction and the Core Premise of Jev

  • Explanation: Discussion of TypeSafe's stealth release of Jev, developer reactions, and Diogo Almeida's background at OpenAI.
  • Key claims:
    • Jev is designed for machine-native intelligence, optimizing for reliable software decisions rather than human-facing text generation.
  • Examples:
    • InstructGPT and RLHF work at OpenAI
  • Terminology:
    • Jev
    • TypeSafe
    • RLHF
    • InstructGPT
    • machine-native intelligence
  • Why it matters: Shifts the focus of AI optimization from human consumption to software execution and decision-making.

The AI Utility Gap

  • Explanation: Why AI is superhumanly good at certain tasks like coding and deep research, yet useless at basic data entry or insurance underwriting.
  • Key claims:
    • Modern LLMs are token-predicting transformers optimized for human-facing strings, not discrete software decisions.
  • Examples:
    • ChatGPT vs. customer support agents at companies like Anthropics
  • Terminology:
    • token-predicting transformer
    • string LLMs
    • discrete decisions
  • Why it matters: Explains why high-performing language models fail at basic software automation tasks.

Optimization, Compute, and RLHF

  • Explanation: Comparing string generation optimization with decision-making optimization, and the asymmetry introduced by RLHF.
  • Key claims:
    • You get what you optimize for; string LLMs are optimized for strings, not software actions.
  • Examples:
    • Keyboard typing vs. automated actions
  • Terminology:
    • RLHF
    • reward model
    • inference time
    • asymmetry in optimization
  • Why it matters: Highlights the misalignment between current ML training objectives and real-world software needs.

Classifiers vs. Intelligent Agents

  • Explanation: Debating whether Jev is 'just a classifier' or a fundamental advance in machine-native AI.
  • Key claims:
    • Classifiers are interfaces of usefulness, but Jev provides a deeper paradigm for software integration.
  • Examples:
    • Meta-powered vs. Google-powered classifiers
  • Terminology:
    • classifier
    • logit bias
    • function calling
  • Why it matters: Reframes how developers should think about the outputs of AI models relative to software architectures.

Key Points

Machine-Native Intelligence

  • Explanation: Models optimized for software consumption and reliable decision-making rather than human reading.
  • Evidence: Diogo Almeida's experience at OpenAI working on RLHF and InstructGPT.
  • Practical implication: Enables reliable software automation without relying solely on unstructured text outputs.

The Optimization Trap

  • Explanation: Training models on string generation creates fundamental limitations when applied to discrete software tasks.
  • Evidence: Comparison of math problem-solving capabilities versus accounting automation.
  • Practical implication: Engineers must design models specifically for the tasks and execution environments they target.

The Role of RLHF in Calibration

  • Explanation: RLHF introduces asymmetry that helps align model outputs with preferred behaviors during inference.
  • Evidence: Discussion of reward models and post-training dynamics.
  • Practical implication: Post-training is critical for making models reliable enough for production software environments.

Frameworks, Models & Processes

Machine-Native Intelligence Framework

  • How it works: Aligns model optimization directly with software decision constraints rather than human text consumption.
  • Components:
    • Discrete decision outputs
    • Software-native action spaces
    • Calibrated probability distributions
  • When to use: Building backend automation, insurance underwriting agents, and reliable software workflows.

Examples & Case Studies

Developers dismissing Jev as 'just a classifier' upon release.

  • Illustrates: The misunderstanding of Jev's deeper architectural focus on machine-native decision making.
  • Lesson: Look past simple interface comparisons to understand a model's foundational optimization target.

Actionable Takeaways

  • Immediate:
    • Evaluate AI models based on what they are optimized for, not just benchmark scores.
    • Distinguish between human-facing text generation and software-facing decision making.
  • Strategic:
    • Design machine-native models for backend software automation rather than wrapper-based interfaces.
    • Prioritize model reliability and calibration over raw string fluency in enterprise deployments.
  • Questions to investigate:
    • How do logit biases and probability outputs compare between Jev and standard LLMs?
    • What are the specific architectural differences between reinforcement learning for text versus software agents?

Claims Worth Verifying

  • Jev is optimized for making reliable decisions that software can act on rather than generating human-facing strings. (Product Architecture Claim)
  • RLHF destroys the raw calibration of pre-trained models by introducing mode-dropping loss. (Technical Machine Learning Claim)

Notable Quotes

"Jev's real advance isn't speed or cost. It's a different idea of what AI should be optimized for." "I think AI is just so unbelievably smart, yet so unbelievably useless at the kinds of things you'd really expect it to be useful for."

Compressed Summary

  • Jev focuses on machine-native intelligence over human string generation.
  • Current LLMs are optimized for strings, making them poor at discrete software decisions.
  • Post-training and RLHF shape model reliability and calibration.
  • True utility in AI requires aligning optimization with software execution constraints.
  • Keywords: machine-native, typesafe, jev, rlhf, transformers, calibration
  • Core insight: AI systems must be optimized for machine-native decision-making and software execution rather than human-facing text generation to achieve true automation utility.

Core insights

5
Architecturemedium noveltystrong evidence

LLMs optimized for string generation are fundamentally misaligned for discrete software decision-making tasks, explaining the AI utility gap.

Why it matters

It explains why high-performing language models fail at basic software automation tasks and suggests that agentic systems should use models trained for discrete decisions, not text.

Generalization

Any AI system intended for software execution should align its optimization objective with the action space of the target environment, not with human-readable output.

Modern LLMs are token-predicting transformers optimized for human-facing strings, not discrete software decisions.
Open source video
You get what you optimize for; string LLMs are optimized for strings, not software actions.
Open source video
Tradeoffhigh noveltymoderate evidence

RLHF improves alignment but destroys calibration via mode-dropping loss, creating a trade-off for decision-critical applications.

Why it matters

Calibration is essential for reliable decision-making in software automation, so blindly using RLHF may harm agentic system reliability.

Generalization

Post-training techniques that optimize for human preference may inadvertently degrade probabilistic calibration, which is critical for machine-native tasks.

RLHF destroys the raw calibration of pre-trained models by introducing mode-dropping loss.
Open source video
RLHF introduces asymmetry that helps align model outputs with preferred behaviors during inference.
Open source video
Architecturemedium noveltymoderate evidence

Machine-native intelligence framework proposes optimizing models for software consumption: discrete decision outputs, software-native action spaces, and calibrated probability distributions.

Why it matters

Provides a blueprint for building AI that integrates reliably with software systems, moving beyond wrapper-based interfaces.

Generalization

Agentic systems should be built on models whose output interface matches the software's action space, rather than parsing text.

Aligns model optimization directly with software decision constraints rather than human text consumption.
Open source video
Discrete decision outputs
Open source video
Software-native action spaces
Open source video
Calibrated probability distributions
Open source video
Failure Modemedium noveltymoderate evidence

The AI utility gap: LLMs are superhuman at tasks like coding and deep research yet useless at basic data entry or insurance underwriting due to optimization mismatch.

Why it matters

Challenges assumptions about the generality of LLMs and highlights the need for task-specific optimization in agentic systems.

Generalization

General-purpose language models may fail at narrow discrete tasks despite impressive language capabilities, necessitating specialized architectures for automation.

AI is just so unbelievably smart, yet so unbelievably useless at the kinds of things you'd really expect it to be useful for.
Open source video
Why AI is superhumanly good at certain tasks like coding and deep research, yet useless at basic data entry or insurance underwriting.
Open source video
Mental Modelmedium noveltyweak evidence

Dismissing Jev as 'just a classifier' misses the deeper paradigm: classifiers can be interfaces for software-native decision-making, not merely narrow models.

Why it matters

Reframes how developers should evaluate model outputs relative to software architectures, looking past surface interfaces.

Generalization

When integrating AI into software, focus on the underlying optimization target and output semantics rather than the apparent model type.

Classifiers are interfaces of usefulness, but Jev provides a deeper paradigm for software integration.
Open source video
Developers dismissing Jev as 'just a classifier' upon release.
Open source video

Deep dives

4

Calibration preservation in RLHF for decision-critical agents

Research question

How can post-training optimize for human preference alignment without destroying the probabilistic calibration needed for reliable discrete decisions?

Why

Calibration is a precondition for reliable software automation; if RLHF mode-drops the distribution, agentic systems may be confidently wrong.

RLHF destroys the raw calibration of pre-trained models by introducing mode-dropping loss.
Open source video
RLHF introduces asymmetry that helps align model outputs with preferred behaviors during inference.
Open source video
Source video

Optimization objective alignment for machine-native agents

Research question

What changes in reward modeling and policy optimization when the target action space is discrete software decisions rather than human-facing strings?

Why

The AI utility gap suggests current training objectives are mismatched to automation tasks; aligning objectives could unlock reliability in data entry, underwriting, and tool use.

Modern LLMs are token-predicting transformers optimized for human-facing strings, not discrete software decisions.
Open source video
You get what you optimize for; string LLMs are optimized for strings, not software actions.
Open source video
Source video

AI utility gap failure taxonomy

Research question

What concrete failure modes occur when string LLMs are inserted into software automation pipelines for discrete decisions?

Why

Without a failure taxonomy, teams cannot tell whether failures come from parsing, ambiguity, calibration, or task framing.

Why AI is superhumanly good at certain tasks like coding and deep research, yet useless at basic data entry or insurance underwriting.
Open source video
Source video

Classifier as interface vs machine-native paradigm

Research question

What architectural features distinguish a machine-native decision model from a narrow classifier or a function-calling wrapper?

Why

If teams dismiss such models as 'just classifiers', they may miss the deeper shift toward software-native output semantics and calibrated decisions.

Classifiers are interfaces of usefulness, but Jev provides a deeper paradigm for software integration.
Open source video
Source video

Article ideas

4

The AI Utility Gap Is an Optimization Mismatch, Not a Capability Gap

LLMs fail at basic automation not because they are insufficiently intelligent, but because they are optimized for human-facing strings; agentic systems must align training objectives with discrete action spaces.

Angle

Architectural critique and practical re-alignment roadmap for agent builders.

Source video

RLHF's Calibration Tax: Why Alignment Can Make Agents Less Reliable

RLHF's mode-dropping behavior trades probabilistic calibration for preference alignment, so decision-critical agents should measure calibration separately and treat alignment as a distinct optimization.

Angle

Trade-off analysis with engineering recommendations for post-training pipelines.

Source video

Stop Calling It Just a Classifier: Machine-Native Models as Software Interfaces

Dismissing machine-native models as classifiers conflates interface simplicity with architectural depth; the right abstraction is output semantics aligned to software actions.

Angle

Paradigm reframe for software architects evaluating AI integration.

Source video

From Strings to Decisions: Redesigning the Model-Software Boundary

The model-software boundary should move from parsed text to discrete decisions with calibrated confidence, reducing ambiguity and parsing failures in automation pipelines.

Angle

Integration architecture patterns and migration path from prompt/parse to decision-native interfaces.

Source video

Project ideas

3

Calibration-Preserving Post-Training Harness

beyond-evals

Adding a calibration-regularized post-training stage after RLHF will restore expected calibration error on discrete decision tasks without sacrificing preference alignment win rate.

Proof of concept

Fine-tune a small transformer on a synthetic discrete decision task; compare pre-trained, RLHF, and RLHF+calibration-regularization checkpoints.

Measurement

Expected calibration error, Brier score, decision accuracy, reward-model win rate.

Source video

Discrete Action Interface Benchmark

new

Models trained for direct discrete decision outputs will achieve lower integration failure rates than string LLMs using prompt-based function calling on the same software action space.

Proof of concept

Build a sandbox with 50 software-action tasks (data entry, underwriting decisions) and compare a string LLM with function calling against a classifier/decision-head model.

Measurement

Task success rate, parse error rate, calibration error, latency.

Source video

Machine-Native Agent Runtime

movement-lab

An agent runtime that consumes calibrated discrete decisions directly, without text parsing, will reduce ambiguity-induced failures by more than 50% versus text-mediated tool calls.

Proof of concept

Implement two runtimes—text-mediated ReAct loop and decision-native API—and run both on a shared tool-use task suite.

Measurement

Failure taxonomy counts, human intervention rate, task completion rate.

Source video

Architectural implications

4

Move from using general-purpose string LLMs to models optimized for discrete software decisions.

Before

Agentic systems use string LLMs with prompt engineering and function calling to drive actions.

After

Agentic systems use machine-native models that directly output discrete decisions and calibrated probabilities.

Consequence

More reliable automation, but requires new model training pipelines and integration patterns; parsing errors reduced.

Source video

Post-training (RLHF) introduces a calibration trade-off that must be managed in decision-critical systems.

Before

RLHF is assumed to uniformly improve model quality and safety.

After

RLHF is recognized to potentially degrade calibration, so alternative post-training or calibration techniques are needed for agentic systems.

Consequence

Need to evaluate calibration explicitly and possibly separate alignment from calibration objectives.

Source video

Model evaluation for agentic systems shifts from benchmark scores to task-specific optimization alignment.

Before

Choose models based on leaderboard scores and general capabilities.

After

Evaluate models on the specific decision tasks they will perform, measuring reliability and calibration.

Consequence

Different selection criteria for production agentic systems, potentially favoring specialized models over general LLMs.

Source video

The abstraction boundary between model and software changes from text to discrete actions.

Before

Model outputs text that is parsed and interpreted by software.

After

Model directly outputs decisions/actions with confidence, integrating with software action spaces.

Consequence

Simpler integration, less ambiguity, but requires models designed for that output interface.

Source video

Tradeoffs and failure modes

4

RLHF alignment vs. calibration

Benefit

Aligns model outputs with preferred behaviors during inference.

Cost or risk

Destroys raw calibration of pre-trained models via mode-dropping loss, harming decision reliability.

RLHF destroys the raw calibration of pre-trained models by introducing mode-dropping loss.
Open source video
Source video

String LLMs for agentic tasks

Benefit

Strong language capabilities, good at coding and deep research.

Cost or risk

Optimized for strings, not discrete software decisions, leading to uselessness at basic automation like data entry or insurance underwriting.

Modern LLMs are token-predicting transformers optimized for human-facing strings, not discrete software decisions.
Open source video
Source video

Classifier simplicity vs. deeper paradigm

Benefit

Classifiers are easy to understand and integrate as interfaces of usefulness.

Cost or risk

Dismissing a model as 'just a classifier' misses its deeper architectural focus on machine-native decision making.

Classifiers are interfaces of usefulness, but Jev provides a deeper paradigm for software integration.
Open source video
Source video

AI utility gap

Benefit

AI is superhumanly good at coding and deep research.

Cost or risk

Useless at basic data entry or insurance underwriting due to optimization mismatch.

I think AI is just so unbelievably smart, yet so unbelievably useless at the kinds of things you'd really expect it to be useful for.
Open source video
Source video

Open questions

4

How do logit biases and probability outputs compare between Jev and standard LLMs?

Why unresolved

The summary raises this as a question to investigate, without providing empirical data.

Research direction

Conduct comparative analysis of logit distributions and calibration metrics between machine-native and string LLMs.

Source video

What are the specific architectural differences between reinforcement learning for text versus software agents?

Why unresolved

The summary poses this as an open question, no details provided.

Research direction

Investigate reward modeling and policy optimization differences when the action space is discrete decisions versus text generation.

Source video

How can calibration be preserved or restored after RLHF for decision-critical applications?

Why unresolved

The summary notes RLHF destroys calibration but doesn't offer solutions.

Research direction

Explore post-training techniques that maintain calibration while aligning with human preferences.

Source video

What are the failure modes when integrating string LLMs into software automation pipelines?

Why unresolved

The summary mentions the AI utility gap but doesn't enumerate specific failure modes.

Research direction

Systematically catalog errors arising from parsing text outputs, ambiguity, and poor calibration in agentic systems.

Source video

Key claims

7
factualVerification needed

Jev is optimized for making reliable decisions that software can act on rather than generating human-facing strings.

Evidence

Jev is optimized for making reliable decisions that software can act on rather than generating human-facing strings.

Question

Does Jev's architecture and training objective differ from standard LLMs in ways that improve discrete decision reliability?

Source video
causalVerification needed

RLHF destroys the raw calibration of pre-trained models by introducing mode-dropping loss.

Evidence

RLHF destroys the raw calibration of pre-trained models by introducing mode-dropping loss.

Question

What is the empirical evidence that RLHF degrades calibration, and what is 'mode-dropping loss'?

Source video
factualVerification needed

Modern LLMs are token-predicting transformers optimized for human-facing strings, not discrete software decisions.

Evidence

Modern LLMs are token-predicting transformers optimized for human-facing strings, not discrete software decisions.

Question

Is the optimization objective of standard LLMs exclusively string generation, and does that limit their performance on discrete decision tasks?

Source video
causalVerification needed

RLHF introduces asymmetry that helps align model outputs with preferred behaviors during inference.

Evidence

RLHF introduces asymmetry that helps align model outputs with preferred behaviors during inference.

Question

What is the mechanism of this asymmetry, and how does it improve alignment during inference?

Source video
comparativeVerification needed

AI is superhumanly good at coding and deep research yet useless at basic data entry or insurance underwriting.

Evidence

Why AI is superhumanly good at certain tasks like coding and deep research, yet useless at basic data entry or insurance underwriting.

Question

Is this generalization accurate across different LLMs and tasks, and what causes the disparity?

Source video
opinionVerification needed

Post-training is critical for making models reliable enough for production software environments.

Evidence

Post-training is critical for making models reliable enough for production software environments.

Question

What specific post-training techniques are most effective for reliability, and how do they balance calibration?

Source video
opinionVerification needed

Classifiers are interfaces of usefulness, but Jev provides a deeper paradigm for software integration.

Evidence

Classifiers are interfaces of usefulness, but Jev provides a deeper paradigm for software integration.

Question

What architectural features of Jev distinguish it from a simple classifier beyond interface?

Source video

Connections

5