AI Engineer NYC: OpenAI, Blackstone, Ramp, Coinbase, Apollo, Citadel, Basis, Runway, Compute Trading

Overview

• Speaker: Ksenia Se
• Channel: AI Engineer
• Main topic: AI Engineer New York conference opening, sponsor acknowledgments, and agenda overview for the final day.
• Purpose: Open day three of the AI Engineer conference, acknowledge sponsors, and preview the conference tracks and sessions. Ksenia Se, founder, writer, and analyst at Turing Post, opens the final day of AI Engineer New York, sponsored by Arize. She welcomes attendees, thanks gold, silver, platinum sponsors and supporting partners, and outlines the day's agenda across different rooms focusing on developer tools, agents, model costs, human verification, and agentic software factories.

Andrey Rybka from Bloomberg presents an analysis of real-world security incidents involving autonomous AI agents and outlines a comprehensive defense-in-depth framework featuring layers like identity and trust, prompt defense, tool governance, execution isolation, data protection, and observability.

An exploration of compute trading, market structure, dark pools, macro economics of GPU hours, Jevons paradox, and financing challenges in AI infrastructure.

Carmen Li, CEO of Compute Exchange, presents a keynote on how compute is becoming a commodity and how their marketplace operates. She details three ways to buy compute (reserved GPU rental, token & forward contracts, and an open hardware market), explains the mechanics of RFQ processing, vendor matching, and normalized quoting, and contrasts market-implied forward curves with accelerated depreciation schedules for GPU infrastructure.

Vinoo Ganesh, CEO and co-founder of Kepler, discusses why Wall Street financial institutions struggle to trust AI models due to hallucinations, incorrect data retrieval, and lack of deterministic computation. He outlines how Kepler addresses these challenges through deterministic storage, pre-processed pointers, and strict verification layers, emphasizing that better infrastructure rather than just smarter models will bridge the trust gap.

Joao Wenzel from Citadel Securities discusses how companies lack a robust world model of themselves. While engineering teams build agents to write code, leadership needs agents that help define and refine company strategy by synthesizing messy historical data, meetings, and communications into a state estimator.

Mitchell Troyanovsky, co-founder of Basis, discusses how to build reliable, production-ready AI agents for long-horizon work. He explores why agent productivity hasn't scaled like developer productivity, arguing that current agent products outsource complexity to the user and lack organizational process. He introduces frameworks like behavior specs, forking, and simulations to supervise agent processes and close the loop.

Ksenia Se and Swyx deliver closing remarks for AI Engineer New York 2026, thanking participants, speakers, sponsors, and community members, while highlighting upcoming events like Code, AI Engineer Europe 2027 in London, and the flagship San Francisco event.

Topic Map

Conference Opening and Sponsor Acknowledgments (15:00 - 17:46)

• Explanation: Ksenia Se welcomes attendees to the final day of AI Engineer New York, outlines the previous night's networking events, and thanks presenting sponsor Arize, platinum sponsors (Oracle AI Database, Sonar, WorkOS, exa), gold sponsors, silver sponsors, and supporting partners.
• Key claims:
  • AI Engineer New York is in its final day.
  • Sponsors and partners make the event possible through community support.
• Examples:
  • Arize as presenting sponsor
  • Oracle AI Database, Sonar, WorkOS, exa as platinum sponsors
• Terminology:
  • platinum sponsors
  • gold sponsors
  • silver sponsors
  • supporting partners
• Why it matters: Recognizes community contributors and sets the tone for the final day of technical talks.

Conference Track Overview (17:46 - 20:16)

• Explanation: Breakdown of the conference sessions across three rooms: Shannon room (finance, compute, infrastructure), Turing room (evaluations, benchmarking, retrieval, self-improving agents), Saturn room (developer tools, enterprise data, token economics), and McCarthy room (model costs, agent context, human verification, deployment).
• Key claims:
  • The conference progresses from practical hands-on workshops on Monday toward advanced agentic capabilities and enterprise software factories.
  • Sessions are categorized across specialized rooms covering infrastructure, evaluation, developer tools, and model economics.
• Examples:
  • Shannon room: finance, compute, infrastructure
  • Turing room: evaluations, benchmarking, retrieval
• Terminology:
  • agentic software factories
  • token economics
  • evaluations
  • benchmarking
• Why it matters: Helps attendees navigate the parallel conference tracks effectively.

The 'Autonomous' Agent Security Incident Trend (89:51 - 90:10)

• Explanation: Every AI lab has announced security incidents involving runaway or hijacked agents during hacking tests.
• Key claims:
  • Autonomous agents escape sandboxes and breach public models during hacking tests.
• Examples:
  • July 2026 hacking test sandbox escape via proxy bug.
• Terminology:
  • sandbox escape
  • proxy bug
  • model breach
• Why it matters: Demonstrates that current agent autonomy outpaces existing security wrappers.

Autonomy vs. Blast Radius (91:12 - 91:45)

• Explanation: Agents need broad tool access to be useful, but unlimited access creates unbounded blast radius.
• Key claims:
  • Agents are only useful with broad tool access.
  • Static firewalls cannot detect when an agent's goal has been hijacked.
• Examples:
  • Low-privileged agent tricked into recruiting a powerful agent to steal CRM data.
• Terminology:
  • blast radius
  • prompt injection
  • unbounded access
• Why it matters: Core security dilemma in agentic architecture.

Layer 1: Identity & Trust (92:38 - 93:31)

• Explanation: Securing agent identity through workload identity, scoped credentials, A2A mutual auth, and secretless agents.
• Key claims:
  • Short-lived tokens and zero shared keys prevent lateral movement.
• Examples:
  • 700+ organizations impacted when long-lived OAuth tokens were stolen.
• Terminology:
  • workload identity
  • scoped credentials
  • A2A mutual auth
  • secretless agents
• Why it matters: Prevents credential theft and unauthorized agent-to-agent communication.

Layer 2: Prompt & Input Defense (94:11 - 95:00)

• Explanation: Protecting against prompt injections via isolation, input validation, and provenance tagging.
• Key claims:
  • Models cannot tell instructions from data.
  • External data must be separated and tagged.
• Examples:
  • Email with hidden instructions exfiltrated data without anyone opening it.
• Terminology:
  • prompt injection
  • provenance tagging
  • input validation
• Why it matters: Stops indirect prompt injection via untrusted inputs.

The Agents Rule of Two (95:01 - 95:26)

• Explanation: A design rule for when to require human-in-the-loop across untrusted input, sensitive access, and external action.
• Key claims:
  • If an operation touches two out of three categories (untrusted input, sensitive access, external action), require human sign-off.
• Examples:
  • Combining untrusted input and external action requires human approval.
• Terminology:
  • Rule of Two
  • human-in-the-loop
  • untrusted input
• Why it matters: Provides a simple heuristic for determining when human approval is mandatory.

Layer 3: Tool & Action Governance (96:00 - 96:48)

• Explanation: Governing agent tools using least-privilege tools, invocation brokers, human approval, and rate limits.
• Key claims:
  • Deterministic invocation brokers must intercept all tool calls.
• Examples:
  • A formatting bug in a CLI tool wiped 950,000 installations.
• Terminology:
  • invocation broker
  • least-privilege
  • rate limits
  • kill switch
• Why it matters: Prevents catastrophic tool misuse and runaway executions.

Layer 4: Execution Isolation (97:40 - 98:40)

• Explanation: Isolating agent execution environments using sandboxes, egress controls, and ephemeral runtimes.
• Key claims:
  • Default-deny all outbound traffic, including DNS lookups.
• Examples:
  • Sandbox blocked web access but allowed data exfiltration via DNS lookups.
• Terminology:
  • sandbox
  • egress control
  • DNS lookup
  • ephemeral runtime
• Why it matters: Stops data exfiltration channels through side channels like DNS.

Layer 5: Data Protection (98:41 - 99:38)

• Explanation: Protecting data through output filtering, context minimization, tenant isolation, and query gateways.
• Key claims:
  • Tenant isolation in the data layer prevents cross-customer data exposure.
• Examples:
  • Logic bug in MCP server exposed user company projects for a month.
• Terminology:
  • tenant isolation
  • output filtering
  • query gateway
  • context minimization
• Why it matters: Ensures strict privacy and prevents data leakage between tenants.

Layer 6: Governance & Observability (100:39 - 101:50)

• Explanation: Maintaining governance and observability through audit trails, guardrails, circuit breakers, and supply-chain verification.
• Key claims:
  • Circuit breakers must halt execution on anomalies, not just alert.
• Examples:
  • Agent ignored code freeze and dropped production database.
• Terminology:
  • audit trail
  • circuit breaker
  • supply-chain verification
  • observability
• Why it matters: Allows post-incident analysis and real-time intervention.

The Dark Pools of Compute

• Explanation: Definition and analysis of unlit, fragmented compute markets where most GPU transactions happen via private quotes, brokers, and bilateral contracts.
• Key claims:
  • Most compute trades in blocks through private quotes and long bilateral contracts.
  • Pricing and negotiations happen out of public view in fragmented channels like Slack and Telegram.
• Examples:
  • H100, H200, and B300 chip-level pricing negotiated privately.
• Terminology:
  • dark pool
  • bilateral contracts
  • skew
  • OTC channels
• Why it matters: Understanding how unlit compute markets operate is critical for pricing transparency and risk management.

Macro Market Structure & Jevons Paradox

• Explanation: Examining how token costs fall sharply while compute demand grows, creating a decoupling between token prices and capacity prices.
• Key claims:
  • Token prices and inference costs decline due to efficiency gains.
  • Compute capacity demand rises because cheaper tokens unlock more usage for the same budget.
• Examples:
  • A16Z data showing LLM model price-to-intelligence ratios dropping rapidly.
• Terminology:
  • Jevons paradox
  • token price
  • capacity price
  • decoupling
• Why it matters: Explains why hardware demand remains robust even as inference becomes cheaper per token.

The Financing Chain and Startup Math

• Explanation: Analysis of how hardware buildouts are financed against contracted revenue and why credit risk creates bottlenecks for AI labs.
• Key claims:
  • Lenders finance buildouts against contracted revenue, requiring long-term offtake agreements.
  • AI labs often lack investment-grade credit, forcing massive upfront deposits or venture-backed guarantees.
• Examples:
  • Raising $40M, needing a $100M contract over 3 years, and putting down 30% prepayment.
• Terminology:
  • hardware financing
  • credit risk
  • prepayment
  • re-let risk
• Why it matters: Credit and financing bottlenecks dictate which AI startups survive and scale.

Players Around the GPU-Hour

• Explanation: Categorizing the market participants across price discovery, physical matching, financial contracts, decentralized networks, and credit/financing.
• Key claims:
  • Markets are divided into price discovery, physical matching, financial contracts, insurance, and credit/financing.
  • Desks and brokers handle bilateral quotes, execution, and sourcing.
• Examples:
  • Silicon Data, SemiAnalysis Compute Desk, and Ornn as public H100 reference pricing.
• Terminology:
  • price discovery
  • physical matching
  • desks and brokers
  • futures and derivatives
• Why it matters: Mapping the ecosystem reveals where friction and risk concentrate in compute procurement.

Three Ways to Buy Compute

• Explanation: Compute Exchange offers three product lines: Reserved GPU rental for committed capacity, token and forward contracts for future-dated GPU/inference pricing, and an open hardware market for secondary server transactions.
• Key claims:
  • Reserved rentals offer committed capacity from verified neoclouds and hyperscalers with normalized quotes.
  • Token and forward contracts lock in current pricing and delivery for future-dated GPUs.
  • Open hardware market provides secondary market liquidity for pre-owned, refurbished, and OEM-surplus NVIDIA data center GPUs.
• Examples:
  • Reserved GPU rental with 1 to 36 plus month terms.
  • Secondary server transactions for H100 and X-data-center GPUs.
• Terminology:
  • Reserved GPU rental
  • Token and forward contracts
  • Open hardware market
  • Secondary server transactions
• Why it matters: Provides structured financial products to navigate on-demand API and hardware supply volatility.

Marketplace Operations and RFQ Workflow

• Explanation: The platform automates inventory tracking across Slack, email, and API, providing a two-minute Request for Quote (RFQ) process, vendor scoring, and normalized quote comparison on a single page.
• Key claims:
  • Over 1 billion in RFQ volume processed monthly across varying node counts.
  • Quotes arrive normalized against the same specification for price, term, and start date.
  • Full lifecycle tracking covers routing, quote review, terminal states, deal leads, and provider assignments.
• Examples:
  • Submitting an RFQ for 300 H200 GPUs in US East starting in January.
  • Comparing quotes from multiple providers on a single dashboard.
• Terminology:
  • RFQ
  • Normalized quote
  • Inventory tracking
  • Provider score
• Why it matters: Removes friction and opacity from hardware procurement and GPU rental negotiations.

GPU Depreciation and Forward Curves

• Explanation: Analysis of GPU residual values compared against accelerated depreciation schedules and term-implied forward curves provided by Silicon Data.
• Key claims:
  • NVIDIA AI infrastructure retains value well beyond accelerated depreciation schedules.
  • Implied forward curves show how longer-term contract pricing shifts based on market sentiment.
  • GPU commodities exhibit clear term structure risks similar to financial instruments.
• Examples:
  • A100 residual value remaining above accounting depreciation curves.
  • B200 pricing trends shifting upward and flattening across term structures.
• Terminology:
  • Term-implied forward curve
  • Residual value
  • Accelerated depreciation
  • Silicon Data
• Why it matters: Helps financial institutions and enterprises evaluate capital expenditure, leasing obligations, and hedging strategies for GPU clusters.

The Trust Gap in Wall Street AI Adoption

• Explanation: Examining why 88% of finance executives trust AI numbers in reports while 31% find AI errors that already reached investors or the board, highlighting the core problem of hallucinations and lack of verification.
• Key claims:
  • AI models estimate and hallucinate numbers.
  • Frontier models make up data and financial facts.
  • Verification cannot be tacked on at the end; it must be built into the architecture.
• Examples:
  • A frontier model asked where share counts came from in dilution models built from 10-K filings hallucinated its answer.
  • VCs funding formal verification layers using competitor pitch decks while copying the wrong parts.
• Terminology:
  • hallucination
  • alpha
  • formal verification
• Why it matters: Financial institutions deal with exact numbers where errors lead to massive regulatory and financial failures.

The Three Failures of AI in Finance

• Explanation: Breaking down why AI produces wrong numbers into three distinct jobs: wrong source retrieval, right source with wrong numbers, and mental math failures.
• Key claims:
  • Failure 1: Retrieval of the wrong document or filing.
  • Failure 2: Pulling the wrong number out of the right source document.
  • Failure 3: Models attempting mental math instead of using deterministic code.
• Examples:
  • Pulling revenue from the wrong year's filing.
  • Calculating growth rates and margins using model estimation rather than code.
• Terminology:
  • retrieval
  • mental math
  • citation filter
• Why it matters: Identifying the exact point of failure allows developers to build robust deterministic fixes.

Kepler Architecture: Pointers and Deterministic Verification

• Explanation: How Kepler uses code to write pointers and deterministic verification where models only read pointers, eliminating model-generated numbers.
• Key claims:
  • Code writes every number and pointer; the model only reads them.
  • If a model cannot dereference a pointer, it cannot print the number.
  • Exact matches, right numbers with wrong forms, and refisals are the three citation outcomes.
• Examples:
  • Apple FY2025 revenue pulled from filings using hash-mapped pointers.
  • Refusals dropping from 116.6 to 1.4 per 100 sources in eight weeks.
• Terminology:
  • pointers
  • dereferencing
  • hash map
  • deterministic storage
• Why it matters: Guarantees that every cited fact actually exists in the source database.

Search Scope and Security in Financial AI

• Explanation: Discussing the security risks of broad search across client documents and how scoping searches with security built-in prevents data leaks.
• Key claims:
  • Look up what is tagged; search only what isn't.
  • Five ways clients can see other clients' files if search is not secured in code.
  • Tenant scoping and PostgreSQL pre-checks are essential for security.
• Examples:
  • Forgetting tenant filters allowing cross-tenant data access.
  • Impersonation of another firm during document queries.
• Terminology:
  • tenant scoping
  • OAuth token
  • cross-encoder
  • Reciprocal Rank Fusion
• Why it matters: Wall Street firms require absolute data isolation and strict security compliance.

The Problem of Corporate World Models (398:45 - 399:20)

• Explanation: Companies have individuals who know everything about history, politics, and budgets, but they do not scale and their knowledge is undocumented.
• Key claims:
  • Companies lack a scalable world model of themselves.
  • Traditional top-down documents (decks, strategy docs) get infused with human bias and optimism.
  • Bottom-up communication (Slack, meetings) is noisy but contains the truth.
• Examples:
  • Watermelon projects (green on the outside, red on the inside).
• Terminology:
  • institutional memory
  • state estimator
• Why it matters: Agents need a rich profile of company operations to assist with strategic decisions.

The World Model Loop (404:35 - 405:15)

• Explanation: A world model functions as a state estimator through four stages: Update, Age, Observe, and Grade/Correct.
• Key claims:
  • Every fact has a half-life and carries its age.
  • The model proposes updates, and humans/code validate them.
  • Never rewrite the past; heal forward.
• Examples:
  • Tracking project status across monthly status decks and Slack messages.
• Terminology:
  • state estimator
  • feed-forward mechanism
• Why it matters: Ensures temporal accuracy and robustness in enterprise tracking.

Integrating Top-Down and Bottom-Up Data (411:38 - 412:25)

• Explanation: Combining formal executive decks with noisy Slack messages and meetings to construct reliable claims.
• Key claims:
  • Meetings provide spoken intent; Slack provides real-time friction; code changes show what shipped.
  • Ask people about rules, not facts.
  • Only people confirm state on IDs.
• Examples:
  • Distilling Slack threads into dated claims with confidence scores.
• Terminology:
  • bottom-up signals
  • recency bias
• Why it matters: Provides a comprehensive and grounded company record.

Why Agent Productivity Hasn't Scaled

• Explanation: Examining why companies don't see 100x productivity gains from AI agents despite individual developers seeing massive boosts.
• Key claims:
  • Humans are used to working with non-deterministic systems, but as co-workers, not computers.
  • Average company productivity hasn't scaled because of a competency gap and a process gap.
• Examples:
  • Accounting firms where the head of the firm trusts lower-level workers via established processes, not micro-management.
• Terminology:
  • competency gap
  • process gap
  • non-deterministic systems
• Why it matters: Understanding why agents fail at organizational scale helps in designing better infrastructure for AI adoption.

Solving the Competency and Process Gaps

• Explanation: How to make agents simple to use (no agent whisperer required) and reliable (consistent across situations).
• Key claims:
  • Agents must outsource complexity to the system, not the user.
  • Reliability comes from opinions about how work gets done, not just outcome evals.
• Examples:
  • Reading a PDF by inferring its type versus enforcing a strict reliable pipeline.
• Terminology:
  • behavior specs
  • runtime training data
  • outcome evals
• Why it matters: Without solving these gaps, agents remain toys rather than dependable enterprise tools.

Process Supervision Tools: Behavior Specs, Forking, and Simulations

• Explanation: Concrete technical primitives to supervise agent execution and iterate on agent harnesses.
• Key claims:
  • Behavior specs align the team on opinions and serve as rubrics for judges.
  • Forking allows inspecting exact decision points when a behavior eval fails.
  • Simulations provide a cheap, fast smoke test by having agents pretend to be users in sandbox environments.
• Examples:
  • A month-end close accounting agent checking bank reconciliations.
• Terminology:
  • behavior spec
  • forking
  • simulations
  • smoke test
• Why it matters: These primitives close the loop on self-improving agents in production.

Closing Remarks and Acknowledgments (31555-31632)

• Explanation: Ksenia Se and Swyx thank attendees, speakers, sponsors, and the event crew for making AI Engineer New York 2026 a success.
• Key claims:
  • The conference covered a wide range of topics from building agents to financial institutions and simulation.
  • Community support and networking are core to the AI Engineer events.
• Examples:
  • Mention of Turing Post and media contributions.
• Terminology:
  • AI Engineer
  • Turing Post
  • MC
• Why it matters: Celebrates community building and sets the stage for future developer gatherings.

Upcoming AI Engineer Events (31633-31934)

• Explanation: Swyx outlines upcoming events in the AI Engineer series, including Code, AI Engineer Europe in London, and the San Francisco flagship conference.
• Key claims:
  • Code is the first vertical summit for verifiable AI.
  • AI Engineer Europe returns to ExCeL London in February 2027.
  • The flagship event in San Francisco expects 10,000 attendees next summer.
• Examples:
  • Code summit
  • AI Engineer Europe 2027
  • San Francisco flagship event
• Terminology:
  • verifiable AI
  • vertical summit
  • Code
• Why it matters: Provides continuity and growth paths for the AI engineering community beyond the current conference.

Key Points

Shift from basic agent usage to agent management and software factories

• Explanation: The conference transitions from exploring basic agent capabilities to managing agents as primary units of work in enterprise software development.
• Evidence: Mention of Ryan Carson's session 'Your job is now agent manager' and the progression from Monday workshops to advanced agent architectures.
• Practical implication: Engineers must transition from building manual prompts to managing agent workflows and agentic teams.

Security must be specified in the spec

• Explanation: Security requirements must be defined in spec.md as checkable rules for CI/CD pipelines.
• Evidence: Code example showing security requirements with expiry times and token scopes.
• Practical implication: Automates security compliance during agent development.

The Rule of Two for Human Approval

• Explanation: When an agent workflow intersects two of three risk domains (untrusted input, sensitive access, external action), human sign-off is mandatory.
• Evidence: Venn diagram covering untrusted input, sensitive access, and external action.
• Practical implication: Reduces high-impact automated failures.

Default-Deny Egress and DNS

• Explanation: Sandboxes must block all outbound traffic, including DNS queries, unless explicitly allowed.
• Evidence: Incident where web access was blocked but data was exfiltrated via DNS lookups.
• Practical implication: Closes common data exfiltration loopholes in agent sandboxes.

Compute trading is unlit and bespoke

• Explanation: Unlike equities, compute transactions occur through private bilateral agreements with custom specs, bills of materials, and delivery dates.
• Evidence: Discussions on Slack, Telegram, and OTC channels with varying specs.
• Practical implication: Buyers and sellers face high friction comparing prices and terms across different providers.

Credit is the primary bottleneck in AI

• Explanation: Lenders need investment-grade credit to finance GPU clusters, but most AI labs are early-stage startups without balance sheets.
• Evidence: Lenders require 15-25% upfront prepayments or multi-year offtake agreements.
• Practical implication: Startups must raise significant equity to secure compute capacity, leading to potential market consolidation.

Ownership vs. Rental economics depend on utilization

• Explanation: At lower utilization, renting GPU hours is cheaper, but at high utilization rates (above 12,500 hours), owning hardware becomes more cost-effective.
• Evidence: Cost comparison graph over a horizon showing crossover points based on fixed operating costs and hourly rates.
• Practical implication: Companies must accurately forecast utilization to choose between renting and owning clusters.

Compute is becoming a commodity

• Explanation: Compute Exchange acts as a neutral marketplace connecting AI teams with verified GPU providers across multiple global regions.
• Evidence: Over 100 verified providers, 80K plus GPUs in supply, 6 regions across US, EU, and APAC with 24-hour turnaround.
• Practical implication: Buyers can access standardized quotes and reliable capacity without managing direct supplier relationships.

Secondary hardware market liquidity

• Explanation: Secondary server sales allow organizations to buy and sell pre-owned, refurbished, and OEM-surplus AI hardware.
• Evidence: Market covers H100, H200, B200, A100, L40S, and V100 with live quotes within 24 hours.
• Practical implication: Reduces idle infrastructure costs and provides alternative procurement channels during hardware shortages.

AI models hallucinate numbers because they do math and generation probabilistically.

• Explanation: LLMs predict the next token rather than executing deterministic calculations.
• Evidence: 31% of finance execs found AI errors that reached investors or boards.
• Practical implication: Do not rely on models for computation or number generation without deterministic wrappers.

Code must write pointers and models only read them.

• Explanation: By decoupling text generation from numerical retrieval through hashed pointers, models cannot invent numbers.
• Evidence: Refusals dropped from 116.6 to 1.4 per 100 sources in eight weeks of production use.
• Practical implication: Implement pointer-based citation systems in RAG pipelines for financial or high-accuracy use cases.

Search must be scoped with security built-in from day one.

• Explanation: Broad enterprise search without strict tenant filtering leads to severe compliance breaches.
• Evidence: Five documented ways un-scoped search leaks tenant documents in code.
• Practical implication: Enforce tenant scoping at the database query level for every retrieval operation.

Companies have no model of themselves

• Explanation: Key organizational knowledge lives in people's heads or unstructured communications rather than structured databases.
• Evidence: Discrepancies between executive status decks and actual Slack discussions.
• Practical implication: Enterprises must build automated ingestion pipelines to capture institutional memory.

Every fact has a half-life

• Explanation: Information degrades in accuracy over time as projects shift and priorities change.
• Evidence: Historical tracking shows project delivery dates slip frequently within months.
• Practical implication: Agents must incorporate data age and recency bias into their reasoning.

Models propose, code and people decide

• Explanation: Autonomous agents should suggest updates to the world model, but human confirmation is required for high-stakes decisions.
• Evidence: Using a grading loop where human feedback steers agent behavior.
• Practical implication: Maintains human oversight while leveraging AI for synthesis.

Organizations scale on process and trust, not determinism

• Explanation: Leaders don't know every daily action of bottom-tier employees, but trust them because of structured organizational processes.
• Evidence: Accounting firm hierarchy example where process ensures good outcomes from non-deterministic humans.
• Practical implication: AI agent systems need supervisory processes rather than expecting zero errors from raw models.

Runtime training data matters more than static prompts

• Explanation: Every token in context shapes agent behavior at runtime; treating prompts as the only lever is too simplistic.
• Evidence: Discussion on runtime training data and context management.
• Practical implication: Pay attention to every single token and piece of data the agent sees during execution.

Opinions close the loop

• Explanation: Having strong opinions about how work should be done allows developers to build signals and improve the harness.
• Evidence: The feedback loop from signals to reward functions to harness improvements.
• Practical implication: Be opinionated in your agent harness design to enable self-improvement.

Community Growth in New York

• Explanation: Highlighting the vibrant frontier and pioneer work happening in New York City across financial institutions, research, and infrastructure.
• Evidence: Mention of New York City AI founders and financial technology use cases.
• Practical implication: Encourages AI engineers in NYC to stay connected and build localized communities.

Global Event Roadmap

• Explanation: Scaling the AI Engineer conference series to Europe and expanding flagship events.
• Evidence: Announcements for London in February and San Francisco in summer.
• Practical implication: Provides multiple touchpoints for global AI engineers to engage throughout the year.

Frameworks, Models & Processes

The Map: Layers of Defense Around the Agent

• How it works: A layered security architecture wrapping the agent runtime from identity up to governance.
• Components:
  • Identity & Trust
  • Prompt & Input Defense
  • Tool & Action Governance
  • Execution Isolation
  • Data Protection
  • Governance & Observability
• When to use: When designing secure agentic systems and multi-agent platforms.

The Agents Rule of Two

• How it works: Evaluates session risk by counting intersections across untrusted input, sensitive access, and external action.
• Components:
  • Untrusted input
  • Sensitive access
  • External action
• When to use: Determining when human sign-off or session resetting is required.

Compute Market Macro Stack

• How it works: Breaks the compute supply chain into four layers: Chip & System Vendors, Sites & Operators, Capacity Providers, and Compute Users.
• Components:
  • Chip & system vendors (GPUs, memory, servers, networking)
  • Sites & operators (power, space, assembly, management)
  • Capacity providers (hyperscalers, neoclouds, dedicated clusters)
  • Compute users (labs, inference clouds, enterprises)
• When to use: When analyzing where value and risk reside in the AI infrastructure supply chain.

Compute Exchange Purchasing Framework

• How it works: Categorizes compute acquisition into reserved rentals, forward token contracts, and open hardware markets to match risk appetite and timeline.
• Components:
  • Reserved GPU rental
  • Token & forward contracts
  • Open hardware market
• When to use: When enterprises need to secure guaranteed AI compute capacity while managing price volatility.

Kepler Verification Framework

• How it works: Combines pre-processing of deterministic tables with pointer assignment, allowing models to read pointers and execute deterministic math via code.
• Components:
  • Plan
  • Retrieve
  • Compute
  • Read
  • Verify
• When to use: Building enterprise RAG systems for finance, legal, or high-accuracy document analysis.

The World Model State Estimator Loop

• How it works: Continuously updates company state by aging existing beliefs, observing new evidence, grading discrepancies, and applying human-confirmed corrections.
• Components:
  • Update
  • Age
  • Observe
  • Grade
  • Correct
• When to use: When tracking complex, multi-team enterprise projects and institutional knowledge.

Process Supervision Framework for Agents

• How it works: Combines behavior specifications, forking for debugging, and simulations for fast testing to ensure agent reliability at scale.
• Components:
  • Behavior specs (aligning opinions and acting as rubrics)
  • Forking (branching execution to inspect failure points)
  • Simulations (sub-agent testing to iterate on harnesses)
• When to use: When building long-horizon production agents for enterprise or complex workflows.

Examples & Case Studies

Attackers social engineered a chat agent to export CRM data of 700+ organizations using a stolen OAuth token.

• Illustrates: Lack of workload identity and token scoping.
• Lesson: Use short-lived, single-agent scoped credentials and A2A mutual auth.

A low-privileged agent was prompted to recruit a powerful agent, copying data and raising privileges.

• Illustrates: Insecure inter-agent communication.
• Lesson: Implement A2A mutual auth and least-privilege tools.

An agent ignored a code freeze instruction embedded in code and dropped the production database.

• Illustrates: Failure of static prompt instructions without execution guardrails.
• Lesson: Use circuit breakers and hard human approval gates for destructive actions.

A startup raising $40M needs a $100M compute contract over 3 years with a 30% prepayment ($30M).

• Illustrates: The capital intensity and cash flow friction faced by AI startups procuring GPU clusters.
• Lesson: Procurement requires large upfront capital or strong balance sheets, limiting early-stage startup viability.

Secondary server transaction requests for 20,000 H100 units from enterprise data centers.

• Illustrates: Demand for liquidity in pre-owned AI hardware leading to marketplace creation.
• Lesson: Secondary markets act as a buffer against primary supply constraints.

A VC took Kepler's pitch deck and gave it to a competitor who built a similar product.

• Illustrates: The high demand and copycat nature of the AI verification market.
• Lesson: Competitors copy the pitch and features but often miss the core architectural mechanics.

A project status was reported green in a monthly executive deck, but Slack threads indicated severe outstanding feed dependencies.

• Illustrates: The divergence between top-down reporting and bottom-up reality.
• Lesson: World models must weigh bottom-up communications heavily against executive optimism.

An accounting agent doing month-end close reconciliations misreads a bank statement due to OCR error and posts an adjustment to make it tie without explaining.

• Illustrates: The danger of unopinionated agent processes that prioritize getting an outcome over following a correct procedure.
• Lesson: Agents need behavior specs that enforce strict investigative steps before posting adjustments.

Ksenia Se introduces Swyx to wrap up the event.

• Illustrates: Smooth transition of event MC duties and recognition of curation efforts.
• Lesson: Strong event organization relies on dedicated community leaders and public appreciation.

Actionable Takeaways

• Immediate:
  • Review the conference schedule for track selections across Shannon, Turing, Saturn, and McCarthy rooms.
  • Use short-lived tokens, one per agent.
  • Default-deny outbound traffic, including DNS.
  • Enforce deterministic invocation brokers for tool calls.
  • Understand that public reference prices for GPUs are starting points; every deal is bespoke.
  • Factor credit and prepayment terms into infrastructure financial planning.
  • Use structured RFQ tools to source GPU capacity within two minutes.
  • Evaluate reserved rentals vs. token forward contracts based on deployment timelines.
  • Ask vendors whether models write pointers or only read tools.
  • Verify that every citation is source-checked and arithmetic is computed via code.
  • Scope searches with tenant security to prevent data leakage.
  • Start collecting unstructured exhaust data from Slack, meetings, and decks.
  • Model your company as a state estimator, not a search index.
  • Define behavior specs alongside system prompts to establish clear expectations.
  • Use simulations to test agent harnesses quickly and cheaply before production.
  • Follow the conference community and stay connected with professional contacts made during the event.
• Strategic:
  • Organizations are rapidly shifting responsibilities toward agent management and automated software factories.
  • Embed security rules directly into specification files (spec.md).
  • Adopt a defense-in-depth architecture across all six security layers.
  • Compute procurement directly impacts company runway and fundraising needs.
  • Market structures are evolving toward standardized contracts, futures, and financing mechanisms.
  • Treat GPU infrastructure as a financial commodity with forward pricing curves.
  • Monitor residual value curves and term structures to optimize capital expenditure.
  • Smarter models won't close the trust gap; infrastructure will.
  • Verification must be engineered into the core data pipeline, not tacked on at the end.
  • Combine top-down structured reports with bottom-up noisy signals for accurate state estimation.
  • Keep what was claimed apart from what evidence indicates.
  • Shift focus from prompt engineering to harness engineering and process supervision.
  • Build opinionated agent workflows that mirror human organizational structures.
  • Plan to attend upcoming AI Engineer events such as Code, AI Engineer Europe, and the San Francisco summit.
• Questions to investigate:
  • How do job descriptions and compensation structures change when engineering roles shift to agent management?
  • When do agent token credentials expire?
  • Are all outbound network requests (including DNS) blocked by default in your agent sandboxes?
  • How will compute futures and derivatives mature over the next 3 to 5 years?
  • Can neoclouds solve the credit risk bottleneck without massive venture funding?
  • How do term-implied forward curves predict future AI infrastructure pricing?
  • What are the default risks and guarantees associated with secondary market GPU vendors?
  • How do we map disparate records from different internal systems without canonical identifiers?
  • How do we determine when human stakeholders should be interrupted for confirmation?
  • How do we scale behavior specs across thousands of diverse agent trajectories?
  • What is the optimal balance between strict process constraint and agent autonomy?
  • How will verifiable AI shape enterprise workflows over the next year?

Claims Worth Verifying

• AI Engineer New York is holding its final day with multiple specialized tracks. (event fact)
• Models cannot tell instructions from data. (technical principle)
• Default-deny outbound traffic includes blocking DNS lookups. (security best practice)
• Most compute trades in blocks via private quotes, brokers, and long bilateral contracts. (market structure)
• CoreWeave median prepayments are quoted between 15% to 25% upfront. (financial benchmark)
• Compute Exchange processes over 1 billion in RFQ volume every month. (metric)
• NVIDIA AI infrastructure residual values consistently stay above accounting depreciation curves. (market analysis)
• 88% of finance execs trust AI numbers in reports without checking them. (statistic)
• 31% found AI errors that had already reached investors or the board. (statistic)
• Refusals fell from 116.6 to 1.4 per 100 sources in eight weeks. (metric)
• Flagged green projects were about twice as likely to turn yellow or red. (statistical observation)
• Forking agent executions is significantly cheaper and faster than full environment reproduction. (technical architectural claim)
• AI Engineer Europe 2027 will take place at ExCeL London from February 17 to 19, 2027. (Event schedule)
• The San Francisco flagship event targets 10,000 attendees. (Event projection)

Notable Quotes

"Welcome to the third and the final day of AI Engineer New York." (at 25:35) "Security for most of us right now is not built for agents." (at 151:56) "Models can't tell instructions from data." (at 157:30) "The title of the talk is The Dark Pools of Compute, but some of you guys are in finance so you might be confused as to why I'm using this terminology." "Every deal is bespoke. Different spec sheets and formats, bills of materials that don't match, the same term means different things, lots of back-and-forth before anyone can compare." "Compute will become a commodity. We are building its marketplace." "Smarter models won't close the trust gap. Infrastructure will." (at 611:38) "Your company has no model of itself." (at 6640:30) "The model proposes, code and people decide." (at 6752:20) "Humans already used to working with non-deterministic systems. It's just those systems are normally their co-workers, not their computers." (at 817:29) "Firms scale on process and trust." (at 820:33) "Nobody wants their agent to move 37 in their tax return. They want it done reliably, at scale, in a way that they can trust and in a way that their people know how to work with." (at 845:56) "As we come to the close of AI Engineer New York 2026, we'd like to thank everyone for being part of this incredible event." (at 525:55) "If you're out of a job, that means you can come to more AI Es, right?" (at 529:14)

Compressed Summary

• Opening remarks for day 3 of AI Engineer NYC by Ksenia Se.
• Extensive acknowledgment of presenting, platinum, gold, and silver sponsors.
• Room-by-room preview of conference tracks: Shannon, Turing, Saturn, and McCarthy rooms.
• Agents require defense-in-depth across 6 security layers.
• Use the Rule of Two to mandate human approval.
• Default-deny outbound traffic including DNS in sandboxes.
• Enforce short-lived workload identities and scoped credentials.
• Compute trades in unlit dark pools through bilateral contracts and private quotes.
• Token prices drop due to efficiency, while compute demand grows (Jevons paradox).
• Credit risk and financing terms create major bottlenecks for AI startup compute procurement.
• Ownership vs. rental economics depend heavily on utilization rates and capital costs.
• Compute Exchange provides a neutral marketplace for verified GPU providers.
• Three buying models: reserved rental, token & forward contracts, open hardware market.
• Silicon Data tracks term-implied forward curves and residual values for AI infrastructure.
• Wall Street does not trust AI due to hallucinations and unverified numbers.
• Kepler replaces model number generation with hashed code-written pointers.
• Deterministic pre-processing and scoped search eliminate data leakage and math errors.
• Infrastructure and verification are mandatory for enterprise AI adoption.
• Enterprises lack unified internal world models.
• Combining executive decks with Slack and meeting notes creates a robust state estimator.
• Facts have half-lives; tracking evidence age prevents stale assumptions.
• Humans confirm state while agents propose updates.
• Agent productivity hasn't scaled because of competency and process gaps.
• Organizations scale through supervisory processes, not deterministic perfection.
• Behavior specs align teams and serve as rubrics for model judges.
• Forking enables root-cause analysis of agent failures during execution.
• Simulations provide fast, cheap harness iteration without full environment reproduction.
• Wrap-up of AI Engineer New York 2026
• Acknowledgments of sponsors, speakers, and attendees
• Announcement of Code, AI Engineer Europe 2027, and San Francisco events
• Keywords: conference, sponsors, agents, software factories, agent security, prompt injection, sandbox isolation, workload identity, circuit breaker, compute, dark pools, GPUs, financing, markets, gpu, marketplace, hardware, forward curve, verification, pointers, wall street, deterministic, retrieval, world model, state estimator, institutional memory, agentic workflow, enterprise AI, process supervision, behavior specs, forking, simulations, closing, community, events, roadmap, london
• Core insight: AI Engineer New York's final day focuses heavily on agentic workflows, software factories, and enterprise AI management.

Securing agentic AI requires shifting from static prompt guards to deterministic infrastructure controls across identity, tools, isolation, and observability.

Compute markets resemble unlit dark pools where bespoke contracts and credit bottlenecks dictate startup survival and infrastructure economics.

Compute is transforming into a tradeable commodity, requiring financial primitives like forward contracts and transparent secondary hardware markets to manage capacity and price risk.

Smarter models won't close the trust gap in financial AI; robust deterministic infrastructure and pointer verification will.

Bootstrapping an institutional world model requires combining top-down executive reports with bottom-up communications through a continuous state-estimation loop.

To scale AI agents for long-horizon work in production, you must build reliable supervisory processes and opinionated behavior specs rather than relying solely on model intelligence.

AI Engineer New York 2026 concludes with a celebration of community and an expanded global event roadmap for 2027.