The Neon Show · Published 2026-09-18

The Real Reason AI Agents Are Wasting 10x More Power | Avinash Lakshman, Weilliptic

privateReview: unreviewedConfidence: high
llmagentspythonrustinfrastructurecommoditizationagent-runtimepower-efficiencyllm-commoditizationdata-sovereigntyopen-weight-models
Open on YouTube ↗

Summary

Overview

  • Speaker: Avinash Lakshman (Interviewed by Siddhartha Ahluwalia)
  • Channel: The Neon Show
  • Main topic: AI infrastructure, LLM commoditization, power inefficiencies in agentic AI, and enterprise AI startup playbook.
  • Purpose: To educate founders, developers, and investors on the realities of AI infrastructure, the economics of LLMs, and architectural considerations for building scalable agentic systems. Avinash Lakshman, founder and CEO of Weilliptic Inc. and veteran creator of Cassandra, Dynamo, and Hedwig, discusses the evolution of AI infrastructure, why LLMs are becoming a commodity similar to hard disks or SSDs, and the hidden power drain of running agentic workflows on Python. He highlights the infrastructural bottlenecks in enterprise AI, drawing parallels from cloud evolution, and shares lessons from building and exiting previous deep-tech companies.

Topic Map

LLM Commoditization

  • Explanation: LLMs are following the exact trajectory of hard disks, SSDs, and specialized servers, becoming commoditized layers where the intelligence shifts entirely to software.
  • Key claims:
    • LLMs will become commodity hard disk drives or SSDs.
    • Infrastructure costs for LLMs are super high, but pricing pressure will force costs super low.
    • Hardware vendors pounded their chests about 10x servers, but nobody cares about raw server specs anymore; intelligence moved to software.
  • Examples:
    • Hard disk drives and specialized servers in the late 90s and early 2000s selling at 10x prices before becoming pure commodity.
  • Terminology:
    • LLMs
    • Commoditization
    • Inference providers
  • Why it matters: Understanding commoditization helps founders and investors avoid over-investing in raw foundational model infrastructure expecting sustained high margins.

The Python Problem in Agentic AI

  • Explanation: As everyone builds agentic AI systems using Python, they are ignoring extreme power inefficiencies. Python is argued to be one of the most inefficient languages ever created for large-scale agent workloads.
  • Key claims:
    • Python is the most inefficient language ever made by mankind.
    • A non-trivial agent in Python uses ~15 watts, whereas an equivalent agent implemented in Rust consumes 0.7 watts.
    • At scale across millions of agents, the energy footprint difference between Python and Rust creates massive data center power crises.
  • Examples:
    • Comparing power utilization between Python-written agents and Rust-written agents at scale.
  • Terminology:
    • Agentic AI
    • Python
    • Rust
    • Power utilization effectiveness (PUE)
  • Why it matters: Highlights a critical engineering bottleneck that will plague enterprise AI deployments as agent usage scales exponentially.

Cloud Infrastructure vs AI Infrastructure Parallels

  • Explanation: Drawing parallels from how AWS, GCP, and Azure commoditized storage and compute, AI infrastructure is experiencing a similar cycle of centralized spend followed by optimization layers.
  • Key claims:
    • The cloud did not become commoditized right away; it was expensive until overlay services emerged.
    • AWS outages and regional outages prove that businesses need overlay solutions running on top of commodity cloud providers.
    • AI infra providers risk being wiped out by LLM providers unless they abstract data and compute sovereignty.
  • Examples:
    • AWS S3 running on pure commodity hardware while software layers handle intelligence and storage.
  • Terminology:
    • Cloud infrastructure
    • Data sovereignty
    • Compute sovereignty
  • Why it matters: Provides a historical playbook for anticipating how the AI infrastructure market will consolidate and where lasting enterprise value resides.

Key Points

Open weight models will win

  • Explanation: Open weight models democratize access and mirror open-source software disruption against proprietary infrastructure.
  • Evidence: Historical parallels with open-source software competing against proprietary enterprise software.
  • Practical implication: Enterprises should design architectures that remain agnostic to specific foundational models.

The Python efficiency bottleneck

  • Explanation: Interpreted languages like Python consume orders of magnitude more power per agent execution than compiled languages like Rust.
  • Evidence: Comparative power metrics cited (15 watts for Python agents vs 0.7 watts for Rust agents).
  • Practical implication: High-scale agentic applications will eventually require migration away from Python for core execution loops to survive data center power limits.

Data and compute sovereignty in enterprise AI

  • Explanation: Enterprises demand that their data and compute remain within specific geographic and jurisdictional boundaries.
  • Evidence: Enterprise clients in regions like Singapore requiring local data residency guarantees.
  • Practical implication: Infrastructure players must offer distributed, cryptographically secure compute platforms to satisfy enterprise compliance.

Frameworks, Models & Processes

The Infrastructure Playbook

  • How it works: Build core infrastructure layers that abstract hardware complexity, solve data sovereignty, and deliver operational efficiency as a managed service.
  • Components:
    • Commoditized underlying hardware/cloud
    • Software abstraction layer
    • Cryptographically secure audit trails
    • Managed service delivery model
  • When to use: When building deep-tech infrastructure companies facing intense competition from hyperscalers.

Examples & Case Studies

Building Dynamo and Cassandra at scale during early distributed systems evolution.

  • Illustrates: How foundational storage infrastructure shifts from specialized hardware to software-driven commodity.
  • Lesson: Keep teams small, focus on solving core architectural bottlenecks, and maintain frugality.

Actionable Takeaways

  • Immediate:
    • Evaluate the power and compute efficiency of agentic workflows.
    • Consider migrating performance-critical agent loops from Python to compiled languages like Rust.
  • Strategic:
    • Anticipate LLM commoditization and build defensible software layers on top.
    • Focus on enterprise data and compute sovereignty to capture high-margin B2B contracts.
  • Questions to investigate:
    • How will data center power grids handle the projected exponential growth of agentic AI workloads?
    • What open-weight models will dominate enterprise deployments over the next 3 years?

Claims Worth Verifying

  • Python agent power utilization is 15 watts compared to 0.7 watts for Rust agents. (technical benchmark)
  • Combolt acquired Hedwig for $225 million. (financial metric)

Notable Quotes

"LLMs will become hard disk drives or SSDs. They are all commodity today." (at 0:02) "In my opinion, that language needs to be de-breeded from the data centers." (at 0:38) "A non-trivial agent written in Python consumes about 15 watts, whereas the equivalent agent implemented in Rust consumes 0.7 watts."

Compressed Summary

  • LLMs are rapidly commoditizing like hard drives and SSDs.
  • Python is critically inefficient for agentic AI, consuming up to 20x more power than Rust.
  • Enterprise AI adoption requires strict data and compute sovereignty.
  • Small, frugal engineering teams outperform bloated organizations in infrastructure.
  • Keywords: llm, agents, python, rust, infrastructure, commoditization
  • Core insight: As LLMs commoditize and agentic AI scales, the hidden power crisis of interpreted languages like Python will force a massive architectural shift toward efficient compiled languages like Rust.

Core insights

6
Empirical Resulthigh noveltyweak evidence

The host-side runtime language of an agent, not the model, can dominate the power profile: a non-trivial Python agent is claimed to draw ~15W versus ~0.7W for the Rust equivalent (~20x). This reframes agent cost/efficiency from an inference-token problem into a process/execution-substrate problem.

Why it matters

If the magnitude holds, capacity planning, data-center power budgets, and per-agent unit economics are governed by the agent's execution runtime as much as by model calls, which is invisible to most token- and latency-centric observability stacks.

Generalization

In long-running autonomous workloads, steady-state overhead of the interpreter/process substrate can outweigh the marginal cost of the actual work, so substrate choice becomes an architectural decision rather than an implementation detail.

A non-trivial agent written in Python consumes about 15 watts, whereas the equivalent agent implemented in Rust consumes 0.7 watts.
Open source video
A non-trivial agent in Python uses ~15 watts, whereas an equivalent agent implemented in Rust consumes 0.7 watts.
Open source video
Predictionhigh noveltyweak evidence

The inefficiency claim is framed as a scale-dependent failure rather than a per-agent annoyance: at millions of concurrent agents the Python-vs-Rust gap is asserted to create a data-center power crisis, implying a threshold at which the runtime choice becomes an operational blocker.

Why it matters

Suggests efficiency work should be prioritized by fleet size; small deployments can absorb Python overhead, but high-fanout agent fleets may hit power/thermal/PUE ceilings that cannot be solved by model optimization or batching.

Generalization

Cost curves that are flat at pilot scale can become discontinuous at fleet scale, so architectural choices should be evaluated at the projected steady-state concurrency, not at demo scale.

At scale across millions of agents, the energy footprint difference between Python and Rust creates massive data center power crises.
Open source video
Predictionmedium noveltymoderate evidence

LLMs are predicted to commoditize along the HDD/SSD/specialized-server trajectory, with value migrating out of the model layer into the software layer above it — so defensibility comes from abstraction and operations, not from holding a foundational model.

Why it matters

Drives where an engineering team should invest: model-agnostic abstractions, routing, data/compute control planes and enterprise integrations rather than deep coupling to a single provider's API surface.

Generalization

When a capability layer commoditizes, durable margin shifts to the layer that owns orchestration, state, compliance, and switching costs.

LLMs will become hard disk drives or SSDs. They are all commodity today.
Open source video
Hardware vendors pounded their chests about 10x servers, but nobody cares about raw server specs anymore; intelligence moved to software.
Open source video
Architecturemedium noveltymoderate evidence

Enterprise AI infrastructure providers are framed as at risk of being absorbed by LLM providers unless they abstract data and compute sovereignty — i.e., the defensible abstraction is jurisdictional control over where data and compute execute, not the model itself.

Why it matters

Points to a specific architectural boundary with commercial value: a sovereignty/control layer (residency guarantees, cryptographic audit trails, distributed secure compute) that sits between enterprise workloads and commodity model/cloud providers.

Generalization

Compliance and jurisdiction constraints create architectural boundaries that commoditized compute cannot cross, making them a source of platform stickiness independent of model quality.

AI infra providers risk being wiped out by LLM providers unless they abstract data and compute sovereignty.
Open source video
Enterprise clients in regions like Singapore requiring local data residency guarantees.
Open source video
Architecturemedium noveltymoderate evidence

Reliability of commodity cloud is treated as insufficient on its own: regional and provider outages are cited as evidence that businesses need overlay solutions layered on top of commodity cloud providers, mirroring how AWS S3 runs on commodity hardware with software layers supplying the intelligence.

Why it matters

Argues for designing agentic and data systems with an independent overlay/control plane that can survive provider-region failure, rather than assuming the hyperscaler's availability envelope is the system's availability envelope.

Generalization

Commoditized substrate plus a software overlay that owns routing, replication, and correctness is a recurring pattern for turning unreliable/cheap hardware into reliable services.

AWS outages and regional outages prove that businesses need overlay solutions running on top of commodity cloud providers.
Open source video
AWS S3 running on pure commodity hardware while software layers handle intelligence and storage.
Open source video
Practicemedium noveltyweak evidence

Open-weight models are expected to win by analogy to open-source software disrupting proprietary enterprise software, implying that enterprises should keep architectures agnostic to any specific foundational model.

Why it matters

Model agnosticism becomes a concrete design constraint (adapter boundaries, portable prompts/eval harnesses, swappable inference backends) rather than a hedge taken on faith.

Generalization

Designing for substitutability of a rapidly commoditizing dependency reduces migration cost when the winning supplier changes.

Open weight models democratize access and mirror open-source software disruption against proprietary infrastructure.
Open source video
Enterprises should design architectures that remain agnostic to specific foundational models.
Open source video

Deep dives

4

Reproducing the Python-vs-Rust agent power gap under controlled workloads

Research question

Under identical agent workloads and hardware, what is the measured CPU energy per completed agent-task in Python versus Rust, and how much of the gap is interpreter overhead versus model/tool wait states?

Why

If the 20x magnitude holds, agent capacity planning and per-task cost models are dominated by runtime choice and invisible to token-centric observability.

A non-trivial agent in Python uses ~15 watts, whereas an equivalent agent implemented in Rust consumes 0.7 watts.
Open source video
Python is the most inefficient language ever made by mankind.
Open source video
Source video

Locating the fleet-scale crossover where runtime power dominates inference power

Research question

At what agent concurrency and duty cycle does runtime power consumption exceed inference power consumption as the dominant data-center load driver?

Why

A crossover point turns language/runtime choice from style into a hard capacity constraint, and tells teams when to migrate off Python.

At scale across millions of agents, the energy footprint difference between Python and Rust creates massive data center power crises.
Open source video
Source video

Testing whether LLM inference pricing actually trends toward commodity

Research question

Do inference prices and gross margins across LLM providers decline toward commodity levels as storage hardware did?

Why

This determines whether enterprise value shifts from model ownership to routing, compliance, and operations layers.

LLMs will become commodity hard disk drives or SSDs.
Open source video
Infrastructure costs for LLMs are super high, but pricing pressure will force costs super low.
Open source video
Source video

Measuring whether overlay control planes outperform native hyperscaler availability

Research question

Do overlay architectures measurably improve availability and recovery during provider-region outages compared with native multi-region primitives?

Why

If overlays do not beat native availability, the extra control plane cost is unjustified; if they do, reliability becomes a defensible abstraction.

The cloud did not become commoditized right away; it was expensive until overlay services emerged.
Open source video
Source video

Article ideas

4

Your Agent's Power Bill Is a Language Decision

Agent runtime substrate is a first-order cost and capacity driver, so teams should measure joules per agent-task before optimizing model calls or inference batching.

Angle

Benchmark-driven argument that runtime language belongs in architecture reviews, not just performance tuning.

Source video

The 20x Agent Power Gap Is a Fleet-Scale Cliff, Not a Micro-Optimization

Python agent overhead is tolerable at pilot concurrency but becomes an operational blocker at projected fleet scale; architecture decisions should be evaluated at steady-state agent counts.

Angle

Capacity-planning lens on why pilot benchmarks mislead for agent fleets.

Source video

LLM Commoditization Means Your Defensibility Is the Overlay

If LLMs follow the HDD/SSD commodity trajectory, durable value moves to routing, compliance, and operational abstractions above the model layer.

Angle

Investment and architecture argument against deep coupling to any single model provider.

Source video

Overlay or Outage: Why Commodity Cloud Availability Is Not Your Agent's Availability

Agentic systems should not inherit the hyperscaler's availability envelope; an independent overlay control plane is required to convert commodity cloud reliability into service reliability.

Angle

Reliability-engineering counterpoint to 'the cloud is someone else's uptime problem.'

Source video

Project ideas

4

Agent Power Crucible

beyond-evals

For identical non-trivial agent workloads with mocked model calls, a Rust implementation uses at least 10x fewer CPU joules per completed task than Python, and the gap is dominated by interpreter overhead rather than I/O wait.

Proof of concept

Implement the same agent loop (tool calls, memory, retries) in Python and Rust; instrument CPU package energy (RAPL) while executing N tasks with model calls mocked to fixed latency.

Measurement

Joules per task, CPU time per task, context switches per task, and ratio Python/Rust; decomposition of energy in compute versus wait.

Source video

Crossover Instrument

movement-lab

There is a measurable agent concurrency and duty cycle at which runtime power exceeds inference power, and this crossover can be predicted from per-agent-hour and per-token energy models.

Proof of concept

Build an instrumented two-runtime agent deployment with configurable concurrency; mock or proxy model calls to fixed energy per token; sweep fleet size and duty cycle.

Measurement

Crossover concurrency, runtime vs inference joules per hour, total facility power proxy, and PUE sensitivity.

Source video

Residency-Attested Agent Runtime

gatehouse

Per-action cryptographic attestation for data residency and compute location can be added to an agent runtime with less than 10% latency overhead while enforcing jurisdictional pinning.

Proof of concept

Prototype a regional execution broker that pins agent steps to a jurisdiction and emits signed attestations for data placement and inference location; benchmark against centralized baseline.

Measurement

Added p50/p99 latency, throughput delta, key-management overhead, and audit coverage completeness.

Source video

Model Swap Bench

beyond-evals

A model-agnostic agent harness reduces foundation-model migration cost, but behavior and tool-call validity shift measurably even when the adapter interface is fixed.

Proof of concept

Define a fixed agent harness with a model adapter; run the same eval suite against multiple open-weight and proprietary models; record the diff required to swap.

Measurement

Files/lines changed per swap, eval pass-rate delta, tool-call validity rate, and cost per task per model.

Source video

Architectural implications

3

Agent execution substrate is a cost/power lever comparable to model choice.

Before

Python orchestrates and executes the full agent loop end-to-end.

After

Performance-critical agent execution loops move to a compiled runtime (e.g., Rust) with Python retained for orchestration, tooling, and glue.

Consequence

Requires measurable power/CPU-per-agent telemetry and a benchmark harness to justify and validate the migration; risks splitting the codebase across two languages.

Source video

Value accrues to the abstraction layer above commoditized models and hardware.

Before

Application logic is written directly against a specific LLM provider's API and hosted on a single cloud region.

After

A software abstraction layer owns model routing, data placement, and audit trails over commodity models and cloud.

Consequence

Provider swaps and jurisdiction changes become configuration rather than rewrites, at the cost of an extra abstraction to maintain and debug.

Source video

Enterprise requirements impose residency and compute-location constraints on agent infrastructure.

Before

All agent state, context, and inference run in a single default region.

After

Compute and data are distributed and pinned to jurisdictional boundaries with cryptographically secure audit trails.

Consequence

True data/compute sovereignty becomes the product boundary that LLM providers are least likely to cross, but adds latency, replication, and key-management complexity.

Source video

Tradeoffs and failure modes

4

Python as the agent runtime

Benefit

Python is the incumbent for agent frameworks, tooling, and model SDKs, minimizing time-to-first-agent and leveraging the widest ecosystem.

Cost or risk

Claimed order-of-magnitude higher per-agent power consumption that, at fleet scale, is asserted to create data-center power crises.

A non-trivial agent written in Python consumes about 15 watts, whereas the equivalent agent implemented in Rust consumes 0.7 watts.
Open source video
Source video

Building AI infrastructure under LLM provider consolidation

Benefit

Infrastructure players can sell operational efficiency and abstraction over commodity hardware and models as a managed service.

Cost or risk

Providers can absorb the layer above them; an infra startup without a sovereignty abstraction is exposed to being wiped out.

AI infra providers risk being wiped out by LLM providers unless they abstract data and compute sovereignty.
Open source video
Source video

Data and compute sovereignty as a feature

Benefit

Satisfies enterprise compliance (e.g., local data residency) and yields high-margin B2B contracts that commodity clouds cannot easily serve.

Cost or risk

Distributed, cryptographically secure compute is materially harder to operate than centralized commodity infrastructure.

Infrastructure players must offer distributed, cryptographically secure compute platforms to satisfy enterprise compliance.
Open source video
Source video

Overlay services on commodity cloud

Benefit

Overlay layers convert cheap, unreliable commodity cloud into reliable service, decoupling application availability from provider outages.

Cost or risk

An additional control plane to build, operate, and keep consistent with the substrate it abstracts.

AWS outages and regional outages prove that businesses need overlay solutions running on top of commodity cloud providers.
Open source video
Source video

Open questions

4

Is the ~15W vs ~0.7W per-agent power gap reproducible under realistic agent workloads, and how much of it is interpreter CPU overhead versus I/O-bound model/tool waiting?

Why unresolved

The summary reports a single comparative figure with no methodology, workload description, or measurement conditions.

Research direction

Build a controlled benchmark: identical non-trivial agent (tool calls, memory, loop) implemented in Python and Rust, measuring CPU joules per agent-task while holding model calls constant, and decomposing time in interpreter versus wait states.

Source video

At what fleet size does runtime power consumption overtake inference power consumption as the dominant driver of data-center load?

Why unresolved

The summary asserts a crisis at millions of agents but gives no crossover point, agent duty cycle, or per-call energy reference.

Research direction

Model joules-per-agent-hour for the runtime layer and per-token for inference across concurrency levels to find the crossover, then validate against an instrumented deployment.

Source video

Can jurisdictional data/compute sovereignty be delivered with cryptographic audit trails without unacceptable latency or operational cost?

Why unresolved

The summary presents sovereignty as the required abstraction but gives no mechanism, overhead figure, or implementation detail.

Research direction

Prototype a residency-enforcing compute layer with per-action cryptographic attestation and measure added latency, throughput, and key-management burden versus a centralized baseline.

Source video

Which open-weight models will dominate enterprise deployments over the next three years, and does model agnosticism actually reduce migration cost in practice?

Why unresolved

The summary offers a historical analogy to open-source disruption, not evidence about model-level substitution costs.

Research direction

Track the effort required to swap foundational models in a fixed agent harness across eval suites, measuring changes in behavior, tool-call validity, and cost.

Source video

Key claims

8
comparativeVerification needed

A non-trivial agent consumes ~15W in Python versus ~0.7W in Rust.

Evidence

A non-trivial agent written in Python consumes about 15 watts, whereas the equivalent agent implemented in Rust consumes 0.7 watts.

Question

What is the measured CPU power per agent-task for identical Python and Rust agent implementations under the same workload and hardware?

Source video
opinionVerification not requested

Python is the most inefficient language ever made by mankind.

Evidence

Python is the most inefficient language ever made by mankind.

Source video
predictionVerification needed

LLMs will commoditize into hard-drive/SSD-like commodity layers.

Evidence

LLMs will become hard disk drives or SSDs. They are all commodity today.

Question

Do inference prices and margins trend toward commodity levels across providers over time, as they did for storage hardware?

Source video
causalVerification needed

At fleet scale, the Python-vs-Rust runtime gap causes data-center power crises.

Evidence

At scale across millions of agents, the energy footprint difference between Python and Rust creates massive data center power crises.

Question

At what concurrency does agent runtime power consumption become a binding constraint on data-center PUE and capacity?

Source video
causalVerification needed

AI infrastructure providers risk being eliminated by LLM providers unless they abstract data and compute sovereignty.

Evidence

AI infra providers risk being wiped out by LLM providers unless they abstract data and compute sovereignty.

Question

Do AI infrastructure vendors without a sovereignty/residency abstraction actually lose share to vertically integrated model providers?

Source video
causalVerification needed

Regional cloud outages demonstrate that businesses need overlay solutions on commodity cloud providers.

Evidence

AWS outages and regional outages prove that businesses need overlay solutions running on top of commodity cloud providers.

Question

Do overlay architectures measurably improve availability versus native multi-region provider primitives during real outage events?

Source video
causalVerification needed

Intelligence and value have shifted to software, making raw server specifications irrelevant.

Evidence

Hardware vendors pounded their chests about 10x servers, but nobody cares about raw server specs anymore; intelligence moved to software.

Question

Do buyers of AI infrastructure actually select on software/abstraction capability rather than hardware specifications?

Source video
factualVerification needed

Combolt acquired Hedwig for $225 million.

Evidence

Combolt acquired Hedwig for $225 million.

Question

Was there an acquisition of Hedwig for $225 million, and by which acquirer?

Source video

Connections

4