AI Engineer · Published 2025-06-25

Fun stories from building OpenRouter and where all this is going - Alex Atallah, OpenRouter

privateReview: unreviewedConfidence: high
openrouterinferencellmmiddlewaremarketplacellm-routinginference-middlewaremulti-modelmodel-marketplacedistillationstreamingMCPcost-optimization
Open on YouTube ↗

Summary

Overview

  • Speaker: Alex Atallah
  • Channel: AI Engineer
  • Main topic: The founding story and evolution of OpenRouter, the emergence of multi-model LLM workflows, inference-native middleware, and the future of AI software markets.
  • Purpose: To share historical stories from building OpenRouter, discuss industry shifts toward multi-model architectures and inference commoditization, and detail technical approaches like inference-native plugins. Alex Atallah, CEO of OpenRouter, reflects on the early days of building OpenRouter in early 2023, the emergence of open-source models (like LLaMA and Alpaca), the transition from simple model aggregation to an optimized LLM marketplace, and technical challenges like building inference-native middleware (plugins) with near-zero latency.

Topic Map

The Origins of OpenRouter and Early Market Context

  • Explanation: Discusses the landscape at the start of 2023 following the release of ChatGPT, exploring whether inference would be a winner-take-all market dominated by OpenAI.
  • Key claims:
    • Inference may be the largest market ever in software.
    • OpenAI was initially seen as the far-and-away leading model.
    • Early user demand for alternative models was driven by moderation and censorship restrictions on proprietary APIs.
  • Examples:
    • Users running D&D roleplay and fictional stories getting banned by OpenAI moderation policies in January 2023.
  • Terminology:
    • Inference
    • Moderation
    • OpenAI
    • OpenRouter
  • Why it matters: Establishes the initial hypothesis and market problem that led to the creation of OpenRouter as a multi-model routing tool.

The Open Source Race and Model Distillation

  • Explanation: Traces the open-source movement starting with LLaMA 1 in February 2023 and Stanford Alpaca in March 2023, representing the first successful distillation of style and knowledge.
  • Key claims:
    • LLaMA 1 outperformed GPT-3 on many benchmarks while being a fraction of the size.
    • Stanford Alpaca fine-tuned LLaMA 1 on GPT-3 outputs for under $600.
    • Distillation unlocked knowledge being distilled into software without massive training budgets.
  • Examples:
    • Stanford Alpaca project in March 2023.
    • Meta's LLaMA 1 release.
  • Terminology:
    • LLaMA
    • Alpaca
    • Distillation
    • Fine-tuning
    • Open weights
  • Why it matters: Highlighted the proliferation of open-source models and the need for developers to easily access and compare multiple models.

Window.ai and Transition to an LLM Marketplace

  • Explanation: Explains the launch of Window.ai in April 2023 as a Chrome extension allowing bring-your-own-model (BYOM) functionality, leading to the launch of OpenRouter in May 2023.
  • Key claims:
    • Developers and users needed a way to bring their choice of model to web apps.
    • OpenRouter evolved from a simple model aggregator into an optimized marketplace for language models offering better prices, better uptime, and no subscriptions.
  • Examples:
    • Window.ai Chrome extension swapping OpenAI for open-source deployments in web apps.
  • Terminology:
    • Window.ai
    • BYOM
    • Plasma
    • Marketplace
  • Why it matters: Demonstrates the evolution of OpenRouter from an experiment into a centralized marketplace with millions of monthly active users and billions of tokens.

Inference-Native Middleware and Plugins

  • Explanation: Details the technical challenge of providing MCP-like capabilities and middleware for inference, introducing OpenRouter's plugin system for input/output transformations.
  • Key claims:
    • Unlike MCPs, OpenRouter plugins allow transformations on both inputs and outputs in real-time streams.
    • Web search and tool calling can be dynamically injected into any language model without requiring custom client implementations.
  • Examples:
    • WebSearchPlugin augmenting language model outputs with web citations in real-time streams.
  • Terminology:
    • Middleware
    • Plugins
    • MCP
    • SSE
    • Tool calling
  • Why it matters: Solves complex routing, normalization, caching, and stream cancellation challenges across over 60 active providers and 400 models.

Key Points

The future of AI is multi-model

  • Explanation: Customers and developers do not rely on a single model; they leverage different models for different purposes to unlock massive performance and cost gains.
  • Evidence: OpenRouter token market share data showing diversified usage across Google Gemini, Anthropic Claude, OpenAI, and DeepSeek.
  • Practical implication: Developers should build applications with multi-model abstraction layers to avoid vendor lock-in.

Inference is a commodity

  • Explanation: Hyper-scalers provide the same foundational inference capability at varying prices and latencies, making inference a commodity.
  • Evidence: Comparison of LLM pricing and throughput across multiple providers on OpenRouter.
  • Practical implication: Application developers should optimize cost and uptime by routing requests dynamically across providers.

Inference cost will become the dominant operating expense for knowledge companies

  • Explanation: As AI agents and knowledge workflows scale, inference cost will dwarf traditional compute infrastructure costs.
  • Evidence: Month-over-month growth of token usage and active models on OpenRouter.
  • Practical implication: Selecting, routing, and orchestrating inference efficiently is critical for unit economics.

Frameworks, Models & Processes

OpenRouter Plugin Architecture

  • How it works: An inference-native middleware framework running within OpenRouter that intercepts requests and responses, allowing plugins to modify inputs and augment streaming outputs.
  • Components:
    • BasePlugin
    • PluginCompletionGenerator
    • Input/Output transformation hooks
    • Streaming Server-Sent Events (SSE) deserializer
  • When to use: When building applications that need real-time web search, citations, or tool calling across heterogeneous LLM providers.

Examples & Case Studies

Stanford researchers created Alpaca by fine-tuning LLaMA 1 on GPT-3 outputs for under $600.

  • Illustrates: The power of model distillation and transfer of style and knowledge from large models to small models.
  • Lesson: Open-source models can achieve high performance at a fraction of the cost of proprietary training.

Actionable Takeaways

  • Immediate:
    • Use single API endpoints with near-zero switching costs between models.
    • Utilize fine-grained privacy controls and API-level overrides.
  • Strategic:
    • Adopt a multi-model strategy to insulate your application from upstream provider outages and price hikes.
    • Invest in inference-native middleware and orchestration.
  • Questions to investigate:
    • Will intelligence remain commoditized or will proprietary architectural breakthroughs create new moats?
    • How will multi-modal transfusion models change software interfaces?

Claims Worth Verifying

  • OpenRouter routing latency is down to about 30 milliseconds using custom cache setups. (Technical Performance)
  • Stanford Alpaca fine-tuned LLaMA 1 for less than $600 total cost. (Historical Cost)

Notable Quotes

"Inference may be the largest market ever in software." "The future is multi-model."

Compressed Summary

  • OpenRouter began as an experiment in early 2023 to explore multi-model routing and open-source models.
  • The AI ecosystem shifted from a winner-take-all OpenAI monopoly to a diverse multi-model marketplace.
  • OpenRouter provides a single unified API with over 400 models and 60+ providers.
  • Inference-native middleware (plugins) enables real-time input and output transformations during streaming.
  • Keywords: openrouter, inference, llm, middleware, marketplace
  • Core insight: As AI inference becomes a commodity and the ecosystem shifts to multi-model architectures, intelligent routing and inference-native middleware are critical for software developers.

Core insights

6
Architecturehigh noveltymoderate evidence

OpenRouter's plugin system is inference-native middleware: it intercepts both requests and streaming responses inside the proxy, so web search, tool calling, citations, and output augmentation can be injected into any model without per-model client-side implementations. This differs from MCP-style approaches, which act on the client side and cannot transform outputs mid-stream.

Why it matters

It relocates capability injection (tool calling, grounding, citations) from the application/client layer to the inference proxy layer, so a single integration works uniformly across dozens of providers and hundreds of models. This changes where responsibilities live in an agent stack and eliminates a per-model integration matrix.

Generalization

Capabilities that must act on model outputs can be centrally hosted at the inference boundary rather than reimplemented in each client, turning a per-app integration problem into a platform-level middleware problem.

Unlike MCPs, OpenRouter plugins allow transformations on both inputs and outputs in real-time streams.
Open source video
Web search and tool calling can be dynamically injected into any language model without requiring custom client implementations.
Open source video
An inference-native middleware framework running within OpenRouter that intercepts requests and responses, allowing plugins to modify inputs and augment streaming outputs.
Open source video
Empirical Resultmedium noveltymoderate evidence

Real-world model usage is empirically diversified across vendors rather than concentrated on a single frontier model; token market share is spread across Google Gemini, Anthropic Claude, OpenAI, and DeepSeek, and developers deliberately route different tasks to different models for performance and cost gains.

Why it matters

It challenges the 'one best model' assumption and implies applications should be built behind a model-abstraction layer to gain cost and performance benefits and avoid vendor lock-in.

Generalization

For any rapidly-commoditizing dependency, abstracting behind a routing layer is safer and cheaper than committing to a single supplier, because workload-specific selection beats a single global default.

OpenRouter token market share data showing diversified usage across Google Gemini, Anthropic Claude, OpenAI, and DeepSeek.
Open source video
Customers and developers do not rely on a single model; they leverage different models for different purposes to unlock massive performance and cost gains.
Open source video
Mental Modelmedium noveltymoderate evidence

Inference is treated as a commodity: hyperscalers deliver the same underlying capability at varying prices and latencies, so value shifts to routing logic that dynamically picks providers for cost and uptime.

Why it matters

If the capability itself is commoditized, differentiation and margin move to orchestration, routing, failover, and cost controls rather than to the model endpoint. Engineers should treat the model provider as a swappable resource, not a platform commitment.

Generalization

Where a capability becomes fungible, the durable engineering value migrates to the layer that selects, composes, and normalizes the suppliers of that capability.

Hyper-scalers provide the same foundational inference capability at varying prices and latencies, making inference a commodity.
Open source video
Application developers should optimize cost and uptime by routing requests dynamically across providers.
Open source video
Predictionmedium noveltyweak evidence

As agents and knowledge workflows scale, inference cost is predicted to dwarf traditional compute infrastructure costs, making routing and orchestration decisions the dominant lever on unit economics.

Why it matters

If inference becomes the dominant operating expense, then token-level routing, caching, and orchestration are not optimizations but core cost-architecture decisions that determine whether an AI product is viable.

Generalization

For systems whose runtime cost is dominated by a metered external resource, the orchestration layer that controls consumption becomes as important as the application logic itself.

As AI agents and knowledge workflows scale, inference cost will dwarf traditional compute infrastructure costs.
Open source video
Selecting, routing, and orchestrating inference efficiently is critical for unit economics.
Open source video
Empirical Resultlow noveltymoderate evidence

Distillation collapsed the cost of transferring style and knowledge into small open models: Stanford fine-tuned LLaMA 1 on GPT-3 outputs for under $600, and the resulting models were competitive on benchmarks.

Why it matters

It shows that capability transfer can be done by a small team at trivial budget, which is a direct driver of the open-weights proliferation that makes multi-model routing viable in the first place.

Generalization

Distilling a strong teacher into a small student can produce useful, cheaply-hosted models, which shifts the economics of running specialized workloads locally or on cheap inference.

Stanford Alpaca fine-tuned LLaMA 1 on GPT-3 outputs for under $600.
Open source video
Distillation unlocked knowledge being distilled into software without massive training budgets.
Open source video
Mechanismmedium noveltymoderate evidence

The non-obvious engineering burden of an inference marketplace is not model access but cross-provider normalization, caching, and stream cancellation, and routing latency can be driven down to roughly 30 milliseconds with custom caching.

Why it matters

It identifies the real hard problems in a multi-provider inference layer — heterogeneous protocol/response normalization, streaming lifecycle management such as cancellation, and cache-driven latency — which are the failure-prone parts any team building similar middleware should plan for.

Generalization

Aggregating heterogeneous streaming backends requires normalizing their wire formats and correctly propagating cancellation; latency budgets then depend on caching rather than on faster origin calls.

Solves complex routing, normalization, caching, and stream cancellation challenges across over 60 active providers and 400 models.
Open source video
OpenRouter routing latency is down to about 30 milliseconds using custom cache setups.
Open source video

Deep dives

5

Inference-native middleware as a substitute for client-side capability layers (MCP contrast)

Research question

Can input/output stream transformation at the inference proxy fully replace client-side capability injection such as MCP, and under what conditions does the proxy boundary fail?

Why

If capabilities like grounding, citations, and tool calling can be injected once at the proxy for any model, the per-model integration matrix collapses and the abstraction boundary of the agent stack shifts — a structural decision every AI platform team will face.

Unlike MCPs, OpenRouter plugins allow transformations on both inputs and outputs in real-time streams.
Open source video
Web search and tool calling can be dynamically injected into any language model without requiring custom client implementations.
Open source video
An inference-native middleware framework running within OpenRouter that intercepts requests and responses, allowing plugins to modify inputs and augment streaming outputs.
Open source video
Source video

Latency and correctness budget of plugin-augmented streaming output

Research question

How much latency and how many token-boundary or cancellation defects does mid-stream output augmentation add versus a plain stream, and does it scale with provider count?

Why

The claim of near-zero latency for proxy-level output transformation is the load-bearing assumption behind inference-native plugins; if augmentation breaks token boundaries or cancellation under load, the whole middleware pattern becomes unsafe on the critical path.

Augmenting streamed outputs forces the proxy to own SSE parsing, partial-output handling, and cancellation semantics.
Open source video
Unlike MCPs, OpenRouter plugins allow transformations on both inputs and outputs in real-time streams.
Open source video
Source video

Normalization and conformance contracts across heterogeneous providers

Research question

What minimal normalized request/response and stream-lifecycle contract can be enforced across 60+ providers and 400 models, and which providers violate it?

Why

The stated hard problems are normalization, caching, and stream cancellation rather than model access, so the abstraction layer's real engineering risk and test burden lives in provider heterogeneity — exactly the part teams underestimate when they adopt multi-model routing.

Solves complex routing, normalization, caching, and stream cancellation challenges across over 60 active providers and 400 models.
Open source video
OpenRouter routing latency is down to about 30 milliseconds using custom cache setups.
Open source video
Source video

Is inference commoditization a stable equilibrium or a temporary state?

Research question

Does token-share concentration re-concentrate after each frontier release, and does benchmark/price delta predict the degree of re-concentration?

Why

If commoditization is temporary, then the value of a routing layer is cyclical and applications should keep the abstraction thin; if it is stable, routing is durable infrastructure. The answer changes how much investment the orchestration layer deserves.

OpenRouter token market share data showing diversified usage across Google Gemini, Anthropic Claude, OpenAI, and DeepSeek.
Open source video
Hyper-scalers provide the same foundational inference capability at varying prices and latencies, making inference a commodity.
Open source video
Source video

Measuring routing overhead: what does 30 milliseconds actually mean?

Research question

What is the distribution (p50/p99, cache-hit versus cold route) of proxy-added latency, and how does it compare to the origin model's own time-to-first-token?

Why

A single 30 ms figure hides whether the proxy layer is economically negligible or a measurable tax; the measurement methodology determines whether 'near-zero latency' is an engineering fact or a marketing number.

OpenRouter routing latency is down to about 30 milliseconds using custom cache setups.
Open source video
How is the 30 ms measured (p50/p99, cache-hit only), and what is the penalty on cache misses or cold routing?
Open source video
Source video

Article ideas

4

Put Your Agent's Capabilities at the Inference Boundary, Not in the Client

Capabilities that must act on model outputs — grounding, citations, tool calling — should be hosted at the inference proxy because they are the only place that can transform streaming outputs uniformly; client-side approaches leave an unserved class of mid-stream transformations that must otherwise be reimplemented per app and per model.

Angle

Contrast proxy-level middleware with client-boundary tooling on the specific axis of output transformation, then trace the consequence: the proxy becomes critical-path infrastructure governed by streaming correctness.

Source video

Inference Is a Commodity — So the Margin Lives in the Router

Once the same foundational capability is available from multiple hyperscalers at varying price and latency, differentiation stops living at the model endpoint and moves into routing, failover, caching, and cost control; committing to a single provider is therefore a unit-economics decision, not just an architectural one.

Angle

Follow the money: capability becomes fungible, so the durable engineering asset is the selection and normalization layer — and that layer must itself stay fast and testable.

Source video

The Unsexy Work of an LLM Marketplace: Normalization, Caching, and Stream Cancellation

The hard part of aggregating inference is not model access but owning the wire-format normalization, streaming lifecycle, and cancellation semantics of dozens of heterogeneous providers; teams that treat these as plumbing will discover they are the product.

Angle

Failure-mode-first: enumerate where multi-provider streaming breaks and what contracts a proxy must enforce, using the 400-model/60-provider scale as the forcing function.

Source video

A Sub-$600 Fine-Tune Took the Monopoly Out of Model Quality

Cheap distillation of a strong teacher into small open weights was the precondition for the multi-model ecosystem: without a supply of capable, cheaply-hosted students, routing layers would have nothing worth routing to, so the routing thesis is downstream of the distillation result.

Angle

Trace the causal chain from distillation economics to marketplace viability, and ask what breaks the chain if distillation stops working on the current frontier.

Source video

Project ideas

4

Proxy-Injection Parity Harness

gatehouse

Tool calling and web grounding injected by a proxy into models without native tool-calling support achieve task success within 10% of natively tool-calling models, at a bounded added time-to-first-token.

Proof of concept

Build a proxy that intercepts requests and augments streaming responses with retrieved context and citations, then run one fixed tool-using task suite across a no-tool open-weights model with proxy injection versus a native tool-calling model without it.

Measurement

Task success rate, added p50/p99 latency to first token, and citation correctness rate for injected versus native configurations.

Source video

Task-Aware Routing Cost Benchmark

movement-lab

Task-aware dynamic provider routing reduces cost per solved task by at least 30% versus a single fixed frontier model at equal or better quality.

Proof of concept

Implement a router that classifies incoming requests and selects providers by price/latency with failover, then replay a mixed workload of simple extraction, long-context reasoning, and code tasks against both the router and a single-model baseline.

Measurement

Cost per solved task, quality score per task class, and failover-induced error rate.

Source video

Multi-Provider Stream Conformance Suite

beyond-evals

At least one provider among a representative multi-provider set violates normalized stream cancellation or partial-output contract under load, and the violation is discoverable only by structured conformance testing.

Proof of concept

Define a normalized request/response and stream-lifecycle contract, then implement a test suite that exercises routing, failover, mid-stream cancellation, and token-boundary integrity across multiple providers.

Measurement

Number of providers failing each conformance check, shape of divergences, and reproducibility rate of cancellation defects under concurrent load.

Source video

Cheap-Student Narrow-Task Substitution Test

new

On a single narrow task, a cheaply distilled small open model matches a routed frontier model on accuracy at an order-of-magnitude lower cost per correct answer.

Proof of concept

Distill a small open-weights model on teacher outputs for one narrow task within a sub-$600 compute budget, then compare it head-to-head against a routed frontier model on a held-out evaluation set.

Measurement

Cost per correct answer, accuracy delta, and break-even task volume at which distillation amortizes.

Source video

Architectural implications

3

Capability injection (web search, citations, tool calling) moves from the client to the inference proxy.

Before

Each application implements tool calling, web grounding, and citations per model/provider on the client side.

After

A middleware layer inside the inference proxy transforms inputs and augments streaming outputs uniformly for any model.

Consequence

Applications get new capabilities without client changes, but the proxy becomes a critical-path dependency whose latency and correctness govern every dependent app.

Source video

The model endpoint is treated as a swappable, commoditized resource.

Before

Architecture centers on a single provider's SDK and semantics, creating vendor lock-in.

After

Applications sit behind a model-abstraction/routing layer that selects providers by cost, latency, and uptime.

Consequence

Failover and cost optimization become automatic, but the team must maintain a testing matrix across many models and providers and accept lowest-common-denominator semantics.

Source video

Streaming lifecycle semantics become a first-class platform concern.

Before

Each app manages its own SSE parsing, partial output handling, and stream termination.

After

The proxy normalizes streams and handles cancellation across 60+ providers and 400 models.

Consequence

Stream correctness (partial results, cancellations, output augmentation mid-stream) must be validated at the platform layer, since apps can no longer reason about raw provider behavior.

Source video

Tradeoffs and failure modes

4

Inference-native plugins on both inputs and outputs

Benefit

Web search, citations, and tool calling can be injected into any model without client-side implementation.

Cost or risk

Transforming streaming outputs in real time adds latency and complexity, and requires correctly handling partial/streamed responses and cancellation across all providers.

Unlike MCPs, OpenRouter plugins allow transformations on both inputs and outputs in real-time streams.
Open source video
Source video

Multi-model routing

Benefit

Unlocks performance and cost gains and insulates applications from provider outages and price hikes.

Cost or risk

Requires cross-provider normalization, a broader test matrix, and management of heterogeneous semantics across many models.

Solves complex routing, normalization, caching, and stream cancellation challenges across over 60 active providers and 400 models.
Open source video
Source video

Inference as a commodity

Benefit

Price and latency competition across hyperscalers benefits buyers via dynamic provider selection.

Cost or risk

Commoditization compresses differentiation at the model layer, so value and margin depend on routing/orchestration that must itself be kept fast and reliable.

Hyper-scalers provide the same foundational inference capability at varying prices and latencies, making inference a commodity.
Open source video
Source video

Routing latency via custom caches

Benefit

Reported routing latency around 30 milliseconds reduces the overhead of adding a proxy layer.

Cost or risk

Latency depends on custom caching infrastructure that must stay correct as provider behavior and model catalogs change.

OpenRouter routing latency is down to about 30 milliseconds using custom cache setups.
Open source video
Source video

Open questions

4

Will intelligence remain commoditized, or will proprietary architectural breakthroughs re-create durable moats at the model layer?

Why unresolved

The summary presents multi-model diversification and commoditization as observations but does not resolve whether this is a stable equilibrium or a temporary state before a differentiated breakthrough.

Research direction

Track token-share concentration over time and correlate with benchmark/price deltas of new frontier releases to detect re-concentration.

Source video

How do plugin-based input/output transformations interact with streaming latency and correctness at scale?

Why unresolved

The summary claims near-zero latency and stream output augmentation, but does not quantify the added latency or describe failure modes when plugins act on partial streams.

Research direction

Benchmark plugin-augmented versus plain streams for added latency, token-boundary correctness, and cancellation behavior.

Source video

How will multi-modal 'transfusion' models change software interfaces?

Why unresolved

Raised as an open question in the summary without any mechanism or evidence.

Research direction

Prototype interfaces where a single model routes across modalities and measure where the abstraction boundary should sit.

Source video

What is the right abstraction boundary and testing strategy for a model-abstraction layer spanning hundreds of models?

Why unresolved

The summary advocates multi-model abstraction but provides no guidance on normalization contracts or how to test across 400 models and 60+ providers.

Research direction

Define a normalized request/response contract and a conformance test suite that exercises routing, failover, and stream cancellation per provider.

Source video

Key claims

7
comparativeVerification needed

OpenRouter plugins can transform both inputs and outputs in real-time streams, unlike MCPs.

Evidence

Unlike MCPs, OpenRouter plugins allow transformations on both inputs and outputs in real-time streams.

Question

What exactly does MCP do at the client boundary versus OpenRouter's proxy-level plugins, and can MCP-style tooling also modify streamed outputs?

Source video
factualVerification needed

Web search and tool calling can be injected into any language model via the proxy without custom client implementations.

Evidence

Web search and tool calling can be dynamically injected into any language model without requiring custom client implementations.

Question

Does injection work uniformly for models lacking native tool-calling support, and with what latency or reliability penalty?

Source video
factualVerification needed

OpenRouter routing latency is about 30 milliseconds using custom cache setups.

Evidence

OpenRouter routing latency is down to about 30 milliseconds using custom cache setups.

Question

How is the 30 ms measured (p50/p99, cache-hit only), and what is the penalty on cache misses or cold routing?

Source video
factualVerification needed

Real usage is diversified across Gemini, Claude, OpenAI, and DeepSeek rather than concentrated on one model.

Evidence

OpenRouter token market share data showing diversified usage across Google Gemini, Anthropic Claude, OpenAI, and DeepSeek.

Question

What is the actual concentration (e.g., top-1 share) and does it reflect deliberate task routing or just experimentation?

Source video
opinionVerification needed

Inference is a commodity because hyperscalers provide the same capability at varying prices and latencies.

Evidence

Hyper-scalers provide the same foundational inference capability at varying prices and latencies, making inference a commodity.

Question

Are model outputs substitutable enough across providers to justify treating inference as truly fungible for quality-sensitive workloads?

Source video
predictionVerification needed

Inference cost will become the dominant operating expense for knowledge companies.

Evidence

As AI agents and knowledge workflows scale, inference cost will dwarf traditional compute infrastructure costs.

Question

Under what workload and pricing assumptions does inference cost overtake other infra costs, and how sensitive is this to caching and routing?

Source video
factualVerification needed

Stanford Alpaca fine-tuned LLaMA 1 on GPT-3 outputs for under $600.

Evidence

Stanford Alpaca fine-tuned LLaMA 1 on GPT-3 outputs for under $600.

Question

Does the <$600 figure include only training compute, and how did the resulting model compare to GPT-3 in practical (not benchmark) use?

Source video

Connections

5