Lenny's Podcast · Published 2026-10-11

Why every company now needs to think and operate like a lab team | Josh Woodward (VP Google Labs)

privateReview: unreviewedConfidence: high
google labsartificial intelligenceproduct-market fitinnovationexperimentationproduct-market-fitorganizational-designcapability-gapqualitative-evalsexploration-exploitationunlearning-ratelab-team
Open on YouTube ↗

Summary

Overview

  • Speaker: Josh Woodward
  • Channel: Lenny's Podcast
  • Main topic: How every company needs to think and operate like a lab team to keep pace with rapid AI innovation.
  • Purpose: To provide insights, frameworks, and practical lessons from Google Labs on running experimental teams and building AI-driven products in the modern era. In this conversation, Lenny Rachitsky interviews Josh Woodward, VP of Google Labs, about the thesis that every company needs to operate like a labs team. Josh shares insights on how to build and manage the longest-lasting and biggest labs team in the world, unlearning rates, identifying product-market fit using intuition and user reaction metrics, balancing major and minor roles, and avoiding common pitfalls in corporate innovation.

Topic Map

Operating Like a Labs Team

  • Explanation: Why every company must embrace experimentation and operate like a lab to stay ahead of startups and fast-moving competitors.
  • Key claims:
    • Underlying tech is changing so fast that companies must run frequent experiments.
    • Traditional labs nestled inside existing business units usually fizzle out in 3 to 4 years.
    • Labs need their own independence to thrive and experiment freely.
  • Examples:
    • Google Labs, Gemini app, and AI Studio operating independently.
  • Terminology:
    • Google Labs
    • AI Studio
    • Gemini app
  • Why it matters: If established companies don't experiment at the frontier, agile startups will disrupt them.

Finding and Generating Ideas

  • Explanation: Where the best ideas come from and how to spot breakthrough opportunities.
  • Key claims:
    • Best ideas rarely come from design sprints or scheduled brainstorming meetings.
    • They often emerge from small, obsessed groups of people noticing unexpected signals.
    • Creating a list of 'what is almost possible' is an effective tactic for tracking potential breakthroughs.
  • Examples:
    • NotebookLM originating from a small, passionate group exploring British parliamentary debates.
  • Terminology:
    • Almost possible
    • Design sprints
  • Why it matters: Structured brainstorming rarely yields true innovation; curiosity and obsession do.

Product-Market Fit and Metrics

  • Explanation: How to measure product-market fit and gauge whether an idea is worth pursuing.
  • Key claims:
    • Early product-market fit is more art than science.
    • Watching users' eyes and seeing them light up when showing early prototypes is a key metric.
    • The team usually knows an idea isn't working before the leader does.
  • Examples:
    • Gemini app and NotebookLM early traction.
  • Terminology:
    • Product-market fit
    • Pupil dilation
    • Unlearning rate
  • Why it matters: Traditional metrics fail in early-stage R&D; human reaction and founder conviction are stronger indicators.

Team Structure and Culture

  • Explanation: Structuring labs teams, blending roles, and maintaining a balance between execution and exploration.
  • Key claims:
    • Roles are blurring into 'builder' roles, but specialized expertise remains crucial.
    • Lab teams should be small, highly autonomous, and focused on user success.
    • Avoid vanity metrics like rapid headcount growth.
  • Examples:
    • Google Labs team structures and cross-functional collaboration.
  • Terminology:
    • Major and minor roles
    • Builder role
    • TPU harvester award
  • Why it matters: Proper organizational design prevents bureaucratic drag and keeps innovative teams energized.

Key Points

Every company needs to operate like a lab

  • Explanation: Rapid technological change requires all companies to run continuous experiments to avoid being disrupted.
  • Evidence: Josh Woodward's experience running Google Labs and managing rapid AI product development cycles.
  • Practical implication: Allocate dedicated teams to explore frontier models and new workflows independently of core business units.

Unlearning rate is a critical skill

  • Explanation: How fast you can learn something and walk away from it when it no longer serves the product vision.
  • Evidence: Observations of high-performing teams dropping outdated paradigms quickly.
  • Practical implication: Cultivate organizational agility and avoid emotional attachment to failing features or ideas.

Measuring PMF through human reaction

  • Explanation: Early validation relies heavily on qualitative signals like user excitement and pupil dilation rather than spreadsheets.
  • Evidence: Observing user responses during early prototype demonstrations.
  • Practical implication: Show prototypes to users early and watch their visceral reactions to gauge true demand.

Frameworks, Models & Processes

The Labs Life Cycle

  • How it works: Categorizing lab projects into different stages based on user scale and focus.
  • Components:
    • 0 to 1 labs (eyeballs and experimentation)
    • 1 to 10 labs (waterfalls and early scaling)
    • 10 to 100 labs (mass adoption and scaling games)
  • When to use: When organizing innovation portfolios and setting expectations for experimental projects.

Examples & Case Studies

NotebookLM started from a small group obsessed with British parliamentary debates and audio experimentation.

  • Illustrates: Great ideas come from curiosity and side projects rather than rigid design sprints.
  • Lesson: Nurture small, obsessed teams exploring weird ideas.

Actionable Takeaways

  • Immediate:
    • Set aside time to test frontier AI models and tools monthly.
    • Look for 'almost possible' technological shifts in your domain.
    • Watch user reactions to early prototypes instead of relying solely on surveys.
  • Strategic:
    • Structure innovation teams independently from core business units.
    • Focus on unlearning speed as much as learning speed.
    • Balance general builder roles with deep subject-matter expertise.
  • Questions to investigate:
    • How fast can our team unlearn outdated paradigms?
    • Are we creating a safe space for weird ideas to grow?
    • What signals indicate when to kill an idea versus doubling down?

Claims Worth Verifying

  • Most corporate labs fizzle out after 3 to 4 years if nested inside existing business units. (historical observation)

Notable Quotes

"My thesis is that every company needs to start thinking and operating like a labs team." (at 0:00) "Every company definitely needs a set of people, a team, their job is to be at the frontier." (at 0:06)

Compressed Summary

  • Every company must operate like a lab to keep pace with AI disruption.
  • Great ideas come from obsessed, curious teams, not design sprints.
  • Measure early product-market fit through visceral user reactions.
  • Balance autonomous exploration with clear organizational backing.
  • Keywords: google labs, artificial intelligence, product-market fit, innovation, experimentation
  • Core insight: In an era of hyper-fast technological evolution, every organization must cultivate internal lab teams capable of rapid experimentation and unlearning.

Core insights

7
Mechanismhigh noveltymoderate evidence

Maintain a standing 'what is almost possible' list: an explicit backlog of capabilities that are just beyond current model/tooling limits, re-scored whenever a new model or tool drops.

Why it matters

Converts the fast-moving capability frontier into a prioritized, pre-vetted roadmap instead of reacting to each release ad hoc; the backlog is already defined when a capability unlocks.

Generalization

Applies to any system that depends on rapidly improving external capabilities (model upgrades, new tools, longer context): keep a near-threshold capability queue and re-evaluate it on each upstream release rather than re-scoping from scratch.

Creating a list of 'what is almost possible' is an effective tactic for tracking potential breakthroughs.
Open source video
Mental Modelmedium noveltyweak evidence

Treat 'unlearning rate' — how fast you can drop a paradigm when it stops serving the product — as a first-class team metric, parallel to learning speed.

Why it matters

In fast-shifting AI stacks, sunk-cost attachment to outdated prompts, architectures, or models is a primary drag; explicitly measuring walk-away speed makes abandonment a deliberate practice rather than an emotional decision.

Generalization

For any agent/product built on a moving substrate, instrument migration cost and deprecation latency (time from 'this approach is obsolete' to 'removed from the system') as a tracked metric.

How fast you can learn something and walk away from it when it no longer serves the product vision.
Open source video
Cultivate organizational agility and avoid emotional attachment to failing features or ideas.
Open source video
Failure Modehigh noveltymoderate evidence

Give the experimentation team its own independence rather than nestling it inside an existing business unit; nested labs tend to die within 3-4 years.

Why it matters

Incumbent metrics, release cadence, and priorities of a core business unit tend to extinguish frontier experimentation before it produces results; independence is a structural survival requirement, not a perk.

Generalization

Exploration work (new agent architectures, frontier-model prototypes) should live behind a distinct mandate, budget, and success criteria separate from production systems, or it will be absorbed and killed by incumbent constraints.

Labs need their own independence to thrive and experiment freely.
Open source video
Traditional labs nestled inside existing business units usually fizzle out in 3 to 4 years.
Open source video
Practicemedium noveltymoderate evidence

In early stages, validate through visceral human reaction to prototypes ('eyes light up', pupil dilation) rather than quantitative metrics or surveys.

Why it matters

Early product-market fit signals precede measurable usage, so premature reliance on dashboards or surveys can mislead; direct observation of user reaction is a higher-signal, lower-cost early test.

Generalization

For agent systems pre-launch, prioritize small-N qualitative/human-in-the-loop observation sessions over building large automated eval suites; treat the automated benchmark as the scaling artifact, not the discovery tool.

Early product-market fit is more art than science.
Open source video
Watching users' eyes and seeing them light up when showing early prototypes is a key metric.
Open source video
Mechanismmedium noveltyweak evidence

The team closest to the work usually detects that an idea is failing before leadership does — a persistent information asymmetry in the direction of the builders.

Why it matters

If kill/scale decisions are driven only by top-down dashboards, the system learns about failure late; the people building the artifact hold earlier, higher-fidelity signal that needs a deliberate escalation channel.

Generalization

Instrument bottom-up signal paths (on-call, eval operators, prototyping teams) into the decision loop so that failure detection latency is bounded by the closest observer, not by reporting cadence.

The team usually knows an idea isn't working before the leader does.
Open source video
Architecturemedium noveltyweak evidence

Innovation work should be portfolio-staged by scale with different focus per stage: 0-to-1 labs optimize for eyeballs and experimentation; 1-to-10 for early scaling; 10-to-100 for mass adoption.

Why it matters

Applying the same metrics and architecture across maturity stages misallocates effort; each stage has a different objective function that should drive what you build and measure.

Generalization

Agentic products should define stage-appropriate eval regimes and architectures (exploration sandbox at 0-1, scaling/waterfall harness at 1-10, adoption/latency/cost at 10-100) rather than a single uniform evaluation stack.

0 to 1 labs (eyeballs and experimentation)
Open source video
1 to 10 labs (waterfalls and early scaling)
Open source video
10 to 100 labs (mass adoption and scaling games)
Open source video
Tradeofflow noveltyweak evidence

Roles are converging toward a generalist 'builder' position, yet deep specialized expertise remains essential — generalists and specialists must coexist.

Why it matters

Pure generalist teams lose the deep competence needed to push the frontier, while pure specialists lose the cross-functional speed to ship; the design point is a blend, not a replacement.

Generalization

For agent system design, pair a generalist orchestrator with specialized tools/subagents rather than choosing one abstraction; competence and coordination should be decoupled.

Roles are blurring into 'builder' roles, but specialized expertise remains crucial.
Open source video

Deep dives

5

Operationalizing 'unlearning rate' as a measurable team metric

Research question

What proxy metrics can turn 'unlearning rate' from a vibe into a tracked engineering metric, and do they predict product outcomes?

Why

If sunk-cost attachment to outdated prompts, architectures, or models is a primary drag, teams need a concrete deprecation-latency signal rather than an implicit cultural virtue.

How fast you can learn something and walk away from it when it no longer serves the product vision.
Open source video
Cultivate organizational agility and avoid emotional attachment to failing features or ideas.
Open source video
Source video

A standing near-threshold capability backlog that is re-scored on every upstream release

Research question

Does maintaining a re-scored 'almost possible' queue measurably improve adoption speed or hit rate of breakthrough features versus ad-hoc scoping?

Why

The capability frontier moves under the product; a pre-vetted backlog would make adoption of newly unlocked capabilities faster and less reactive.

Creating a list of 'what is almost possible' is an effective tactic for tracking potential breakthroughs.
Open source video
Underlying tech is changing so fast that companies must run frequent experiments.
Open source video
Source video

Structural independence and survival of experimentation teams

Research question

Does nesting an experimentation team inside an existing business unit predict early shutdown, and what structural mechanisms cause it?

Why

If independent labs survive and nested labs fizzle within 3-4 years, then independence is an architectural requirement for exploration, not a perk.

Traditional labs nestled inside existing business units usually fizzle out in 3 to 4 years.
Open source video
Labs need their own independence to thrive and experiment freely.
Open source video
Source video

Stage-specific evaluation regimes for agentic product portfolios

Research question

How should eval regimes and architectures differ across 0-to-1, 1-to-10, and 10-to-100 maturity stages?

Why

Applying one uniform evaluation stack across maturity stages misallocates effort and can smother early discovery or under-measure scaling.

0 to 1 labs (eyeballs and experimentation)
Open source video
1 to 10 labs (waterfalls and early scaling)
Open source video
10 to 100 labs (mass adoption and scaling games)
Open source video
Source video

Bottom-up failure detection latency as a bounded design parameter

Research question

How can organizations instrument explicit escalation channels from builders closest to the artifact so failure detection latency is bounded by closest observation?

Why

If the team usually knows an idea isn't working before leadership does, then dashboard-only decision loops systematically learn about failure late.

The team usually knows an idea isn't working before the leader does.
Open source video
Source video

Article ideas

4

Your AI Roadmap Should Be a Standing 'Almost Possible' List

Teams that maintain and re-score a near-threshold capability backlog on every model or tool release adopt newly unlocked capabilities faster and with less re-scoping than teams that build only what is currently possible.

Angle

Replace reactive roadmapping with a pre-vetted queue of capabilities that are just beyond current limits.

Source video

Unlearning Rate Is the Missing Metric in Your AI Team

The speed at which a team abandons an obsolete paradigm is as important as its learning speed, and deprecation latency should be instrumented and tracked like any other engineering metric.

Angle

Make walk-away speed a deliberate practice rather than an emotional decision.

Source video

The 3-4 Year Half-Life of Nested Labs

Exploration teams must be structurally independent; if they share the metrics, budget, and review cadence of an existing business unit, they will be extinguished before their results arrive.

Angle

Independence is a survival requirement for frontier experimentation, not a perk.

Source video

Stop Using One Eval Stack for Every Maturity Stage

Agentic products need stage-specific eval regimes: qualitative eyeball tests at 0-to-1, scaling harnesses at 1-to-10, and adoption/cost/latency at 10-to-100.

Angle

A uniform benchmark suite misallocates effort and can smother early discovery.

Source video

Project ideas

4

Almost-Possible Radar

movement-lab

A standing near-threshold capability backlog, re-scored automatically whenever an upstream model or tool release lands, increases the number of capabilities adopted within 30 days of becoming feasible compared with ad-hoc scoping.

Proof of concept

Build a lightweight queue of capability entries with feasibility scores, blocker tags, and owner; connect it to a model-release changelog feed that triggers re-scoring and surfaces the top newly feasible candidates; run for two quarters on one product team.

Measurement

Time from capability unlock to prototype; number of re-scored items; adoption hit rate; count of features shipped that were previously marked almost possible.

Source video

Unlearning Rate Instrument

new

Teams that explicitly tag obsolete paradigms and track time from declaration to removal will reduce median deprecation latency by more than 30% in one quarter versus a baseline without instrumentation.

Proof of concept

Add lightweight tags in the prompt registry, model config, and architecture decision log for 'declared obsolete' and 'removed'; build a dashboard showing median deprecation latency and stale couplings; pilot with two teams.

Measurement

Median deprecation latency; number of stale paradigms remaining; migration cost per paradigm shift; self-reported ease of abandonment.

Source video

Stage-Aware Eval Router

gatehouse

Routing agent projects to stage-appropriate eval regimes, rather than a single uniform eval stack, reduces eval cost per decision and improves kill/scale accuracy against outcomes.

Proof of concept

Classify projects into 0-to-1, 1-to-10, or 10-to-100 based on usage, team size, and metric maturity; configure different eval templates and decision criteria per stage; pilot on five agent projects for one quarter.

Measurement

Eval cost per kill/scale decision; decision accuracy compared against later outcomes; time to kill or scale; team-reported relevance of metrics.

Source video

Builder Signal Escalation Channel

movement-lab

An explicit bottom-up escalation channel from the builders closest to the artifact reduces failure detection latency by more than 50% versus dashboard-only kill/continue reviews.

Proof of concept

Create a lightweight signal channel where prototyping teams can flag emerging failure, with weekly review SLAs and a control team using dashboard-only reviews; compare detection latencies.

Measurement

Time from first builder signal to kill/continue decision; number of failures caught before leadership dashboards show them; false-positive rate.

Source video

Architectural implications

4

Experimentation capabilities are structurally more durable when separated from the production business unit.

Before

Frontier experiments run inside an existing business unit sharing its metrics, roadmap, and constraints.

After

Experiments run under an independent mandate with their own budget and success criteria, then graduate into production.

Consequence

Exploration survives long enough to reach results instead of being killed by incumbent metrics within a few years.

Source video

A near-threshold capability backlog only pays off if it is re-scored on each upstream release.

Before

Build only what is currently possible; re-scope work at each model or tool upgrade.

After

Maintain a standing 'almost possible' queue that is re-evaluated as new capabilities land.

Consequence

Faster, less ad-hoc adoption of newly unlocked capabilities and a pre-vetted roadmap.

Source video

Eval regimes should be stage-specific rather than uniform.

Before

A single evaluation suite and metric set applied to every project regardless of maturity.

After

Stage-appropriate metrics: qualitative/eyeball observation at 0-1, scaling harnesses at 1-10, adoption/cost/latency at 10-100.

Consequence

Avoids over-investing in automated benchmarks before demand is established and prevents scaling metrics from smothering early discovery.

Source video

Failure detection latency is bounded by where signal is observed, not by reporting cadence.

Before

Kill/continue decisions driven primarily by leadership dashboards and top-down review.

After

Explicit escalation channels from the builders closest to the artifact into the decision loop.

Consequence

Earlier detection of failing ideas and reduced cost of late course correction.

Source video

Tradeoffs and failure modes

5

Lab independence vs. business alignment

Benefit

Autonomy lets the team experiment freely and pursue non-obvious directions.

Cost or risk

Decoupling from the core business can drift toward work that no longer maps to real user or business needs.

Labs need their own independence to thrive and experiment freely.
Open source video
Source video

Qualitative validation vs. reproducible metrics

Benefit

Visceral user reaction is a high-signal, fast early indicator of real demand.

Cost or risk

It is subjective and not reproducible or scalable, so it cannot govern later decisions on its own.

Early product-market fit is more art than science.
Open source video
Source video

Aggressive unlearning vs. premature abandonment

Benefit

Dropping outdated paradigms quickly avoids sunk-cost drag.

Cost or risk

Walking away too fast can kill viable ideas; the boundary between 'obsolete' and 'needs more time' is unclear.

How fast you can learn something and walk away from it when it no longer serves the product vision.
Open source video
Source video

Headcount as a growth signal

Benefit

Fast team growth looks like momentum and may accelerate execution.

Cost or risk

Vanity metrics like rapid headcount growth can mask the absence of genuine user success.

Avoid vanity metrics like rapid headcount growth.
Open source video
Source video

Nestling a lab inside an existing business unit

Benefit

Access to incumbent resources, distribution, and alignment with core priorities.

Cost or risk

Structural drag from business-unit constraints tends to fizzle the lab out within 3 to 4 years.

Traditional labs nestled inside existing business units usually fizzle out in 3 to 4 years.
Open source video
Source video

Open questions

5

Which signals reliably distinguish when to kill an idea versus double down?

Why unresolved

The summary frames this as an open question and gives no decision rule or thresholds.

Research direction

Define and instrument explicit kill/continue criteria (e.g., repeated failed prototype reactions, absent visceral signal) and validate them against outcomes.

Source video

Can 'unlearning rate' be operationalized as a measurable metric rather than a vibe?

Why unresolved

It is named as a critical skill but no measurement or baseline is provided.

Research direction

Propose proxy metrics such as time from 'approach declared obsolete' to removal from the system, or migration cost per paradigm shift, and track them over time.

Source video

Can visceral user reaction (eyes lighting up, pupil dilation) be instrumented at scale, or does it remain inherently qualitative?

Why unresolved

It is presented as a key early metric but described as art more than science.

Research direction

Test behavioral proxies (session length, unprompted re-engagement, unprompted sharing) as scalable surrogates for observed visceral reaction.

Source video

Does the 3-4 year fizzle of nested labs hold generally, and what structural mechanism causes it?

Why unresolved

It is a single historical observation from one organization, flagged as needing verification.

Research direction

Survey or dataset of corporate lab lifespans, comparing nested vs independent structures and isolating the causal constraints (metrics, budget, review cadence).

Source video

How do you preserve lab independence while keeping enough alignment with the core business to stay relevant?

Why unresolved

The summary asserts independence is required but does not resolve the alignment risk it creates.

Research direction

Compare governance models (separate budget/metrics with periodic alignment reviews vs. fully detached) against survival and adoption outcomes.

Source video

Key claims

8
factualVerification needed

Most corporate labs fizzle out after 3 to 4 years if nested inside existing business units.

Evidence

Most corporate labs fizzle out after 3 to 4 years if nested inside existing business units. (historical observation)

Question

Across a broad sample of corporate labs, what is the actual survival distribution, and does nesting inside a business unit predict early shutdown?

Source video
opinionVerification not requested

Every company needs to start thinking and operating like a labs team.

Evidence

My thesis is that every company needs to start thinking and operating like a labs team. (at 0:00)

Source video
causalVerification needed

Best ideas rarely come from design sprints or scheduled brainstorming meetings.

Evidence

Best ideas rarely come from design sprints or scheduled brainstorming meetings.

Question

Do structured ideation processes systematically underproduce high-impact ideas compared with unscheduled, curiosity-driven exploration?

Source video
opinionVerification not requested

Early product-market fit is more art than science.

Evidence

Early product-market fit is more art than science.

Source video
causalVerification needed

The team usually knows an idea isn't working before the leader does.

Evidence

The team usually knows an idea isn't working before the leader does.

Question

Is builder-level failure signal systematically earlier and more accurate than leadership-level signal, and by how much?

Source video
causalVerification needed

Creating a list of 'what is almost possible' is an effective tactic for tracking potential breakthroughs.

Evidence

Creating a list of 'what is almost possible' is an effective tactic for tracking potential breakthroughs.

Question

Does maintaining a near-threshold capability backlog measurably improve adoption speed or hit rate of breakthrough features versus ad-hoc scoping?

Source video
comparativeVerification needed

Roles are blurring into 'builder' roles, but specialized expertise remains crucial.

Evidence

Roles are blurring into 'builder' roles, but specialized expertise remains crucial.

Question

Do blended generalist roles outperform specialization for frontier innovation, or is an explicit blend of both required?

Source video
opinionVerification not requested

Every company definitely needs a set of people, a team, their job is to be at the frontier.

Evidence

Every company definitely needs a set of people, a team, their job is to be at the frontier. (at 0:06)

Source video

Connections

5