AI Engineer · Published 2026-09-01

The End of the Static Screen: Architecting Intent-Driven UX — Gus Iwanaga, commercetools

privateReview: unreviewedConfidence: high
generative-uxagentic-commercedeclarative-uiMCPatomic-designcomponent-catalogintent-driven-uxstructured-output
Open on YouTube ↗

Summary

Overview

  • Speaker: Gus Iwanaga
  • Channel: AI Engineer
  • Main topic: Agentic Commerce and Generative UX/UI Architecture
  • Purpose: To provide product, engineering, and UX leaders with a mental model and architectural framework for building generative UI and agentic workflows in B2B commerce. The presentation explores the shift from static software interfaces to generative, intent-driven user experiences powered by AI agents. The speaker, Gus Iwanaga from commercetools, highlights the historical problem of users adapting to complex software systems and proposes three UI rendering protocols (Controlled, Declarative, and Open-Ended) to balance LLM autonomy with design system compliance. He demonstrates prototype iterations using atomic design principles to solve the challenges of layout generation and component catalog curation.

Topic Map

The Problem with Static Software

  • Explanation: Over the last 40 years, software has forced users to adapt to its static mental models, resulting in cognitive overload and fragmented user experiences across multiple SaaS applications.
  • Key claims:
    • Users adapt to software instead of software adapting to users.
    • SaaS proliferation has increased cognitive load and accrued organizational debt.
  • Examples:
    • Navigating complex CRM screens and ad manager tables with overwhelming feature sets.
  • Terminology:
    • static screen
    • cognitive load
    • mental model
    • SaaS proliferation
  • Why it matters: It establishes the core pain point that generative AI and agentic orchestration can solve by shifting the cognitive burden to the machine.

Agentic Orchestration and Demo Prototype

  • Explanation: A demonstration of a generative reporting tool (Mosaic) that takes natural language queries, analyzes intent, orchestrates tools, and renders dynamic dashboards.
  • Key claims:
    • AI agents can determine placement, information architecture, and catalog components based on user intent.
    • Iterative prototyping reveals that unconstrained LLM output leads to inconsistent and confusing user experiences.
  • Examples:
    • Asking the assistant to create a Q1 sales report and observing different iterations of KPI cards, date ranges, and charts.
  • Terminology:
    • agentic orchestration
    • intent-driven UX
    • Mosaic
    • auto-generation
  • Why it matters: It demonstrates the practical reality of building generative UI and exposes the need for strict guardrails.

Three UI Rendering Protocols

  • Explanation: An architectural breakdown of three ways to render user intent: Controlled (Prebuilt Components), Declarative (A2UI / JSON-Render / OpenAI), and Open-Ended (MCP Apps).
  • Key claims:
    • Controlled protocols offer high determinism and pixel-perfect control but lack flexibility.
    • Declarative protocols define a catalog of building blocks that the agent assembles, balancing design system compliance with flexibility.
    • Open-ended protocols give full UI freedom via sandboxed HTML frames but reduce determinism and increase security risks.
  • Examples:
    • Comparing ChatGPT's opinionated restaurant map component with structured dashboard card assemblies.
  • Terminology:
    • Controlled
    • Declarative
    • Open-Ended
    • A2UI
    • JSON-Render
    • MCP Apps
    • sandboxed iframe
  • Why it matters: It gives engineers a decision framework for how much control to cede to the LLM when building AI-driven interfaces.

UX and Catalog Challenges in Generative UI

  • Explanation: Discusses the primary challenges faced when building generative UI: information architecture (layout/slots/components) and catalog curation (contract between agent and UI).
  • Key claims:
    • Left-alone LLM placement is random, so teams must borrow atomic design principles.
    • The component catalog is the critical contract between the agent and the UI; every property and constraint matters.
    • UX teams must shift from designing individual pixels to defining systems, schemas, catalogs, and rules.
  • Examples:
    • Borrowing atomic design (atoms, molecules, organisms, templates, pages) to structure layouts, headers, main slots, and sub-slots.
  • Terminology:
    • atomic design
    • information architecture
    • component catalog
    • slots
    • sub-slots
    • curation
  • Why it matters: It addresses the practical hurdles of scaling generative UI without breaking brand guidelines or UX best practices.

Key Points

You adapt to software, not the other way around

  • Explanation: For decades, users have had to learn the specific mental models and workflows of every enterprise application they use.
  • Evidence: Screenshots of complex CRM and advertising tools with dense feature sets.
  • Practical implication: Generative UX aims to flip this dynamic so software adapts to human intent.

The danger of unconstrained LLM generation

  • Explanation: Without strict structural guardrails, LLMs produce erratic UI outputs that confuse users and fail enterprise standards.
  • Evidence: Four different prototype iterations of a sales report query yielding inconsistent timeframes, data formats, and layouts.
  • Practical implication: Engineers must implement declarative protocols and component catalogs to ensure consistency.

The shift from designing pixels to defining systems

  • Explanation: Designers and product managers no longer design static screens; instead, they define schemas, catalogs, rules, and intent structures.
  • Evidence: Comparison between traditional pixel design and schema-driven curation pipelines.
  • Practical implication: Design and engineering teams must collaborate on defining structured component contracts for AI agents.

Frameworks, Models & Processes

Three UI Rendering Protocols

  • How it works: Classifies generative UI architectures based on the level of control exerted over the LLM output.
  • Components:
    • Controlled (Prebuilt Components)
    • Declarative (A2UI, JSON-Render, OpenAI)
    • Open-Ended (MCP Apps in sandboxed iframes)
  • When to use: Controlled for mission-critical or highly regulated enterprise workflows; Declarative for broad cross-screen applications needing design system compliance; Open-Ended for maximum third-party flexibility.

Atomic Design for Generative UI

  • How it works: Maps top-down layout composition to bottom-up component catalogs using hierarchical slots and sub-slots.
  • Components:
    • Layout (Layout -> Slots -> Sub-slots -> Components)
    • Catalog (ProductCard, KPICard, Charts)
    • Curated Mapping
  • When to use: When orchestrating complex agentic dashboards and multi-component screens.

Examples & Case Studies

An agent generated four different variations of a Q1 sales report dashboard from the same natural language query.

  • Illustrates: The lack of consistency and determinism in early generative UI prototypes.
  • Lesson: Foundational schemas and declarative component contracts are required to stabilize AI-generated interfaces.

Applying atomic design principles to structure page layouts into header, main slots, sub-slots, and eligible component categories.

  • Illustrates: How to solve the challenge of agentic layout composition.
  • Lesson: Giving the LLM structured slots and catalogs prevents random placement and enforces design system rules.

Actionable Takeaways

  • Immediate:
    • Evaluate your current software stack for static friction points.
    • Explore declarative UI rendering protocols like JSON-Render or A2UI.
    • Adopt atomic design principles for component structuring.
  • Strategic:
    • Shift design thinking from static screen creation to schema and catalog curation.
    • Balance agent autonomy with brand and design system constraints.
    • Prepare for the integration of MCP (Model Context Protocol) apps into enterprise workflows.
  • Questions to investigate:
    • How much autonomy should we give our AI agents over UI layout?
    • What is the performance overhead of sandboxed iframe rendering for open-ended MCP apps?
    • How do we maintain deterministic behavior in non-deterministic LLM pipelines?

Claims Worth Verifying

  • Forty years of static screens have forced users to adapt to software instead of software adapting to users. (philosophical assertion)
  • Declarative UI protocols like A2UI and JSON-Render provide a reliable bridge between LLM intent and design system compliance. (technical architecture claim)

Notable Quotes

"You adapt to the software. Not the other way around." "The catalog is the contract between the agent and your UI: every component, every property, every constraint. Get this wrong and nothing downstream can save you." "We don't design the pixel anymore."

Compressed Summary

  • Static software forces humans to bear high cognitive load.
  • Generative UI requires balancing agent autonomy with strict design system guardrails.
  • Three core rendering protocols: Controlled, Declarative, and Open-Ended.
  • Atomic design and component catalogs are essential for stabilizing AI-generated layouts.
  • Keywords: generative-ux, agentic-commerce, declarative-ui, mcp, atomic-design
  • Core insight: Architecting generative UX requires shifting from designing static pixels to curating structured component catalogs and declarative rendering schemas that guide AI agents safely.

Core insights

5
Architecturemedium noveltystrong evidence

Agent-generated UI should be governed by an explicit control-axis decision: Controlled (prebuilt components), Declarative (catalog assembly via A2UI/JSON-Render/OpenAI), or Open-Ended (MCP apps in sandboxed iframes). Choosing the protocol is an architectural decision because it determines determinism, design system compliance, flexibility, and security risk in advance.

Why it matters

This gives engineers a crisp boundary for where render-time autonomy lives: inside a component registry, a declarative layout schema, or a sandboxed app frame. The wrong choice causes either brittle UI or unpredictable output.

Generalization

Any system that lets an LLM influence user-facing output needs a control-axis protocol decision, from chat widgets to full dashboards, rather than tuning prompts in the hope of controlling layout.

Controlled protocols offer high determinism and pixel-perfect control but lack flexibility.
Open source video
Declarative protocols define a catalog of building blocks that the agent assembles, balancing design system compliance with flexibility.
Open source video
Open-ended protocols give full UI freedom via sandboxed HTML frames but reduce determinism and increase security risks.
Open source video
Mechanismmedium noveltystrong evidence

The component catalog is a machine-consumed contract between the agent and the UI. Each property and constraint in the catalog, not the prompt, determines what an agent can express or render.

Why it matters

The catalog becomes the primary integration surface between LLM reasoning and UI runtime; it must be schema-validated, versioned, and observably rendered. Missing or loosely typed constraints lead to off-brand or invalid output.

Generalization

Wherever an LLM must act on a bounded world—UI components, tool calls, file operations—the enumerated catalog is a stronger restraint than prose instructions.

The component catalog is the critical contract between the agent and the UI; every property and constraint matters.
Open source video
Architecturemedium noveltymoderate evidence

Atomic design provides a practical two-level pattern for generative UI: use a top-down slot/sub-slot layout skeleton for information architecture and a bottom-up curated component catalog for content. This reconciles flexible LLM composition with visual/business consistency.

Why it matters

Layout randomness is a failure mode in agent-generated screens. A rigid page skeleton constrained by slots and eligible component categories preserves UX coherence while still letting the agent assemble screen content.

Generalization

The same pattern applies to any generated document/canvas: keep the large-scale structure deterministic and let the model select from constrained composable blocks.

Left-alone LLM placement is random, so teams must borrow atomic design principles.
Open source video
Maps top-down layout composition to bottom-up component catalogs using hierarchical slots and sub-slots.
Open source video
Empirical Resultmedium noveltymoderate evidence

Unconstrained LLM generation is an empirical reliability failure for UI: the same natural-language request produced multiple inconsistent layouts, timeframes, and data formats. Consistency must be engineered as a separate property, not assumed from model behavior.

Why it matters

Agent-generated interfaces need structural validation and canonical rendering because users experience semantic drift as bugs, even when every individual output is plausible.

Generalization

Before relying on LLM output for user-visible surfaces, run repeated-prompt experiments and measure layout, semantics, and temporal consistency; treat variance as a defect if the application needs steady UX.

Iterative prototyping reveals that unconstrained LLM output leads to inconsistent and confusing user experiences.
Open source video
Four different prototype iterations of a sales report query yielding inconsistent timeframes, data formats, and layouts.
Open source video
Practicemedium noveltymoderate evidence

The core product-design skill shifts from designing static screens and pixels to designing the systems, schemas, catalogs, and rules that an agent uses to construct intent-driven interfaces.

Why it matters

Design teams no longer ship a finished visual artifact; they ship machine-readable constraints. This changes the collaboration boundary between design and engineering and requires schema review as a first-class product activity.

Generalization

In agentic products, the human-facing artifact becomes a contract for machine composition; the limiting design artifact is the schema, not the mockup.

UX teams must shift from designing individual pixels to defining systems, schemas, catalogs, and rules.
Open source video

Deep dives

4

A decision framework for choosing Controlled, Declarative, or Open-Ended rendering per UI surface

Research question

Under what conditions should a team choose each of the three rendering protocols for a given surface, and how can a decision rubric account for task criticality, intent variability, and security exposure without fragmenting the product's UX?

Why

The protocol taxonomy is a strong architectural boundary, but applying it per surface requires operational criteria; otherwise teams default to one protocol and get brittle or unsafe UI.

Controlled protocols offer high determinism and pixel-perfect control but lack flexibility.
Open source video
Declarative protocols define a catalog of building blocks that the agent assembles, balancing design system compliance with flexibility.
Open source video
Open-ended protocols give full UI freedom via sandboxed HTML frames but reduce determinism and increase security risks.
Open source video
Source video

Schema governance and catalog evolution for declarative agent UI

Research question

How can a component catalog remain both small enough to govern and expressive enough to cover open-ended user intent, and what versioning and compositional schema mechanisms make this sustainable?

Why

The catalog becomes the primary integration contract between LLM and UI runtime; without governance, catalog growth and drift will silently narrow or corrupt the agent's expressive space.

The component catalog is the critical contract between the agent and the UI; every property and constraint matters.
Open source video
Declarative protocols define a catalog of building blocks that the agent assembles, balancing design system compliance with flexibility.
Open source video
Source video

Runtime validation stack for agent-assembled UI

Research question

What layered runtime validations—JSON schema, slot constraints, component eligibility, design-token rules—are necessary and sufficient to prevent invalid or off-brand generated UI from reaching users?

Why

Unconstrained LLM output is empirically inconsistent; treating generated UI as structured output with enforceable constraints at runtime is likely the only reliable way to make declarative UI production-safe.

Unconstrained LLM generation is an empirical reliability failure for UI: the same natural-language request produced multiple inconsistent layouts, timeframes, and data formats.
Open source video
Declarative protocols define a catalog of building blocks that the agent assembles, balancing design system compliance with flexibility.
Open source video
Source video

Measuring semantic and layout consistency of intent-driven generated interfaces

Research question

What metrics can detect regressions in layout and semantic consistency when the same user intent is rendered repeatedly by a generative UI agent?

Why

The consistency failure was observed qualitatively, but engineering teams need a measurable signal to compare rendering protocols, prompt strategies, and catalog changes before users experience drift as bugs.

Four different prototype iterations of a sales report query yielding inconsistent timeframes, data formats, and layouts.
Open source video
Iterative prototyping reveals that unconstrained LLM output leads to inconsistent and confusing user experiences.
Open source video
Source video

Article ideas

4

The Component Catalog Is the New Contract: Why Prompt Tuning Can't Fix Generated UI

Design systems should be treated as machine-readable constraints, not prose documentation; when the component catalog defines every valid property and constraint, schema violations—not prompt wording—become the controllable source of off-brand UI.

Angle

Engineering and design-systems argument for shifting from prompt-level interventions to schema-level contract design in agentic UI.

Source video

Choose Your Agent's Leash: A Practical Guide to Controlled, Declarative, and Open-Ended UI Rendering

Selecting the UI rendering protocol is an architectural decision that must be made per surface based on determinism, brand compliance, and security risk—not as a statement about how 'smart' the AI should be.

Angle

Decision-oriented architecture piece comparing the three rendering protocols and their consequences for validation, observability, and trust.

Source video

Atomic Design Is the Missing Guardrail for Generative UX

LLM-determined layout is random, but teams can make generated interfaces coherent by combining a top-down slot and sub-slot page skeleton with a bottom-up curated component catalog—deterministic structure, flexible content.

Angle

Pattern essay linking atomic design to generative UI: constraints on the layout skeleton are the interface-level analogue of type and token rules for LLM outputs.

Source video

Design Leaders Will Ship Schemas, Not Screens

When an agent assembles UI from intent, the design team's primary deliverable is no longer static mockups; it is machine-readable schemas, component catalogs, and composition rules, and this shift must reshape design org processes today.

Angle

Practice-oriented thought piece for design leadership on how review, handoff, and hiring change when the schema is the interface.

Source video

Project ideas

3

LayoutDrift: A reproducibility harness for intent-driven interfaces

beyond-evals

Repeated identical natural-language queries rendered through a slot-constrained declarative pipeline produce measurably lower layout/semantic variance than the same queries rendered through an unconstrained LLM-to-JSON pipeline.

Proof of concept

Instrument a dashboard generator that renders the same KPI request N times under two modes: unconstrained LLM dashboard generation vs. generation constrained by a slot/sub-slot skeleton and component catalog; compare the resulting component trees and datasets.

Measurement

Normalized structural edit distance between rendered layouts, semantic field agreement for timeframes/data formats, and invalid-component rate per mode.

Source video

CatalogGuard: Schema-validation enforcement for agent-generated UI

gatehouse

For a fixed set of intents and components, a schema-validated component catalog reduces invalid or off-brand generated UI (design-token violations, invalid props, disallowed component combinations) compared to a prose-only catalog of the same component set.

Proof of concept

Build two catalogs for the same mock design system—one described only in prompting prose, one expressed as JSON schemas with slot constraints—and use each to generate the same set of user-intent dashboards from an LLM.

Measurement

Proportion of generated UI trees that fail validation: missing required props, disallowed component types, design-token violations, and out-of-slot placements.

Source video

RenderProtocol Advisor: Rubric-based matching of workflows to UI rendering protocols

new

A transparent rubric based on task criticality, intent variability, third-party trust, and compliance sensitivity can recommend Controlled, Declarative, or Open-Ended rendering with high agreement against expert judgments on real B2B workflows.

Proof of concept

Define a 5-question rubric, score 10 B2B commerce UI surfaces from the Mosaic-style reporting domain, ask an expert panel to independently assign a protocol, and compare recommendations via agreement metrics.

Measurement

Cohen's kappa between rubric recommendation and expert majority, plus a qualitative audit of disagreement cases to refine the rubric.

Source video

Architectural implications

4

UI rendering decisions can be made per workflow or per surface, rather than being fixed for the application.

Before

The frontend was a fixed set of screens and the LLM only produced text or structured data.

After

Choose Controlled, Declarative, or Open-Ended rendering for each surface and have a runtime that interprets intent into slots, components, or sandboxed frames.

Consequence

The application needs a renderer boundary that can validate component usage, isolate third-party UI, and audit what the agent chose to display.

Source video

Unconstrained LLM layout is random and inconsistent.

Before

Layout and information architecture were decided by designers and hard-coded by frontend developers.

After

Design language is expressed as slot/sub-slot skeletons plus eligible component categories that agents must assemble within.

Consequence

The runtime needs to enforce structural placement rules, not only component-level style rules.

Source video

The component catalog becomes the contract between agent and UI.

Before

Design systems and component libraries were documentation plus developer tooling.

After

They become queryable schemas whose constraints directly steer agent output.

Consequence

Schema ownership, versioning, validation, and observability shift from documentation to core production infrastructure.

Source video

Open-ended MCP Apps render arbitrary HTML in sandboxed frames.

Before

Third-party UI extensions are not treated as first-class agent-driven surfaces.

After

Third-party code can render arbitrary UI in a controlled security boundary, giving maximum flexibility.

Consequence

Security architecture, content isolation, and behavioral audit become prerequisites before UI freedom can be granted.

Source video

Tradeoffs and failure modes

4

Controlled rendering protocol

Benefit

High determinism and pixel-perfect control, suitable for regulated or mission-critical workflows.

Cost or risk

Lacks flexibility; cannot adapt to novel intents or unstructured user needs.

Controlled protocols offer high determinism and pixel-perfect control but lack flexibility.
Open source video
Source video

Open-Ended rendering protocol

Benefit

Full UI freedom and maximum third-party flexibility via sandboxed HTML frames.

Cost or risk

Reduced determinism and increased security risks from arbitrary app-rendered content.

Open-ended protocols give full UI freedom via sandboxed HTML frames but reduce determinism and increase security risks.
Open source video
Source video

Declarative rendering protocol

Benefit

Balances design system compliance with flexibility by constraining the agent to a component catalog.

Cost or risk

Catalog curation becomes a persistent burden; every omitted property or constraint narrows the space of valid generated experiences.

Declarative protocols define a catalog of building blocks that the agent assembles, balancing design system compliance with flexibility.
Open source video
Source video

Unconstrained LLM layout

Benefit

The agent freely determines placement and content in response to intent.

Cost or risk

Placement is random and output becomes inconsistent and confusing without structural guardrails.

Left-alone LLM placement is random, so teams must borrow atomic design principles.
Open source video
Source video

Open questions

4

How should a team decide which rendering protocol to apply to each surface without fragmenting the user experience across the product?

Why unresolved

The protocol taxonomy is presented as a framework, but no decision procedure is given for a heterogeneous application.

Research direction

Build a decision rubric or cost model comparing task criticality, variability of intent, and security exposure.

Source video

How can component catalogs stay both small enough to govern and expressive enough to cover open-ended user intent?

Why unresolved

The catalog is the contract, but the summary does not address governance, versioning, or catalog growth.

Research direction

Study catalog coverage vs. catalog size and develop schema languages that make constraints compositional.

Source video

What runtime validation guarantees should be applied to declarative agent output before rendering?

Why unresolved

The summary identifies the catalog as the critical contract but does not specify how to enforce it beyond structure.

Research direction

Treat generated UI as structured output and validate against JSON schema, slot constraints, and design-token rules at runtime.

Source video

How should consistency of agent-generated UI be measured so that regressions can be detected?

Why unresolved

The failure mode was observed qualitatively but no metric is proposed.

Research direction

Develop layout and semantics similarity metrics across repeated generations of the same intent.

Source video

Key claims

7
factualVerification needed

AI agents can determine placement, information architecture, and catalog components based on user intent.

Evidence

AI agents can determine placement, information architecture, and catalog components based on user intent.

Question

Can this capability be reproduced at production reliability across varied B2B commerce queries?

Source video
factualVerification needed

The same natural-language query generated four different dashboard variants in an early prototype.

Evidence

Four different prototype iterations of a sales report query yielding inconsistent timeframes, data formats, and layouts.

Question

How often do repeated identical queries drift in layout or data semantics in larger agentic UI systems?

Source video
causalVerification needed

Unconstrained LLM output leads to inconsistent and confusing user experiences.

Evidence

Iterative prototyping reveals that unconstrained LLM output leads to inconsistent and confusing user experiences.

Question

Is perceptual confusion measurable through task completion or eye-tracking when UI layout varies?

Source video
comparativeVerification needed

Declarative UI protocols balance design system compliance with flexibility.

Evidence

Declarative protocols define a catalog of building blocks that the agent assembles, balancing design system compliance with flexibility.

Question

Can a declarative catalog achieve parity with controlled UI on brand compliance while still covering novel intents?

Source video
comparativeVerification needed

Open-Ended protocols reduce determinism and increase security risks.

Evidence

Open-ended protocols give full UI freedom via sandboxed HTML frames but reduce determinism and increase security risks.

Question

How do sandboxing technologies mitigate or fail to mitigate the security risks in practice?

Source video
opinionVerification needed

The component catalog is the critical contract between the agent and the UI.

Evidence

The component catalog is the critical contract between the agent and the UI; every property and constraint matters.

Question

What proportion of generated-UI quality failures can be traced to incomplete or ambiguous catalog schemas?

Source video
predictionVerification needed

UX teams must shift from designing pixels to defining systems, schemas, catalogs, and rules.

Evidence

UX teams must shift from designing individual pixels to defining systems, schemas, catalogs, and rules.

Question

Will design organizations measurably change hiring, tooling, and review processes under generative UI?

Source video

Connections

5