AI Engineer · Published 2026-09-10

The Spatial Harness: Bringing Agents to the Canvas — Max Drake, tldraw

privateReview: unreviewedConfidence: high
tldrawcanvasagentsspatialmultiplayerspatial-canvasagent-harnessmulti-agentmultimodal-groundingObservabilitydependency-graphagent-runtime
Open on YouTube ↗

Summary

Overview

  • Speaker: Max Drake
  • Channel: AI Engineer
  • Main topic: Agents on the Canvas and Spatial User Interfaces for AI
  • Purpose: To demonstrate how tldraw's infinite canvas and SDKs serve as a spatial harness for AI agents, enabling multi-agent collaboration, spatial task management, and multimodal UI interactions. Max Drake, product engineer at tldraw, explores the intersection of AI agents and infinite canvas applications. He discusses why agents struggle in 2D space, the development of tldraw's infinite canvas SDKs as an agent engine, teaching LLMs about canvas geometry and interaction via 'teach' and agent starter kits, and multi-agent human collaboration through experimental 'fairies' systems.

Topic Map

Introduction to Agents on the Canvas

  • Explanation: Opening demo where Max Drake asks Claude Code to find a Notion document from an email and build it on the tldraw desktop app.
  • Key claims:
    • Agents can be directed to find documents and build applications on spatial canvases.
    • Canvas interfaces offer unique UX benefits when working with LLMs.
  • Examples:
    • Finding a Notion link from an email and building a fluid simulation demo.
  • Terminology:
    • tldraw desktop app
    • Notion document
    • Claude Code
  • Why it matters: Demonstrates the practical workflow of instructing coding agents to build UI prototypes directly within a spatial canvas app.

Background on tldraw and Infinite Canvas SDKs

  • Explanation: Overview of tldraw as a startup based in London, providing free infinite canvas whiteboarding apps and developer SDKs.
  • Key claims:
    • Getting infinite canvas mathematics and scaling right is difficult.
    • tldraw SDK acts as the engine powering many collaborative apps and AI canvas integrations.
  • Examples:
    • Miro competitors, slide designers, and Replit's agent canvas built on top of tldraw.
  • Terminology:
    • infinite canvas SDK
    • multiplayer built-in
    • live sync
  • Why it matters: Establishes why tldraw is uniquely positioned to solve the canvas-rendering bottleneck for AI applications.

Why Agents Struggle in 2D Space

  • Explanation: Analysis of why text-in/text-out coding agents fail at UI and alignment tasks in 2D canvas environments without specialized tooling.
  • Key claims:
    • Agents are trained on text-in/text-out and struggle to understand 2D spatial relationships.
    • Aligning UI elements requires significant engineering work to bridge agents with visual space.
  • Examples:
    • Trying to get LLM agents to align shapes or design visual components.
  • Terminology:
    • 2D space
    • UI alignment
    • matrix math
  • Why it matters: Highlights the fundamental mismatch between text-trained LLMs and spatial visual interfaces.

Teaching LLMs About the Canvas (Teach & Agent Starter Kit)

  • Explanation: Explanation of projects like 'teach' and the tldraw agent starter kit designed to teach LLMs about canvas DOM structures and screenshots.
  • Key claims:
    • Teaching LLMs about canvas structure involves combining screenshots, JSON data, and spatial prompts.
    • The agent starter kit wraps LLMs in a harness enabling them to work agentially on the canvas.
  • Examples:
    • Prompting a cat agent to bring red shapes over from elsewhere on the canvas.
  • Terminology:
    • agent starter kit
    • teach project
    • DOM structure
    • MIT license
  • Why it matters: Provides developers with open-source tools to turn standard LLMs into canvas-aware agents.

Multi-Agent Collaboration: Fairies

  • Explanation: Introduction of 'fairies', a multi-agent collaboration project where individual agent entities ('fairies') collaborate with humans and each other on the canvas.
  • Key claims:
    • Fairies can be configured, given attributes like hats or colors, and assigned tasks.
    • Multi-agent coordination allows users to inspect agent state without reading raw chat logs.
  • Examples:
    • Assigning financial memo tasks to a group of fairy agents.
  • Terminology:
    • fairies
    • orchestrator fairy
    • group chat
    • multi-agent coordination
  • Why it matters: Reimagines multi-agent interactions as visible, collaborative spatial entities rather than opaque chat windows.

The Spatial Harness and Dependency Graphs

  • Explanation: Discussion of internal hackathons at tldraw, dependency graphs for product launches, and the 'tech tree' app for task management.
  • Key claims:
    • Complex launches use massive dependency graphs to track tasks.
    • The tech tree app treats individual dependency graph tasks as autonomous coding agents.
  • Examples:
    • Hand tracking canvas controls and paddle ball games built via desktop app windows.
  • Terminology:
    • tech tree app
    • dependency graph
    • tldraw desktop app
    • code mode
  • Why it matters: Shows how spatial task trees can act as a fully functional project management and execution environment.

Key Points

Agents Are Bad at 2D Space

  • Explanation: Because LLMs are trained on text-in/text-out, they lack native intuition for 2D spatial positioning, resizing, and UI alignment.
  • Evidence: Developers encounter repeated failure modes when asking agents to build canvas apps without spatial harnesses.
  • Practical implication: Engineers must build custom spatial wrappers and tools so agents can perceive and manipulate canvas geometry.

The Canvas as a Spatial Harness

  • Explanation: Treating the infinite canvas as an agent runtime environment provides a shared workspace for humans and multiple AI agents.
  • Evidence: tldraw's SDK supports multiplayer live sync, cursors, and viewports out of the box.
  • Practical implication: Canvas apps become powerful interfaces for orchestrating multi-agent systems visually.

State Over Chat Logs

  • Explanation: In multi-agent spatial setups, users can inspect agent states directly on the canvas rather than digging through lengthy chat interfaces.
  • Evidence: Fairy agents show planning status, assigned tasks, and progress directly next to their visual avatars.
  • Practical implication: Improves observability and transparency in complex multi-agent workflows.

Frameworks, Models & Processes

tldraw Infinite Canvas SDK

  • How it works: Provides core primitives for selection, transformation, rendering, and multiplayer live synchronization in infinite 2D space.
  • Components:
    • Infinite canvas engine
    • Multiplayer live sync
    • Selection and viewpoint management
    • Agent runner harness
  • When to use: When building collaborative tools, design apps, whiteboards, or spatial agent interfaces.

Examples & Case Studies

Prompting an agent with a grey cat drawing to find red shapes elsewhere on the canvas.

  • Illustrates: Agent visual perception and goal-seeking behavior in spatial environments.
  • Lesson: Agents can navigate canvas viewports, identify target shapes, and execute spatial movement tasks.

Creating a multi-fairy group chat to draft financial memo figures for an upcoming board meeting.

  • Illustrates: Multi-agent task distribution and orchestration on the canvas.
  • Lesson: Specialized agents can coordinate subtasks, assign work, and report completion status visually.

Using tldraw desktop app as a tiling window manager to run code and play games like Pong.

  • Illustrates: Exposing desktop application instances and file systems to coding agents.
  • Lesson: Agents with computer access can manipulate OS-level windows and ephemeral UIs.

Actionable Takeaways

  • Immediate:
    • Explore the tldraw agent starter kit and MIT-licensed code on their website.
    • Experiment with combining screenshots and JSON canvas data for agent prompting.
  • Strategic:
    • Infinite canvases are emerging as primary user interfaces for multi-agent orchestration.
    • Bridging text-based LLMs with spatial primitives unlocks new paradigms in software design and project management.
  • Questions to investigate:
    • How do spatial harnesses scale when managing hundreds of concurrent agents?
    • What are the latency and token overhead costs of screenshot-based spatial reasoning?

Claims Worth Verifying

  • tldraw SDK powers agent canvas experiences for tools like Replit. (factual)
  • The agent starter kit code is available under an MIT license. (factual)

Notable Quotes

"am i qualified to be giving this talk? I like to think so. I've been doing like agents on the canvas stuff since before chatgpt came out." (at 1:45) "it turns out agents are really really bad at working in 2D space and understanding 2D space." (at 7:34) "fairies have friends, right? And they can here we go. And they can fairies can work together." (at 15:32)

Compressed Summary

  • Infinite canvases solve the spatial reasoning bottleneck for AI agents.
  • tldraw provides SDKs and agent starter kits for building canvas-native AI apps.
  • Multi-agent 'fairies' enable visual collaboration between humans and AI agents.
  • Spatial task trees and desktop app integrations allow agents to manage real-world workflows.
  • Keywords: tldraw, canvas, agents, spatial, multiplayer
  • Core insight: Treating the infinite canvas as a spatial harness bridges the gap between text-based LLMs and 2D visual software engineering.

Core insights

7
Failure Modemedium noveltymoderate evidence

Text-in/text-out trained coding agents have a structural mismatch with 2D spatial work: they lack native intuition for positioning, resizing, and UI alignment, so spatial capability must be engineered around them rather than assumed.

Why it matters

If an agent's failure on canvas tasks is a representation/training mismatch rather than a capability gap, teams should invest in spatial grounding layers and tool interfaces instead of swapping to bigger models or better prompts.

Generalization

Any domain where the model's native modality (text) doesn't match the task's state space (geometry, audio, video) will exhibit repeatable failure modes that require an explicit bridge abstraction, not just better prompting.

Agents are trained on text-in/text-out and struggle to understand 2D spatial relationships.
Open source video
Because LLMs are trained on text-in/text-out, they lack native intuition for 2D spatial positioning, resizing, and UI alignment.
Open source video
Mechanismhigh noveltymoderate evidence

Giving LLMs a usable model of a canvas requires a composite observation channel: screenshots plus structured JSON plus spatial prompts, rather than any single representation.

Why it matters

Designers of agent runtimes face a concrete representation choice; this claims the answer is hybrid (pixels for perception, structured data for precision, prompt scaffolding for intent) rather than picking one channel.

Generalization

For agents operating on rich visual/structured state (dashboards, CAD, spreadsheets, GUIs), the observation layer should fuse lossy perceptual views with lossless structured views.

Teaching LLMs about canvas structure involves combining screenshots, JSON data, and spatial prompts.
Open source video
Architecturemedium noveltystrong evidence

The 'agent starter kit' is explicitly a harness that wraps an LLM to make it act agentially on the canvas, decoupling the general model from canvas-specific tools and loop.

Why it matters

This is the concrete answer to 'agents are bad at 2D': the harness — not the model — owns spatial perception, tool-calling, and action sequencing. That places harness design as the primary engineering surface.

Generalization

Harnesses (perception adapters + tools + control loop) are becoming the differentiated layer for agents in specialized domains, with the foundation model treated as a swappable component.

The agent starter kit wraps LLMs in a harness enabling them to work agentially on the canvas.
Open source video
Provides developers with open-source tools to turn standard LLMs into canvas-aware agents.
Open source video
Architecturehigh noveltymoderate evidence

The infinite canvas itself is being repositioned from a drawing surface to an agent runtime: a shared workspace where humans and multiple agents coexist, with multiplayer live sync, cursors, and viewports provided by the SDK rather than built by the agent developer.

Why it matters

If collaboration primitives (sync, presence, viewport) come for free from the canvas substrate, the marginal cost of adding multi-agent, multi-human orchestration drops sharply — the canvas becomes the coordination plane.

Generalization

Reusing an existing realtime multi-user substrate as the agent runtime avoids rebuilding presence, sync, and shared-state semantics; the agent inherits multi-human collaboration machinery.

Treating the infinite canvas as an agent runtime environment provides a shared workspace for humans and multiple AI agents.
Open source video
tldraw's SDK supports multiplayer live sync, cursors, and viewports out of the box.
Open source video
Practicehigh noveltymoderate evidence

In spatial multi-agent setups, agent state is rendered adjacent to each agent's avatar (planning status, assigned tasks, progress), replacing chat-log inspection as the primary observability surface.

Why it matters

Observability moves from scrolling transcripts to spatial state inspection, which changes both the UX and the data model: agents must expose structured, positionable state rather than only emitting text.

Generalization

Agent observability improves when internal state is bound to a spatial or structural object the user can look at directly, rather than being reconstructed from linear logs.

Multi-agent coordination allows users to inspect agent state without reading raw chat logs.
Open source video
Fairy agents show planning status, assigned tasks, and progress directly next to their visual avatars.
Open source video
Mental Modelhigh noveltymoderate evidence

The 'tech tree' app converts each node of a launch dependency graph into an autonomous coding agent, turning a project-management artifact into an execution environment.

Why it matters

This collapses planning and execution into the same graph: the dependency structure used to schedule work becomes the agent orchestration topology, which changes how work decomposition, blocking, and fan-out are represented.

Generalization

Any DAG already used for planning (build graphs, data pipelines, ticket dependencies) can be reinterpreted as an agent orchestration graph, reusing existing dependency semantics as control flow.

The tech tree app treats individual dependency graph tasks as autonomous coding agents.
Open source video
Complex launches use massive dependency graphs to track tasks.
Open source video
Mechanismmedium noveltyweak evidence

Multi-agent collaboration is designed as configurable spatial personas ('fairies') with attributes like hats or colors and a designated orchestrator, coordinating through group chat and shared canvas context.

Why it matters

Identity, role assignment, and orchestration are encoded as first-class visual/configurable properties rather than emergent prompt behavior, giving a concrete way to structure and differentiate agents.

Generalization

Giving agents explicit identity/role attributes and a named orchestrator makes multi-agent systems legible to users and easier to route tasks, at the cost of requiring a coordination protocol.

Fairies can be configured, given attributes like hats or colors, and assigned tasks.
Open source video
orchestrator fairy
Open source video

Deep dives

5

Observation-channel ablation for canvas agents (screenshots vs JSON vs spatial prompts)

Research question

Which mix and cadence of screenshots, structured JSON state, and spatial prompt scaffolding maximizes canvas task success per token spent, and where does the hybrid strictly dominate any single channel?

Why

The summary asserts that teaching LLMs about canvas structure requires combining screenshots, JSON data, and spatial prompts, but gives no ablation or cost model. Without that, harness builders cannot decide how much perceptual context to pay for on every agent step, and multimodal grounding claims remain unfalsifiable.

Teaching LLMs about canvas structure involves combining screenshots, JSON data, and spatial prompts.
Open source video
Agents can be directed to find documents and build applications on spatial canvases.
Open source video
Source video

Canvas substrate as agent runtime and coordination plane

Research question

Does reusing an existing realtime multi-user substrate (live sync, presence, viewports) as the agent runtime reduce the cost of building multi-agent, multi-human orchestration versus building coordination from scratch, and what coupling costs does it impose?

Why

The claim is that collaboration primitives come free from the canvas SDK, so the marginal cost of adding agents drops. That is a testable architecture bet: the agent inherits sync and presence but also inherits the canvas data model for its own state, which may constrain what agents can express.

tldraw SDK acts as the engine powering many collaborative apps and AI canvas integrations.
Open source video
Getting infinite canvas mathematics and scaling right is difficult.
Open source video
Source video

Dependency-graph-as-orchestration: failure, retry and blocking semantics

Research question

When every node of a planning DAG is bound to an autonomous coding agent, what retry, rollback, and dependency-blocking policies are needed before partial completion stops requiring human intervention?

Why

Collapsing planning and execution into one graph makes graph correctness equal execution correctness. The summary presents the tech tree as a hackathon-era idea with no described error handling, which is exactly where DAG-as-orchestration either becomes practical or deadlocks.

The tech tree app treats individual dependency graph tasks as autonomous coding agents.
Open source video
Complex launches use massive dependency graphs to track tasks.
Open source video
Source video

Where the harness/model boundary actually lies

Research question

What exactly does a canvas agent harness provide (perception adapters, tool surface, control loop, state serialization), and how much of it is portable to non-tldraw canvases or non-canvas spatial domains?

Why

If the harness, not the model, owns spatial perception and action sequencing, harness design becomes the primary engineering surface and the foundation model becomes swappable. That is a strong architectural claim that only holds if the harness's responsibilities can be enumerated and ported.

The agent starter kit wraps LLMs in a harness enabling them to work agentially on the canvas.
Open source video
Aligning UI elements requires significant engineering work to bridge agents with visual space.
Open source video
Source video

Spatial agent state as the observability surface

Research question

Does rendering agent planning status, task assignment, and progress next to each agent's avatar reduce time-to-diagnose multi-agent failures compared with transcript-based debugging, and what does it cost in auditability?

Why

Moving observability out of linear logs into positionable state objects changes the agent's data model: agents must publish structured, addressable state rather than emit text. The trade is searchable, diffable history for glanceable, spatially-anchored inspection, which needs a controlled comparison to justify.

Multi-agent coordination allows users to inspect agent state without reading raw chat logs.
Open source video
Fairy agents show planning status, assigned tasks, and progress directly next to their visual avatars.
Open source video
Source video

Article ideas

4

The Harness Is the Product: Why 'Agents Are Bad at 2D' Is an Engineering Problem, Not a Model Problem

Persistent agent failures on spatial and other non-text state spaces are representation mismatches, so the durable engineering value accrues to the harness layer — perception adapters, tool surfaces, and control loops — while the foundation model stays a swappable component.

Angle

Reframe the 'better model / better prompt' reflex as a harness-design discipline: the model's modality never matches the task's state space, so an explicit bridge abstraction is mandatory, and whoever owns that bridge owns the product.

Source video

Pixels, JSON, and Prompts: The Three-Channel Observation Layer for Agents on Rich State

Agents operating on rich visual or structured state should fuse a lossy perceptual channel (screenshots) with a lossless structured channel (JSON) and an intent channel (spatial prompts); picking any single representation caps accuracy, and the interesting engineering question is the mix and cadence, not the choice.

Angle

Argue against the screenshot-only 'computer use' framing and the structured-API-only framing simultaneously, using canvas geometry as the concrete case where each channel alone fails.

Source video

Your Dependency Graph Is Already an Agent Orchestrator — You Just Haven't Enforced Its Semantics

Any DAG already used for planning can be reinterpreted as an agent orchestration topology, but the reuse is only safe once blocking, retry, and partial-completion semantics are enforced by the orchestration layer, because graph correctness becomes execution correctness.

Angle

Take dependency-graph planning artifacts as the control flow and show what breaks when nodes become autonomous agents rather than human-assigned tasks.

Source video

Stop Debugging Agents in a Chat Log: Spatial State as the Primary Observability Interface

Multi-agent observability improves when internal state is bound to a spatial or structural object the user can look at directly, but the gain is paid for by giving up the linear, searchable audit trail — so the right answer is state-first inspection with logs demoted to a secondary forensic layer.

Angle

Compare transcript-based debugging against spatially-anchored agent state and name the data-model requirement: agents must publish structured, addressable state, not just text.

Source video

Project ideas

4

Spatial Observation Ablation Bench

beyond-evals

On canvas manipulation tasks (alignment, spatial search, shape relocation), a hybrid observation channel of screenshot + JSON state + spatial prompt outperforms the best single channel on task success rate, and JSON-only degrades most on perceptually-grounded alignment tasks while screenshot-only degrades most on precision placement.

Proof of concept

Build a fixed task suite of canvas layout tasks in a tldraw-style canvas, run the same LLM through three observation configurations (screenshot-only, JSON-only, hybrid) with an identical tool surface and control loop, and log per-step token cost, latency, retries, and success.

Measurement

Task success rate per configuration, tokens and latency per successful task, and retry counts; report the accuracy-per-token frontier rather than raw accuracy.

Source video

DAG Harness with Explicit Blocking and Failure Semantics

movement-lab

Binding each node of a launch dependency graph to an autonomous agent with explicit blocking, retry-with-context, and rollback policies reduces human interventions per graph by a measurable margin versus an equivalent flat task list, without increasing duplicate or conflicting work.

Proof of concept

Implement a minimal orchestration layer over a real dependency graph where each node dispatches an agent, dependencies gate execution, failed nodes retry with prior-attempt context, and partial completions mark downstream nodes as blocked rather than failed; stress-test on a realistic multi-stage plan.

Measurement

Human interventions per graph, duplicate-action rate, deadlock/blocked-node count, and wall-clock time from graph submission to completion.

Source video

State-vs-Transcript Diagnosis Study

gatehouse

Operators diagnosing injected faults in a multi-agent run reach a correct root cause faster and more accurately when agent planning status, assigned tasks, and progress are rendered as positionable state objects next to each agent than when the same information is presented as a chronological chat transcript.

Proof of concept

Instrument a small orchestrator-plus-worker multi-agent setup to emit both a structured spatial state model and a text transcript, inject a fixed catalog of coordination faults (duplicated assignment, stalled dependency, wrong-target edit), and run a within-subject diagnosis task under both interfaces.

Measurement

Time-to-diagnose and diagnosis accuracy per fault type, plus reported confidence and time spent searching history.

Source video

Coordination-Overhead Tracer for Spatial Multi-Agent Systems

new

Quantifying orchestrator routing, task assignment, and redundant actions in a configurable-persona multi-agent canvas system reveals that coordination overhead (messages and duplicate work per completed task) grows non-linearly with agent count, and that a single designated orchestrator is not sufficient to bound it beyond a small team size.

Proof of concept

Extend a persona-based multi-agent canvas setup (configurable agent attributes plus one orchestrator) with tracing of every assignment, claim, and action, then sweep agent count on a fixed task set and attribute wasted actions to coordination rather than task difficulty.

Measurement

Messages and duplicate actions per completed task as a function of agent count, plus task completion rate and orchestrator queue depth.

Source video

Architectural implications

3

The agent's operating environment is the canvas runtime rather than a chat window.

Before

Agent actions are emitted as text in a conversation and the user interprets them.

After

Agent actions are spatial mutations (move, find, place shapes) inside a shared multiplayer canvas with viewports and cursors.

Consequence

The runtime must expose canvas primitives as tools and manage spatial state, making the canvas SDK an agent framework substrate.

Source video

Observability is derived from agent state placed on the canvas instead of parsed from transcripts.

Before

Understanding what a multi-agent system is doing requires reading lengthy chat logs.

After

Planning status, task assignment, and progress live next to each agent's avatar.

Consequence

Agents must publish structured, addressable state; logging becomes secondary to state visualization.

Source video

Planning artifacts double as execution topology.

Before

A dependency graph is a static plan handed to humans who execute tasks.

After

Each graph node is bound to an autonomous coding agent that executes it.

Consequence

Dependency/blocking semantics must be enforced by the orchestration layer, and graph correctness becomes execution correctness.

Source video

Tradeoffs and failure modes

3

Spatial grounding of LLMs

Benefit

General text-trained LLMs can be made canvas-capable via a harness without retraining.

Cost or risk

Requires building and maintaining a custom spatial wrapper (screenshots, JSON, prompts, tools), which is significant engineering work not provided by the model.

Aligning UI elements requires significant engineering work to bridge agents with visual space.
Open source video
Source video

Canvas as shared multi-agent workspace

Benefit

Multiplayer live sync, cursors, and viewports come out of the box, giving humans and many agents a common plane.

Cost or risk

Agent state and identity must be expressed as canvas objects, coupling agent internals to the canvas data model.

tldraw's SDK supports multiplayer live sync, cursors, and viewports out of the box.
Open source video
Source video

Spatial state over chat logs

Benefit

Users inspect agent state directly on the canvas for better transparency in multi-agent workflows.

Cost or risk

Loses the linear, searchable, diffable audit trail that chat logs provide; state must be structured to be rendered.

In multi-agent spatial setups, users can inspect agent states directly on the canvas rather than digging through lengthy chat interfaces.
Open source video
Source video

Open questions

4

Does a canvas/spatial harness measurably improve agent task success on 2D layout tasks versus text-only tooling, and by how much?

Why unresolved

The summary describes the harness and the failure modes it addresses but provides no benchmark, accuracy, or comparison numbers.

Research direction

Build an eval of canvas layout tasks (alignment, spatial search, shape movement) with and without the spatial harness and report success rate and token cost.

Source video

What is the right mix and cadence of screenshots, JSON state, and spatial prompts in the observation channel?

Why unresolved

The summary states the combination is used but gives no guidance on when each channel dominates or the latency/cost implications.

Research direction

Ablate observation modalities (screenshot-only, JSON-only, hybrid) on canvas tasks to measure accuracy versus token and latency cost.

Source video

How are multi-agent canvas workflows (fairies, orchestrator) evaluated for coordination failure, deadlock, or duplicated work?

Why unresolved

The fairness project is described as experimental with examples but no reliability results or coordination-failure analysis.

Research direction

Instrument the orchestrator/group-chat protocol with tracing of task assignment, blocking, and redundant actions to quantify coordination overhead.

Source video

Can a dependency graph bound to per-node coding agents handle failure, retries, and partial completion without human intervention?

Why unresolved

The tech tree concept is presented as a hackathon-era idea without described error handling or recovery semantics.

Research direction

Prototype a DAG-as-orchestration layer with explicit node retry, rollback, and dependency-blocking policies, then stress-test on a realistic launch plan.

Source video

Key claims

7
causalVerification needed

LLMs trained on text-in/text-out lack native intuition for 2D spatial positioning, resizing, and UI alignment.

Evidence

Because LLMs are trained on text-in/text-out, they lack native intuition for 2D spatial positioning, resizing, and UI alignment.

Question

Do measurable performance gaps on 2D layout tasks persist across model scales and with multimodal input?

Source video
opinionVerification needed

Teaching LLMs about canvas structure requires combining screenshots, JSON data, and spatial prompts.

Evidence

Teaching LLMs about canvas structure involves combining screenshots, JSON data, and spatial prompts.

Question

Which observation modality contributes most to canvas task success, and is the hybrid strictly better than the best single channel?

Source video
factualVerification needed

The tldraw agent starter kit wraps LLMs in a harness enabling them to work agentially on the canvas.

Evidence

The agent starter kit wraps LLMs in a harness enabling them to work agentially on the canvas.

Question

What exactly does the starter-kit harness provide (tools, loop, state serialization) and how portable is it to non-tldraw canvases?

Source video
factualVerification needed

tldraw's SDK supports multiplayer live sync, cursors, and viewports out of the box.

Evidence

tldraw's SDK supports multiplayer live sync, cursors, and viewports out of the box.

Question

What are the scaling limits, conflict-resolution semantics, and per-connection costs of the built-in multiplayer sync?

Source video
factualVerification needed

The tech tree app treats individual dependency graph tasks as autonomous coding agents.

Evidence

The tech tree app treats individual dependency graph tasks as autonomous coding agents.

Question

How are blocking, retries, and partial failures handled when graph nodes are autonomous agents?

Source video
factualVerification needed

Replit's agent canvas is built on top of tldraw.

Evidence

Miro competitors, slide designers, and Replit's agent canvas built on top of tldraw.

Question

Is tldraw confirmed as the underlying canvas engine for Replit's agent canvas, and what does that integration use from the SDK?

Source video
comparativeVerification needed

In multi-agent spatial setups, users can inspect agent states directly on the canvas rather than digging through chat interfaces.

Evidence

In multi-agent spatial setups, users can inspect agent states directly on the canvas rather than digging through lengthy chat interfaces.

Question

Does spatial state inspection reduce time-to-diagnose multi-agent issues compared with transcript-based debugging?

Source video

Connections

4