AI Engineer · Published 2026-09-21

The Dark Arts of Skill Engineering — Paul Bakaus, Renaissance Geek (Impeccable)

privateReview: unreviewedConfidence: high
skill engineeringHarness engineeringSub-agentsanti-attractordesign linterreactive loopskill-engineeringharness-engineeringGuardrailsreactive-loopmulti-agent-evaluation
Open on YouTube ↗

Summary

Overview

  • Speaker: Paul Bakaus
  • Channel: AI Engineer
  • Main topic: Skill engineering and building advanced harness systems for LLMs
  • Purpose: To share practical knowledge, design patterns, and lessons learned on scaling AI agent skills beyond simple prompting into robust harness engineering. Paul Bakaus presents 'The Dark Arts of Skill Engineering', exploring lessons learned from building 'Impeccable', a large enterprise application and skill. He contrasts basic prompting with 'harness engineering', detailing nine dark arts techniques such as blind sub-agent critiques, divergence enforcement, model routing, memory, interactive loops, compiler harnesses, and designing for the weakest model.

Topic Map

Introduction to Impeccable and Skill Engineering

  • Explanation: Introduction of Paul Bakaus and the Impeccable project, establishing the shift from prompt engineering to harness engineering.
  • Key claims:
    • Prompting is just the starter level; harness engineering is where you should end up.
    • Skills are an extension to coding harnesses.
  • Examples:
    • Building a kids reader iPad app using Claude code and frontend-design skill.
  • Terminology:
    • Impeccable
    • Skill engineering
    • Harness engineering
    • Frontend-design skill
    • Claude Code
  • Why it matters: Establishes the core philosophy that prompts alone are insufficient for complex, production-grade AI engineering.

The 9 Dark Arts of Skill Engineering

  • Explanation: Detailed breakdown of nine specific techniques and design patterns developed while building Impeccable.
  • Key claims:
    • Make them argue using blind sub-agents.
    • Force divergence instead of convergence.
    • Route in like a model.
    • Give them memory.
    • The script talks back.
    • Hooks that fight back.
    • Live wire the browser.
    • Compile to every harness.
    • Design for the weakest model.
  • Examples:
    • Using dual blind sub-agents for design critique to avoid anchoring bias.
    • Using anti-attractor seeds to force LLMs out of default aesthetic clusters.
  • Terminology:
    • Adversarial prompting
    • Anti-attractor
    • Sub-agents
    • Design linter
    • Prompt caching
    • Write-gate
  • Why it matters: Provides concrete architectural patterns for overcoming LLM biases, instruction following limits, and consistency issues.

Key Points

Prompting < Harness Engineering

  • Explanation: Simple prompts result in generic outputs (e.g., purple gradients, Inter font), whereas harness engineering structures constraints and agent behaviors.
  • Evidence: Analyzing the limitations of 55 lines of named bans in frontend-design prompts.
  • Practical implication: Engineers should wrap prompts in programmatic harnesses, linters, and multi-agent loops.

The Median is the Model's Gravity

  • Explanation: No amount of artisan prompt prose can overcome a model's latent space biases towards common choices.
  • Evidence: Tailwind's default theme defaults and historical web design trends.
  • Practical implication: You must use architectural mechanisms like anti-attractors to force divergence.

Blind Sub-Agents for Critique

  • Explanation: Asking an LLM to review its own work results in high self-ratings; spawning two blind sub-agents produces balanced critiques.
  • Evidence: Comparison of single-model code review vs. decoupled design director and deterministic detector sub-agents.
  • Practical implication: Implement adversarial sub-agent workflows for code review, design audits, and security checks.

Frameworks, Models & Processes

The Two Blind Sub-Agents Critique Pattern

  • How it works: Spawns two blind sub-agents (e.g. design director and deterministic detector) that evaluate work independently before the main thread synthesizes their findings.
  • Components:
    • Sub-agent A (Design Director: hierarchy, slop, heuristics)
    • Sub-agent B (Detector: browser evidence, deterministic)
    • Main thread synthesis
  • When to use: Code reviews, design reviews, security audits, and ranking outputs.

The Reactive Loop (Live Browser Integration)

  • How it works: A tiny server injects a picker into a dev page; dragging a knob sends server-side events back to the running agent thread.
  • Components:
    • Tiny server injecting picker
    • Node live-poll MJS
    • Server-side event listener
    • Agent auto-wake on exit
  • When to use: Visual tweaking, interactive debugging, and iterative refinement of UI elements.

Examples & Case Studies

Codex refused to spawn sub-agents when first prompted.

  • Illustrates: Different LLMs have different permission models and instruction-following boundaries.
  • Lesson: You must explicitly instruct models when to stop and ask the user if they lack sub-agent permissions.

Generating 100 shades with a random seed and ranking them via a sub-agent.

  • Illustrates: Forcing divergence in color palette generation to avoid repeating default choices.
  • Lesson: Use creative random seeds and ranking sub-agents to escape latent space gravity.

Actionable Takeaways

  • Immediate:
    • Stop relying on pure prose prompts for complex tasks.
    • Implement write-gate hooks to catch formatting and styling errors before files hit disk.
    • Use anti-attractor seeds to push LLM generation out of common clusters.
  • Strategic:
    • Treat skill engineering as an extension of coding harness design.
    • Design your skills for the weakest target model in your deployment fleet.
    • Build passive guardrails rather than relying on commands users forget to run.
  • Questions to investigate:
    • How do different LLM providers handle sub-agent spawning permissions?
    • What are the latency tradeoffs of multi-agent critique loops?
    • How can prompt caching be optimized when running reactive live browser loops?

Claims Worth Verifying

  • Codex has a different permission model than Claude Code regarding sub-agent spawning. (technical behavior)
  • Context.mjs can automatically self-update if a new version of Impeccable is available. (architectural feature)

Notable Quotes

"The median is the model's gravity." "Two blind opinions beat one confident guess." "Passive guardrails beat a command no one remembers to run."

Compressed Summary

  • Prompting is the starter level; harness engineering is required for production AI skills.
  • LLMs suffer from latent space gravity; use anti-attractor seeds to force divergence.
  • Two blind sub-agents reviewing work independently yield better critiques than self-review.
  • Live-wiring the browser via reactive loops enables visual steering.
  • Design for the weakest model in your fleet and implement strict non-compressible gates.
  • Keywords: skill engineering, harness engineering, sub-agents, anti-attractor, design linter, reactive loop
  • Core insight: Prompting is merely a starter level; building robust AI skills requires advanced harness engineering, multi-agent adversarial critiques, and passive guardrails.

Core insights

6
Mechanismmedium noveltymoderate evidence

Self-critique by the same model/thread yields inflated ratings; the fix is architectural, not prompt-level: spawn two *blind* sub-agents with different evaluation roles (a heuristic 'design director' and a deterministic, browser-evidence 'detector'), then have the main thread synthesize their independent findings.

Why it matters

It relocates review responsibility out of the authoring thread into decoupled evaluator agents, which is directly reusable for code review, security audits, and output ranking — and explains why 'review your own work' loops underperform.

Generalization

Any agentic pipeline where a single model both produces and grades output can be improved by separating generation from evaluation across independent, information-isolated evaluators with heterogeneous criteria.

Asking an LLM to review its own work results in high self-ratings; spawning two blind sub-agents produces balanced critiques.
Open source video
Two blind opinions beat one confident guess.
Open source video
Sub-agent A (Design Director: hierarchy, slop, heuristics)
Open source video
Mental Modelhigh noveltymoderate evidence

Latent-space bias toward the 'median' output cannot be fixed with better prompt prose (even 55 lines of named bans fails); divergence must be induced mechanically, e.g. random anti-attractor seeds that push generation out of default clusters, followed by sub-agent ranking of many candidates.

Why it matters

It reframes a class of output-quality problems (generic designs, default palettes, repeated tailwind-style defaults) as a sampling/distribution problem rather than an instruction-following problem, which changes what you build.

Generalization

When a model's modal output is undesirable, generate a wider candidate distribution with randomized seeds/constraints and select post hoc, instead of trying to negate the mode in natural language.

No amount of artisan prompt prose can overcome a model's latent space biases towards common choices.
Open source video
The median is the model's gravity.
Open source video
Analyzing the limitations of 55 lines of named bans in frontend-design prompts.
Open source video
Architecturemedium noveltymoderate evidence

Guardrails should be passive and mechanically enforced in the harness (write-gate hooks and linters that reject bad output before files hit disk) rather than expressed as instructions the model must remember to obey or commands users forget to run.

Why it matters

This is a boundary decision: constraints move from the probabilistic prompt layer to deterministic, non-compressible gates, improving reliability and eliminating a class of silent failures caused by context compression or model forgetfulness.

Generalization

Any rule that must hold with high probability should be enforced by a deterministic hook at the artifact boundary, not by prompt text that can be compressed away or ignored.

Passive guardrails beat a command no one remembers to run.
Open source video
Implement write-gate hooks to catch formatting and styling errors before files hit disk.
Open source video
Design for the weakest model in your fleet and implement strict non-compressible gates.
Open source video
Failure Modemedium noveltyweak evidence

Sub-agent spawning is not a portable capability: models differ in permission models (e.g. Codex refused to spawn sub-agents when first prompted), so skills that depend on multi-agent workflows must explicitly instruct the model to stop and ask the user when it lacks the permission.

Why it matters

A skill built and validated against one harness can silently degrade on another; this forces explicit capability negotiation into skill design rather than assuming a uniform tool surface.

Generalization

Multi-agent or tool-dependent agent designs need runtime capability detection and a defined degradation path, because provider permission surfaces are heterogeneous.

Codex refused to spawn sub-agents when first prompted.
Open source video
You must explicitly instruct models when to stop and ask the user if they lack sub-agent permissions.
Open source video
Compile to every harness.
Open source video
Mechanismhigh noveltymoderate evidence

A reactive loop turns the running agent thread into a long-lived process woken by external UI events: a small server injects a control (a picker/knob) into the dev page, drags are sent as server-side events back to the agent thread, and the agent auto-wakes on exit.

Why it matters

It treats the human as an event source into a persistent agent session, enabling visual/interactive steering without restarting or re-prompting the agent — a concrete alternative to turn-based chat control.

Generalization

Agent runtimes can expose interruptible, resumable sessions bound to external event streams (UI, telemetry, CI), making the agent a reactive daemon rather than a request/response function.

A tiny server injects a picker into a dev page; dragging a knob sends server-side events back to the running agent thread.
Open source video
Agent auto-wake on exit
Open source video
Practicemedium noveltymoderate evidence

Divergence is produced by generate-many-then-rank: sample a large candidate set from a random seed (e.g. 100 shades) and use a sub-agent to rank them, rather than asking for a single best answer.

Why it matters

It is a cheap, testable recipe for escaping default choices that composes with the anti-attractor idea and with sub-agent evaluation, and it is directly measurable (diversity and preference metrics).

Generalization

For any generative task with a strong modal attractor, oversample with randomized conditioning and select with an evaluator model rather than trusting a single greedy generation.

Generating 100 shades with a random seed and ranking them via a sub-agent.
Open source video
Use creative random seeds and ranking sub-agents to escape latent space gravity.
Open source video

Deep dives

5

Blind multi-agent critique vs self-review

Research question

Does blind multi-agent critique with heterogeneous evaluator roles measurably reduce self-evaluation bias compared to same-model self-review, and at what latency/cost?

Why

Self-evaluation bias is a structural flaw in single-agent workflows; separating generation from evaluation into independent, information-isolated sub-agents could be a general-purpose architectural fix for reliable AI pipelines, but its effectiveness and cost are unquantified.

Make them argue using blind sub-agents.
Open source video
Using dual blind sub-agents for design critique to avoid anchoring bias.
Open source video
Engineers should wrap prompts in programmatic harnesses, linters, and multi-agent loops.
Open source video
Source video

Anti-attractor seeding for escaping latent-space median

Research question

Do anti-attractor seeds (randomized constraints) followed by sub-agent ranking produce more diverse and preferred outputs than prompt-level bans, and how does diversity relate to human preference?

Why

If latent-space bias toward modal outputs cannot be overcome with better prompt prose, then divergence must be induced mechanically; this reframes a class of output-quality problems as a sampling/distribution problem, changing what tools engineers build.

No amount of artisan prompt prose can overcome a model's latent space biases towards common choices.
Open source video
Using anti-attractor seeds to force LLMs out of default aesthetic clusters.
Open source video
You must use architectural mechanisms like anti-attractors to force divergence.
Open source video
Analyzing the limitations of 55 lines of named bans in frontend-design prompts.
Open source video
Source video

Deterministic write-gate hooks vs prompt-based guardrails

Research question

What is the observed compliance rate of write-gate hooks versus explicitly prompted commands across sessions, and how does context compression affect prompt-based rule adherence?

Why

Moving reliability-critical rules from probabilistic prompts to deterministic harness gates could eliminate a class of silent failures caused by context compression or model forgetfulness, but it may reduce agent flexibility and add maintenance overhead.

Hooks that fight back.
Open source video
Design linter
Open source video
Write-gate
Open source video
Engineers should wrap prompts in programmatic harnesses, linters, and multi-agent loops.
Open source video
Source video

Reactive agent loops and prompt caching

Research question

How should prompt caching be structured when an agent runs as a long-lived, event-woken reactive thread, and what are the latency and cost implications of interactive steering?

Why

Treating the human as an event source into a persistent agent session enables visual/interactive steering without restarting the agent, but long-lived event-driven threads may undermine prompt-cache assumptions and complicate cost/latency budgeting.

Live wire the browser.
Open source video
The script talks back.
Open source video
Prompt caching
Open source video
Source video

Sub-agent permission heterogeneity and skill portability

Research question

How do different LLM providers and harnesses handle sub-agent spawning permissions, and can a skill detect and gracefully degrade when such permissions are unavailable?

Why

A skill built and validated against one harness can silently degrade on another; designing for capability detection and explicit stop-and-ask paths is necessary for portable multi-agent skills, but the permission surfaces are heterogeneous and poorly documented.

Compile to every harness.
Open source video
Design for the weakest model.
Open source video
Make them argue using blind sub-agents.
Open source video
Source video

Article ideas

4

The Self-Review Illusion: Why Blind Sub-Agents Are the Only Way to Critique AI Output

Self-evaluation by the same model is systematically inflated because the model cannot escape its own generation context; decoupling generation from evaluation via independent, information-isolated sub-agents is a necessary architectural fix for reliable AI pipelines.

Angle

Counterintuitive finding backed by architecture patterns, with practical implications for code review, design critique, and output ranking.

Source video

The Median is the Model's Gravity: Escaping Latent-Space Defaults with Anti-Attractor Seeding

Prompt-level bans fail because they operate in the same latent space that produces the undesired defaults; only mechanical randomization and oversampling followed by evaluator ranking can force divergence from modal outputs.

Angle

Technical deep dive with experimental evidence from frontend design, explaining why 55 lines of named bans didn't work and what to do instead.

Source video

Guardrails That Fight Back: Why Your Rules Should Be in the Harness, Not the Prompt

Reliability-critical constraints must be enforced by deterministic hooks at the artifact boundary, not by prompt instructions that can be compressed away or forgotten; passive guardrails are more reliable than commands users must remember to run.

Angle

Architecture comparison of prompt-based vs harness-based enforcement, with case studies of write-gates and linters.

Source video

The Reactive Agent: Turning LLM Threads into Event-Driven Daemons

Agent runtimes should expose persistent, interruptible sessions bound to external event streams, transforming agents from request/response functions into reactive services that can be steered interactively without restart.

Angle

Vision and architecture for event-driven agent sessions, with a concrete example of live browser steering and discussion of prompt-caching implications.

Source video

Project ideas

3

Blind Critic Bench

beyond-evals

Blind multi-agent critique with two independent evaluator roles reduces self-evaluation bias compared to same-model self-review, as measured by increased correlation with human expert ratings.

Proof of concept

Build a pipeline that takes generated artifacts (e.g., UI designs), runs three conditions: self-review, single blind sub-agent, and two blind sub-agents with distinct criteria (heuristic vs evidence-based); collect ratings and compare against human expert scores.

Measurement

Bias (mean difference between model rating and human rating), rank correlation with human ratings, and cost/latency per condition.

Source video

Anti-Attractor Sampler

movement-lab

Random anti-attractor seeds followed by sub-agent ranking produce a wider and more preferred distribution of outputs than prompt-level bans, without increasing cost per accepted artifact.

Proof of concept

For a design generation task, compare two conditions: (A) prompt with a 55-line ban list, (B) generate 100 candidates with random anti-attractor seeds and rank with a sub-agent; measure diversity and human preference.

Measurement

Diversity metrics (e.g., embedding entropy, unique color palettes, layout clusters) and win rate in pairwise human comparisons against baseline.

Source video

Write-Gate Compliance Harness

gatehouse

Deterministic write-gate hooks achieve higher rule compliance (e.g., style, formatting) than prompt-based instructions across multiple sessions, especially when context compression occurs.

Proof of concept

Implement a linter that blocks file writes on violations in a coding agent, and compare with a prompt that instructs the model to avoid those violations; run 50 coding sessions each and count violations.

Measurement

Compliance rate (violations per 100 writes), false positive rate, and developer override frequency.

Source video

Architectural implications

3

Evaluation is split out of the producing agent into independent blind sub-agents with distinct, non-overlapping evaluation criteria.

Before

The generating agent also reviews and reports on its own output.

After

Generation, heuristic critique, and deterministic evidence collection are separate roles, with the main thread only synthesizing.

Consequence

Review quality becomes an orchestration concern; you can swap, add, or version evaluators independently, but you inherit multi-agent latency and permission dependencies.

Source video

Constraint enforcement is moved below the prompt layer into deterministic hooks at the artifact boundary.

Before

Rules live as long prose ban-lists inside the prompt.

After

Linters and write-gates validate and block output before it is persisted.

Consequence

Rules become non-compressible and consistently applied, but every new rule adds harness code to maintain and can conflict with model autonomy.

Source video

Skills are authored as portable artifacts compiled to multiple harnesses and validated against the weakest model in the fleet.

Before

A skill is tuned and assumed to work against the strongest available model/harness.

After

Capability differences (e.g. sub-agent permissions) are designed for up front, with explicit stop-and-ask paths.

Consequence

Skill engineering becomes a compatibility/matrix problem, trading peak capability on strong models for uniform behavior across a heterogeneous fleet.

Source video

Tradeoffs and failure modes

5

Multi-agent critique loops

Benefit

Blind, role-separated sub-agents yield balanced critiques instead of inflated self-review scores.

Cost or risk

Added latency, cost, and orchestration complexity; benefit magnitude and cost profile are not quantified in the summary.

What are the latency tradeoffs of multi-agent critique loops?
Open source video
Source video

Divergence via seeded oversampling

Benefit

Escapes the model's latent-space median rather than replicating default choices.

Cost or risk

Generating and ranking large candidate sets (e.g. 100 options) multiplies generation and evaluation cost per artifact.

Generating 100 shades with a random seed and ranking them via a sub-agent.
Open source video
Source video

Sub-agent permission heterogeneity

Benefit

Adversarial/multi-agent workflows improve output quality where sub-agents are permitted.

Cost or risk

Skills silently fail or behave differently on harnesses whose permission models disallow spawning sub-agents.

Codex refused to spawn sub-agents when first prompted.
Open source video
Source video

Passive guardrails over explicit commands

Benefit

Rules apply automatically and cannot be forgotten or compressed away.

Cost or risk

Deterministic gates reduce agent flexibility and require harness-level maintenance; stricter hooks may reject legitimate novel output.

Passive guardrails beat a command no one remembers to run.
Open source video
Source video

Reactive live-browser loops vs. prompt caching

Benefit

Interactive steering of the agent from the browser without restarting the session.

Cost or risk

Long-lived, event-driven agent threads may undermine prompt caching assumptions and complicate cost/latency budgeting.

How can prompt caching be optimized when running reactive live browser loops?
Open source video
Source video

Open questions

5

How do different LLM providers and harnesses handle sub-agent spawning permissions, and can a skill detect them at runtime?

Why unresolved

The summary reports a single observed refusal (Codex) without a systematic comparison or a detection mechanism.

Research direction

Build a capability-probe matrix across harnesses and define a standardized fallback/stop-and-ask contract for skills that require sub-agents.

Source video

What are the latency and cost tradeoffs of blind multi-agent critique loops versus single-agent self-review?

Why unresolved

Listed explicitly as an open question; no measurements are given.

Research direction

Benchmark review quality (agreement with human raters) against wall-clock latency and token cost for 1 vs 2 vs N evaluator agents.

Source video

Do blind sub-agent critiques actually reduce self-evaluation bias, or merely increase diversity of opinions?

Why unresolved

The summary asserts improved balance but provides no evaluation methodology or metrics.

Research direction

Run controlled studies measuring self-rating inflation versus blind-panel ratings across models and task types.

Source video

How should prompt caching be structured when the agent runs as a long-lived, event-woken reactive thread?

Why unresolved

Raised as an unresolved question; event-driven loop structure may invalidate cache prefixes.

Research direction

Instrument cache hit rates across reactive-loop sessions and design stable prompt prefixes for persistent agent threads.

Source video

What is the right way to enforce anti-divergence without over-constraining legitimate creative output?

Why unresolved

The summary gives mechanisms (anti-attractor seeds, write-gates) but no guidance on threshold or scope.

Research direction

Measure output diversity distributions and human preference under varying anti-attractor strength and gate strictness.

Source video

Key claims

8
comparativeVerification needed

Asking an LLM to review its own work results in high self-ratings, whereas two blind sub-agents produce more balanced critiques.

Evidence

Asking an LLM to review its own work results in high self-ratings; spawning two blind sub-agents produces balanced critiques.

Question

Does blind multi-agent critique measurably reduce self-evaluation bias compared to same-model self-review on standardized tasks?

Source video
causalVerification needed

No amount of prompt prose can overcome a model's latent-space bias toward common choices; architectural mechanisms such as anti-attractors are required.

Evidence

No amount of artisan prompt prose can overcome a model's latent space biases towards common choices.

Question

Do anti-attractor seeds measurably shift output distributions away from modal choices more than explicit prompt-level bans?

Source video
factualVerification needed

55 lines of named bans in frontend-design prompts were analyzed and found insufficient to prevent generic outputs.

Evidence

Analyzing the limitations of 55 lines of named bans in frontend-design prompts.

Question

What was the measured failure rate of named-ban prompts on generic-output detection tests?

Source video
factualVerification needed

Codex has a different permission model from Claude Code regarding sub-agent spawning.

Evidence

Codex refused to spawn sub-agents when first prompted.

Question

What exactly does each harness's permission model allow for sub-agent/tool spawning, and how does it fail when disallowed?

Source video
factualVerification needed

Context.mjs can automatically self-update if a new version of Impeccable is available.

Evidence

Context.mjs can automatically self-update if a new version of Impeccable is available.

Question

How is the self-update validated, and what are the security and reproducibility implications of auto-updating a skill artifact at runtime?

Source video
factualVerification needed

A small server can inject a UI control into a dev page so that user interaction sends server-side events back into the running agent thread, which auto-wakes on exit.

Evidence

A tiny server injects a picker into a dev page; dragging a knob sends server-side events back to the running agent thread.

Question

What is the event protocol and wake mechanism, and how does it behave under concurrent or interrupted sessions?

Source video
opinionVerification not requested

Skills should be designed for the weakest target model in the deployment fleet rather than the strongest.

Evidence

Design your skills for the weakest target model in your deployment fleet and implement strict non-compressible gates.

Source video
comparativeVerification needed

Passive guardrails are more reliable than commands users or agents must remember to run.

Evidence

Passive guardrails beat a command no one remembers to run.

Question

What is the observed compliance rate of write-gate hooks versus explicitly prompted commands across sessions?

Source video

Connections

5