Joseph Kasser · Published 2021-06-22

Systems thinking as it applies to systems engineering

privateReview: unreviewedConfidence: high
systems thinkingsystems engineeringholistic thinkinghkmfparadigmsystems-thinkingdecompositionabstraction-hierarchyproblem-formulationoptimization-paradoxmulti-perspective-reviewprocess-conformanceevaluation
Open on YouTube ↗

Summary

Overview

  • Speaker: Prof Joseph E Kasser
  • Channel: Joseph Kasser
  • Main topic: Systems thinking, holistic thinking perspectives, and systems engineering education and practice
  • Purpose: To challenge traditional systems engineering paradigms, introduce holistic thinking perspectives, and provide a framework for improving systems engineering education and practice. Prof Joseph E Kasser presents a critical examination of traditional systems engineering, arguing that it relies on a flawed paradigm rooted in conventional reductionist thinking rather than true systems thinking. Through frameworks like the Holistic Thinking Perspectives (HTP) and the Hitchins-Kasser-Massie Framework (HKMF), Kasser explains how systems engineering is often taught and practiced incorrectly, focusing on process and tools rather than understanding problematic situations and capturing emergent system behavior.

Topic Map

Apologies and Warning

  • Explanation: Introduction to the presentation, warning that it presents traditional systems engineering in a different paradigm that may challenge or offend conventional views.
  • Key claims:
    • Traditional systems engineering is viewed through a different paradigm.
    • The talk is designed to make you think and take you out of your comfort zone.
    • Some information may be dated due to years of accumulated research.
  • Examples:
    • Presentations based on 20 years of research and earlier slides.
  • Terminology:
    • Traditional systems engineering
    • Paradigm
    • Holistic thinking
  • Why it matters: Establishes the critical tone and unorthodox perspective of the presentation.

Problem-Solving: Conventional vs. Systems Thinking

  • Explanation: Contrasts conventional thinking, which isolates parts to understand behavior, with systems thinking, which explores the emergent nature of the system as a whole.
  • Key claims:
    • Conventional thinking isolates parts to understand behavior.
    • Systems thinking explores emergent properties of the whole system and its interactions with adjacent systems.
    • Conventional thinking does not explain where solutions come from.
  • Examples:
    • A car not starting on a cold winter morning; conventional thinking looks for a dead battery, while systems thinking examines the car in its environment and traffic context.
  • Terminology:
    • Conventional thinking
    • Systems thinking
    • Emergent nature
    • Adjacent systems
  • Why it matters: Highlights the fundamental difference in how problems are approached under different mindsets.

The Holistic Thinking Perspectives (HTP)

  • Explanation: Presents nine holistic thinking perspectives derived from research and Barry Richmond's seven streams of system thinking, categorized into external, internal, progressive, quantitative, and scientific.
  • Key claims:
    • There are nine holistic thinking perspectives analogous to the blind men and the elephant parable.
    • External perspectives cover big picture and operational views.
    • Internal perspectives cover functional and structural views (analysis).
    • Progressive perspectives cover generic, continuum, and temporal views.
    • Quantitative and scientific perspectives complete the set.
  • Examples:
    • Applying the nine perspectives to a camera (big picture, operational, functional, structural, generic, continuum, temporal, quantitative, scientific).
  • Terminology:
    • Holistic Thinking Perspectives
    • Big Picture
    • Operational
    • Functional
    • Structural
    • Generic
    • Continuum
    • Temporal
    • Quantitative
    • Scientific
  • Why it matters: Provides a comprehensive framework for viewing systems from multiple angles to achieve full understanding.

The Principle of Hierarchies and Optimization Paradox

  • Explanation: Discusses Miller's Law (7 plus or minus 2 objects), fractal hierarchies, and the systems optimization paradox regarding optimizing subsystems vs. the whole system.
  • Key claims:
    • The human brain can only hold about 7 plus or minus 2 objects at once (Miller's Rule).
    • Hierarchies are fractal; systems are subsystems of larger systems.
    • Optimizing subsystems independently does not lead to a system optimum; you must optimize the system as a whole.
  • Examples:
    • Partitioning a complex diagram into six systems and hiding internal details to reduce complexity.
  • Terminology:
    • Miller's Rule
    • Fractal hierarchies
    • Systems optimization paradox
    • Subsystem optimization
  • Why it matters: Explains how to manage complexity by dealing with appropriate hierarchical levels.

Systems Engineering Education and Standards

  • Explanation: Critiques how systems engineering is taught in academia and defined in standards (MIL-STD-499, EIA-632, IEEE-1220, CMMI, ISO-15288), arguing they focus on management and process rather than problem formulation.
  • Key claims:
    • Academic courses focus on parts, lecture-based learning, and process rather than how-to problem solving.
    • Standards like DoD Directive 5000 and EIA-632 focus on managing the process (management of systems engineering) rather than doing systems engineering.
    • Early stages of problem formulation and conceptualization are missing from standards.
  • Examples:
    • The 'egg' process diagram and Waterfall lifecycle models showing lack of front-end problem exploration.
  • Terminology:
    • Systems engineering education
    • SEDC
    • DoD 5000
    • EIA-632
    • IEEE-1220
    • Waterfall lifecycle
  • Why it matters: Identifies root causes of project failures and educational gaps in the discipline.

Pure, Applied, and Domain Systems Engineering (SETA)

  • Explanation: Defines Systems Engineering The Activity (SETA) and categorizes it into pure systems engineering (cognitive skills), applied systems engineering (activities like requirements and V&V), and domain systems engineering.
  • Key claims:
    • Systems engineering is analogous to mathematics (pure and applied).
    • Pure systems engineering involves cognitive skills and systems thinking.
    • Applied systems engineering involves activities like requirements management, architecting, and V&V.
    • Domain systems engineering applies these in specific fields.
  • Examples:
    • Traffic congestion problem: problem domain (traffic management), solution domain (subway system), implementation domain (tunnel boring).
  • Terminology:
    • SETA
    • Pure systems engineering
    • Applied systems engineering
    • Domain systems engineering
  • Why it matters: Clarifies the nature of systems engineering activity and resolves confusion in definitions.

Types of Systems Engineers and George's Dilemma

  • Explanation: Categorizes systems engineers into five types based on capabilities (from problem causer to innovator-leader) and illustrates the plight of a nominal 'good' systems engineer (George) trapped in the B-paradigm.
  • Key claims:
    • There are five types of systems engineers: Type I (problem causer), Type II (apprentice doer), Type III (problem solver), Type IV (problem formulator), Type V (innovator-leader).
    • George follows standards meticulously but fails because he cannot understand what the customer really wants.
    • Outstanding systems engineers use holistic models (CONOPS) and consider system-generic requirements.
  • Examples:
    • George reading hundreds of customer requirements, creating hierarchical models using MBSE tools, yet failing because requirements keep changing.
  • Terminology:
    • Five types of systems engineers
    • A-paradigm
    • B-paradigm
    • CONOPS
    • System-generic requirements
  • Why it matters: Explains why projects fail despite following standard processes and highlights the superiority of outstanding systems engineers.

The Hitchins-Kasser-Massie Framework (HKMF)

  • Explanation: Introduces the HKMF matrix combining Hitchins' five layers (product, system, business, supply chain, socio-economic) with lifecycle phases to analyze systems engineering education and practice.
  • Key claims:
    • Systems engineering practice is heavily concentrated in layer 2 (system) and phases B-F (requirements, design, integration), ignoring layer 3-5 and front-end/operations.
    • Many systems engineering master's degrees do not require a single systems engineering course.
    • Traditional processes treat systems engineering as a white box while operations treat it as a black box.
  • Examples:
    • HKMF area coverage analysis showing gaps in academic curricula and standards.
  • Terminology:
    • HKMF
    • Hitchins layers
    • Lifecycle phases
    • White box vs. black box
  • Why it matters: Provides an analytical tool to evaluate educational curricula and professional practices.

Conclusion, Situation Awareness States, and Lessons Learned

  • Explanation: Summarizes key takeaways, presents a situation awareness state diagram (from unaware to changing the situation), and offers advice on becoming an outstanding systems engineer.
  • Key claims:
    • Conclusions and inferences are only as good as your domain knowledge.
    • Communications and risk management are key to success.
    • Situation awareness states range from not realizing something needs to change to actively improving the situation.
  • Examples:
    • Plastic bag tree analogy illustrating the limits of lack of domain knowledge.
    • Video clip of a man stepping into mud before a woman ('ladies first' explained by risk management).
  • Terminology:
    • Domain knowledge
    • Situation awareness states
    • Risk management
    • CRIP charts
  • Why it matters: Actionable guidance for practitioners to improve their systems thinking and career impact.

Key Points

Conventional vs. Systems Thinking

  • Explanation: Conventional thinking isolates parts to understand behavior, whereas systems thinking explores emergent properties of the whole system in its environment.
  • Evidence: Car failure example showing environmental and interaction factors.
  • Practical implication: Engineers must look beyond isolated components to understand complex problems.

Nine Holistic Thinking Perspectives

  • Explanation: Comprehensive framework covering big picture, operational, functional, structural, generic, continuum, temporal, quantitative, and scientific perspectives.
  • Evidence: Application to camera example across all nine dimensions.
  • Practical implication: Using multiple perspectives ensures complete understanding of a system.

Flaws in Systems Engineering Education

  • Explanation: Academic programs focus on teaching parts, lecture-based learning, and process rather than problem formulation and systems thinking.
  • Evidence: HKMF analysis showing master's programs lacking core systems engineering topics.
  • Practical implication: Curricula need revision to incorporate holistic thinking and problem formulation.

Standards Emphasize Management, Not Engineering

  • Explanation: Standards like DoD 5000, EIA-632, and IEEE-1220 focus on management of the process rather than actual systems engineering tasks.
  • Evidence: Mapping of standards categories showing absence of front-end conceptualization.
  • Practical implication: Relying solely on standards can lead to producing the wrong system efficiently.

The Five Types of Systems Engineers

  • Explanation: Engineers range from Type I (problem causer) to Type V (innovator-leader) based on cognitive and problem-solving capabilities.
  • Evidence: Kasser, Hitchins and Huynh (2009) taxonomy.
  • Practical implication: Matching the right type of engineer to the situation prevents project failure.

Domain Knowledge is Paramount

  • Explanation: Inferences, designs, and conclusions are only as good as your domain knowledge.
  • Evidence: Plastic bag tree anecdote illustrating the danger of lacking domain context.
  • Practical implication: Systems engineers must acquire deep domain knowledge or collaborate with domain experts.

Frameworks, Models & Processes

Holistic Thinking Perspectives (HTP)

  • How it works: Views a system through nine distinct lenses grouped into external, internal, progressive, quantitative, and scientific categories.
  • Components:
    • Big Picture
    • Operational
    • Functional
    • Structural
    • Generic
    • Continuum
    • Temporal
    • Quantitative
    • Scientific
  • When to use: When exploring complex problems and understanding systems as a whole.

Hitchins-Kasser-Massie Framework (HKMF)

  • How it works: A matrix combining Hitchins' five systemic layers with lifecycle phases to map educational coverage and systems engineering activities.
  • Components:
    • Product Layer
    • System Layer
    • Business/SOS Layer
    • Supply Chain Layer
    • Socio-Economic Layer
    • Lifecycle Phases (Needs to Disposal)
  • When to use: To evaluate educational programs, system architectures, and organizational responsibilities.

Systems Engineering Problem-Solving Paradigm (SDP)

  • How it works: An iterative problem-solving process mapping problem definition, solution conception, trade-offs, strategy formulation, implementation, and verification.
  • Components:
    • Define problem space
    • Conceive solutions
    • Trade-off analysis
    • Implement solution system
    • Verify solution
  • When to use: During system development lifecycles.

Situation Awareness States

  • How it works: A six-stage progression model describing awareness and response to problematic situations.
  • Components:
    • Not realizing something needs to change
    • Identifying that something needs to change
    • Accepting situation
    • Grumbling and complaining
    • Working out how to make the change
    • Doing something / Walking away
  • When to use: Assessing organizational readiness and personal reaction to problems.

Examples & Case Studies

A car fails to start on a cold winter morning.

  • Illustrates: Difference between conventional thinking (isolating parts like a dead battery) and systems thinking (considering environment, traffic, and car interactions).
  • Lesson: Systems thinking looks at emergent behavior and context rather than just isolated components.

Kasser observed pink plastic bags on a tree in Singapore upon first arrival.

  • Illustrates: The risk of making flawed inferences without domain knowledge.
  • Lesson: Conclusions and designs are only as good as your domain knowledge.

George meticulously follows standard requirements processes for a customer.

  • Illustrates: The limitation of the B-paradigm where requirements constantly change because the true customer need wasn't modeled.
  • Lesson: Outstanding systems engineers build holistic CONOPS models capturing system-generic and customer needs.

Actionable Takeaways

  • Immediate:
    • Recognize the difference between conventional and systems thinking.
    • Apply the nine Holistic Thinking Perspectives to your projects.
    • Assess whether your projects focus on problem formulation or just process management.
  • Strategic:
    • Integrate systems thinking and front-end problem exploration into systems engineering education.
    • Use frameworks like HKMF to evaluate organizational and educational gaps.
    • Transition from nominal systems engineering (B-paradigm) to holistic modeling (A-paradigm).
  • Questions to investigate:
    • How can academic curricula bridge the gap between process standards and systems thinking?
    • Are current MBSE tools actually helping understand customer needs or just managing documents?
    • How can domain knowledge be better integrated into systems engineering practice?

Claims Worth Verifying

  • Many master's degrees in systems engineering do not require a single systems engineering course. (educational claim)
  • DoD Directive 5000.2 gutted systems engineering by removing the front end and introducing IPPD. (historical claim)
  • Joint Strike Fighter (JSF) cost overrun was predicted by Kasser in 2001. (predictive claim)

Notable Quotes

"When we understand that slide, we'll have won the war." (at 16:38) "The principle of suboptimization states that optimization of each subsystem independently will not lead in general to a system optimum." (at 20:15) "Standards might help you produce the wrong system more effectively." (at 42:24) "Systems, even very large systems, are not developed by the tools of Systems Engineering, but only by the engineers using the tools." (at 41:31)

Compressed Summary

  • Traditional systems engineering relies on a flawed reductionist paradigm.
  • Holistic Thinking Perspectives (HTP) provide nine lenses for complete system understanding.
  • Standards focus on management processes rather than front-end problem formulation.
  • George's dilemma shows why standard requirement processes fail without holistic CONOPS modeling.
  • Domain knowledge is essential for valid inferences and design decisions.
  • Keywords: systems thinking, systems engineering, holistic thinking, hkmf, paradigm
  • Core insight: True systems engineering requires holistic thinking and problem formulation rather than blind adherence to process-heavy standards and tools.

Core insights

7
Tradeoffmedium noveltymoderate evidence

Systems optimization paradox: locally optimizing subsystems does not produce a system optimum; the objective function must be defined and optimized at the whole-system level, meaning component-level KPI tuning can actively degrade global performance.

Why it matters

Directly applies to multi-agent and modular pipelines where each agent/tool is independently tuned for its own success metric (accuracy, latency, token budget) while end-to-end task success degrades. Suggests architecture should carry an explicit system-level objective that subcomponents are scored against, not alongside.

Generalization

Any decomposed pipeline with per-component metrics risks convergence to a local optimum that is worse than the unoptimized whole; the system objective must be the single optimization target and subcomponent metrics must be derived from it.

Optimizing subsystems independently does not lead to a system optimum; you must optimize the system as a whole.
Open source video
Mechanismmedium noveltymoderate evidence

Complexity is managed by partitioning a representation into roughly six-to-seven-element chunks and deliberately hiding internal details, exploiting Miller's Rule (7 plus 2 objects in working memory). Hierarchies are fractal, so the same chunking rule reapplies at every level.

Why it matters

Gives a concrete, quantitative target for abstraction boundaries and context construction: interface surfaces should expose on the order of 6-7 elements and encapsulate the rest. This is a design rule for tool schemas, sub-agent interface descriptions, and context packing rather than a vague call for 'managing complexity'.

Generalization

Abstraction boundaries should be sized to the consumer's working-memory limit (human or model) and recursively applied, so each level presents a small fixed fan-out and hides internals.

The human brain can only hold about 7 plus or minus 2 objects at once (Miller's Rule).
Open source video
Partitioning a complex diagram into six systems and hiding internal details to reduce complexity.
Open source video
Hierarchies are fractal; systems are subsystems of larger systems.
Open source video
Mental Modelhigh noveltymoderate evidence

Holistic Thinking Perspectives define nine distinct viewpoints (big picture, operational, functional, structural, generic, continuum, temporal, quantitative, scientific), each of which yields partial understanding — the blind-men-and-elephant problem. Full understanding requires systematically rotating through all perspectives rather than one default view.

Why it matters

Provides a ready-made multi-perspective checklist that could be operationalized as an evaluation rubric or specification-review template for a system/agent design: does the design have an operational view, a temporal view, a quantitative view, etc.? It explains why single-view reviews miss failures.

Generalization

Multi-view review protocols over a fixed enumerated perspective set reduce blind spots relative to ad hoc or single-perspective review, and can be turned into automated rubric checks.

There are nine holistic thinking perspectives analogous to the blind men and the elephant parable.
Open source video
External perspectives cover big picture and operational views.
Open source video
Internal perspectives cover functional and structural views (analysis).
Open source video
Progressive perspectives cover generic, continuum, and temporal views.
Open source video
Architecturemedium noveltymoderate evidence

Phase separation: systems engineering splits into pure (cognitive skills and systems thinking), applied (requirements, architecting, V&V activities), and domain (field-specific application) — analogous to pure vs. applied mathematics. The cognition layer is distinct from the activity layer.

Why it matters

Argues for an architectural boundary between the reasoning/thinking layer and the executing activity layer. In agent terms: problem framing and modeling should be a separate, independently evaluated capability from tool invocation, requirements bookkeeping, and verification.

Generalization

Separate the cognitive framing capability from the procedural activity capability so each can be taught, measured, and replaced independently.

Pure systems engineering involves cognitive skills and systems thinking.
Open source video
Applied systems engineering involves activities like requirements management, architecting, and V&V.
Open source video
Systems engineering is analogous to mathematics (pure and applied).
Open source video
Mental Modelmedium noveltymoderate evidence

Capability taxonomy of systems engineers runs from Type I (problem causer) through Type II (apprentice doer), Type III (problem solver), Type IV (problem formulator), Type V (innovator-leader); the differentiating top capability is problem formulation, not problem solving.

Why it matters

Gives a staged capability ladder for a technical function, useful as a model for grading agent capability tiers or for deciding where to place human/agent responsibility: most tooling and process investment targets Type III behavior (solving stated problems), while the high-leverage failure is at formulation.

Generalization

Maturity scales should be anchored on ability to formulate the problem, not on throughput of solving already-formulated problems.

There are five types of systems engineers: Type I (problem causer), Type II (apprentice doer), Type III (problem solver), Type IV (problem formulator), Type V (innovator-leader).
Open source video
Failure Modehigh noveltymoderate evidence

Process-conformance failure mode: a practitioner ('George') who meticulously follows standards fails because he cannot determine what the customer actually wants — process compliance is not sufficient, and the standards themselves omit the front-end problem formulation and conceptualization stages.

Why it matters

Names a concrete, observed failure mode that maps to agent systems that pass step-by-step workflow checks yet miss user intent. Trustworthy systems need intent elicitation and objective modeling as first-class steps, plus evaluation of whether the underlying situation was understood, not only whether the prescribed steps ran.

Generalization

Evaluation harnesses that score procedural conformance can certify systems that fail on intent alignment; front-end situation understanding must be a measured stage and not assumed.

George follows standards meticulously but fails because he cannot understand what the customer really wants.
Open source video
Early stages of problem formulation and conceptualization are missing from standards.
Open source video
Standards like DoD Directive 5000 and EIA-632 focus on managing the process (management of systems engineering) rather than doing systems engineering.
Open source video
Mental Modelmedium noveltymoderate evidence

Conventional (reductionist) thinking isolates parts to understand behavior but does not explain where solutions come from; systems thinking instead targets emergent properties of the whole and its interactions with adjacent systems.

Why it matters

Frames decomposition as diagnostic, not generative: analyzing parts explains behavior but does not generate candidate solutions. Solution generation appears to require modeling the system in its environment and its adjacency to other systems — relevant to how agent systems generate plans versus how they diagnose failures.

Generalization

Decomposition is suited to explaining and debugging existing behavior; generating novel solutions requires modeling the whole in context, including interaction with adjacent systems.

Conventional thinking isolates parts to understand behavior.
Open source video
Conventional thinking does not explain where solutions come from.
Open source video
Systems thinking explores emergent properties of the whole system and its interactions with adjacent systems.
Open source video

Deep dives

5

Operationalizing the nine Holistic Thinking Perspectives as an automated design-review rubric

Research question

Can the nine Holistic Thinking Perspectives (big picture, operational, functional, structural, generic, continuum, temporal, quantitative, scientific) be encoded as a detection-and-scoring rubric that a model can apply to a system or agent design artifact, and does multi-perspective coverage catch defects that single-perspective review misses?

Why

A fixed enumerated perspective set turns a vague instruction ('think holistically') into a countable coverage check. If it works, it becomes a reusable review gate for architecture docs, tool schemas, and agent specifications, and it supplies a concrete explanation for why single-view reviews pass designs that later fail.

There are nine holistic thinking perspectives analogous to the blind men and the elephant parable.
Open source video
External perspectives cover big picture and operational views.
Open source video
Internal perspectives cover functional and structural views (analysis).
Open source video
Progressive perspectives cover generic, continuum, and temporal views.
Open source video
Source video

Propagating a whole-system objective down to independently tunable components

Research question

In a decomposed agent or software pipeline, what mechanism makes component metrics derive from a single system-level objective, and does per-component optimization measurably degrade end-to-end outcomes compared with whole-system objective optimization?

Why

This is the highest-leverage architectural consequence in the set: it converts 'per-agent success metrics' from an assumed good into a suspected failure mode, and it demands instrumentation that ties component behavior to end-to-end outcomes rather than to local scores.

Optimizing subsystems independently does not lead to a system optimum; you must optimize the system as a whole.
Open source video
Source video

Making intent elicitation and problem formulation a measured stage, not an assumed one

Research question

Can under-specified or misunderstood user intent be detected before solution work begins, and does inserting an explicit, scored formulation stage reduce rework relative to process-only workflows?

Why

Names a concrete failure mode that transfers directly to agent systems that pass step-by-step workflow checks yet miss user intent. If formulation is unmeasured, evaluation harnesses will certify process-compliant failures; the fix is a first-class gate plus an intent-alignment score.

George follows standards meticulously but fails because he cannot understand what the customer really wants.
Open source video
Early stages of problem formulation and conceptualization are missing from standards.
Open source video
Standards like DoD Directive 5000 and EIA-632 focus on managing the process (management of systems engineering) rather than doing systems engineering.
Open source video
Source video

Does the ~7-element chunking constraint transfer from human working memory to model context and tool/interface schemas?

Research question

At what element count per level do tool schemas and context layouts begin to degrade model task success, and does a small fixed fan-out with hidden internals beat a wide flat surface?

Why

This turns a cognitive-science rule into a testable interface design budget for tool schemas, sub-agent interface descriptions, and context packing — and tests whether the constraint is a property of human reviewers only or of any consumer, including models.

The human brain can only hold about 7 plus or minus 2 objects at once (Miller's Rule).
Open source video
Partitioning a complex diagram into six systems and hiding internal details to reduce complexity.
Open source video
Hierarchies are fractal; systems are subsystems of larger systems.
Open source video
Source video

Do capability tiers separate cleanly — problem solver vs. problem formulator

Research question

Do assessment tasks that explicitly discriminate formulation ability from solving ability produce cleanly separated performance tiers, or is the Type III/Type IV boundary a descriptive spectrum with no measurement signature?

Why

Most tooling and process investment targets solving already-stated problems, while the high-leverage failure sits at formulation. If the tiers don't separate empirically, maturity scales anchored on formulation ability are unsupported; if they do, responsibility assignment between humans and agents can be staged on evidence.

There are five types of systems engineers: Type I (problem causer), Type II (apprentice doer), Type III (problem solver), Type IV (problem formulator), Type V (innovator-leader).
Open source video
Source video

Article ideas

4

Process Conformance Is Not Intent Alignment: Why Passing Every Step Still Fails the User

Evaluation harnesses and lifecycle standards that score procedural conformance can certify systems that never understood the situation; front-end problem formulation must be a measured stage with its own intent-alignment score, or process-compliance becomes a false quality signal.

Angle

Take the standard-following practitioner who fails on intent as the canonical failure, then map it onto agent pipelines that pass workflow checks while missing user goals, and argue for splitting intent elicitation out as a scored gate preceding solution work.

Source video

The Optimization Paradox in Decomposed AI Pipelines: When Tuning Every Agent Makes the System Worse

Independently tuning each agent or tool against its own metric converges to a local optimum that can be worse than the untuned whole; the system-level objective must be the single optimization target and component metrics must be derived from it, not optimized alongside it.

Angle

Treat per-component KPI tuning — accuracy, latency, token budget — as an anti-pattern, show why emergent global behavior is an unchecked side effect of local tuning, and specify the instrumentation needed to score components against end-to-end outcomes.

Source video

Decomposition Explains, Context Generates: Splitting Analysis From Solution Finding

Decomposition is a diagnostic tool for explaining and debugging existing behavior, but it does not tell you where solutions come from; generating candidate solutions requires modeling the whole system in its environment and its interactions with adjacent systems, so debugging and planning deserve separate architectures.

Angle

Contrast diagnostic decomposition with generative context modeling, and show why agent systems that reuse their debugging decomposition for planning produce plan sets that never include the missing environmental interaction.

Source video

Size Your Interfaces to Seven: Chunking as an Engineering Budget for Tool Schemas and Context

If abstraction boundaries are sized to a consumer's working-memory limit and recursively reapplied, then interface surfaces should expose on the order of six to seven elements with internals hidden — a concrete, testable design budget rather than a vague call to manage complexity.

Angle

Convert Miller's Rule into numbers for tool schemas, sub-agent interface descriptions, and context packing, and confront the cost: what fails below a hidden boundary is invisible to review conducted at that level.

Source video

Project ideas

4

Perspective-Coverage Review Gate

beyond-evals

A rubric that scores explicit coverage of the nine Holistic Thinking Perspectives against a design artifact detects defects that a single-perspective review admits, and artifacts with low perspective coverage fail more often in downstream implementation than artifacts with high coverage.

Proof of concept

Build a scorer that labels each perspective as covered/absent with evidence spans for a corpus of design docs and agent specifications, run paired single-perspective vs. nine-perspective reviews, and track which review flagged defects that later materialized.

Measurement

Defect detection rate and false-negative rate per review mode, plus correlation between perspective-coverage score and downstream defects; report the number of defects caught only by the multi-perspective mode.

Source video

Global-Objective vs. Local-Metric Dial

beyond-evals

Pipelines whose component metrics are derived from a single system-level objective achieve higher end-to-end task success than otherwise identical pipelines whose components are tuned independently to local metrics.

Proof of concept

Take one decomposed agent pipeline, instantiate two configurations (independent per-agent tuning vs. objective-derived component scoring), and run both across a shared task suite with the same components and prompts.

Measurement

End-to-end task success delta between configurations, plus the gap between summed component scores and measured end-to-end success (local-global divergence).

Source video

Intent Formulation Gate

gatehouse

Inserting an explicit, scored intent-formulation stage with misalignment detection before solution work reduces rework and raises intent-alignment scores relative to a process-only workflow that proceeds directly to step execution.

Proof of concept

Prototype a front-end stage that restates the user's situation, records open questions, and blocks execution when an alignment threshold is not met; A/B it against a conformance-only workflow on identical tasks.

Measurement

Rework rate, number of clarification turns, and post-hoc human intent-alignment rating, comparing gate-enabled vs. process-only runs.

Source video

Interface Fan-Out Sweep

movement-lab

Tool schemas and context layouts that expose roughly 6-7 top-level elements per level with hidden internals yield higher model task success than wide flat surfaces exposing many more elements at one level.

Proof of concept

Generate the same capability as tool/interface schemas at varying top-level element counts (3, 7, 12, 20) with equivalent underlying functionality, and run the same task set against each layout.

Measurement

Task success rate and error-type distribution as a function of top-level element count per level; test whether the curve peaks in the 6-7 band.

Source video

Architectural implications

4

Optimization is being applied per component rather than per system.

Before

Each subsystem/agent is tuned against its own local metric; global behavior is an emergent side effect.

After

A single system-level objective is defined, and component metrics are derived from and scored against it.

Consequence

Prevents locally optimal but globally degraded configurations; requires instrumentation that ties component behavior to end-to-end outcomes.

Source video

Abstraction surfaces are sized organically rather than to a working-memory budget.

Before

Interfaces and context dumps expose many elements; internals leak across boundaries.

After

Interfaces expose roughly 6-7 elements, with internals hidden, and the same chunking rule reapplied recursively at each hierarchy level.

Consequence

Lower context load for both human reviewers and models; more stable, composable boundaries.

Source video

Process conformance is treated as a proxy for success.

Before

Systems are validated by completing prescribed lifecycle steps; front-end problem formulation is absent from the standards.

After

Situation understanding and problem formulation become explicit, measured stages preceding solution activity.

Consequence

Evaluation must score intent alignment, not just pipeline step completion, otherwise process-compliant failures pass certification.

Source video

Cognition and activity are bundled into one role.

Before

The same function both frames the problem and executes requirements/architecting/V&V activities.

After

Pure (cognitive framing), applied (activity), and domain layers are separated with distinct evaluation.

Consequence

Problem-framing quality can be improved and measured independently of procedural execution quality.

Source video

Tradeoffs and failure modes

4

Subsystem-level optimization in decomposed systems

Benefit

Local tuning is tractable and yields measurable per-component improvements.

Cost or risk

Independent subsystem optimization does not produce a system optimum and can degrade whole-system performance.

Optimizing subsystems independently does not lead to a system optimum; you must optimize the system as a whole.
Open source video
Source video

Process/standard conformance as a quality signal

Benefit

Standards provide repeatable, auditable procedures for managing the engineering process.

Cost or risk

Conformance can be achieved while the actual user need is never understood, and standards omit the front-end formulation stage entirely.

George follows standards meticulously but fails because he cannot understand what the customer really wants.
Open source video
Source video

Complexity reduction by hiding internal detail

Benefit

Partitioning a representation into a small number of systems and hiding internals keeps the working set within about 7 plus 2 elements.

Cost or risk

Hidden internals at the abstraction boundary mean failures and behaviors that live below the boundary are invisible to review conducted at that level.

Partitioning a complex diagram into six systems and hiding internal details to reduce complexity.
Open source video
Source video

Single-perspective analysis of a system

Benefit

One viewpoint (e.g., structural or functional) is cheap and yields fast, focused answers.

Cost or risk

Any single perspective yields only partial understanding, the blind-men-and-the-elephant failure, so confident conclusions can be systematically wrong.

There are nine holistic thinking perspectives analogous to the blind men and the elephant parable.
Open source video
Source video

Open questions

5

Can the nine Holistic Thinking Perspectives be operationalized into an automated, repeatable review rubric for a system or agent design?

Why unresolved

The summary lists and categorizes the perspectives but gives no procedure for scoring or detecting which perspectives a given artifact covers.

Research direction

Build a perspective-coverage rubric and test whether multi-perspective review catches defects that single-perspective review misses.

Source video

How can whole-system objectives be enforced when components are independently tunable?

Why unresolved

The summary asserts the optimization paradox but does not describe a mechanism for propagating system-level objectives down to subsystems.

Research direction

Compare architectures that derive component metrics from a global objective against architectures that tune components independently, measuring end-to-end task success.

Source video

Is there a reliable method for detecting under-specified or misunderstood user intent before solution work begins?

Why unresolved

The summary identifies George's failure as inability to understand what the customer wants but prescribes no elicitation or detection method.

Research direction

Prototype an explicit formulation stage with intent-alignment checks and measure reduction in rework relative to process-only workflows.

Source video

Do capability tiers (problem solver vs. problem formulator vs. innovator-leader) correspond to measurable, distinct levels in practice, or are they a descriptive spectrum?

Why unresolved

The summary presents five engineer types without evidence on how they are assessed or separated.

Research direction

Design assessment tasks that discriminate formulation ability from solving ability and check whether the tiers separate cleanly.

Source video

Does the ~7-element chunking constraint transfer to model context windows and tool/interface schemas?

Why unresolved

The summary cites Miller's Rule for human working memory and hierarchy partitioning but provides no evidence about non-human consumers.

Research direction

Benchmark interface and context layouts with varying element counts per level to test whether a small fixed fan-out improves task success.

Source video

Key claims

8
causalVerification needed

Optimizing subsystems independently does not lead to a system optimum; the whole system must be optimized.

Evidence

Optimizing subsystems independently does not lead to a system optimum; you must optimize the system as a whole.

Question

In decomposed agent or software pipelines, does per-component optimization measurably degrade end-to-end outcomes compared with whole-system objective optimization?

Source video
factualVerification needed

Human working memory holds about 7 plus or minus 2 objects at once.

Evidence

The human brain can only hold about 7 plus or minus 2 objects at once (Miller's Rule).

Question

Is Miller's 7 plus or minus 2 figure still supported as a characterization of working-memory capacity, and does an analogous limit apply to model context usage?

Source video
comparativeVerification needed

Standards such as DoD Directive 5000 and EIA-632 address management of systems engineering rather than doing systems engineering.

Evidence

Standards like DoD Directive 5000 and EIA-632 focus on managing the process (management of systems engineering) rather than doing systems engineering.

Question

Do these standards contain substantive guidance for problem formulation and concept exploration, or only process management?

Source video
factualVerification needed

The early stages of problem formulation and conceptualization are missing from systems engineering standards.

Evidence

Early stages of problem formulation and conceptualization are missing from standards.

Question

Which standards specifically omit front-end problem formulation, and what replaces that stage in practice?

Source video
causalVerification needed

A systems engineer who follows standards meticulously can still fail by failing to understand what the customer really wants.

Evidence

George follows standards meticulously but fails because he cannot understand what the customer really wants.

Question

Is this a single anecdote or a pattern with evidence across projects?

Source video
opinionVerification not requested

There are five types of systems engineers, ranging from problem causer to innovator-leader.

Evidence

There are five types of systems engineers: Type I (problem causer), Type II (apprentice doer), Type III (problem solver), Type IV (problem formulator), Type V (innovator-leader).

Source video
causalVerification needed

Conventional reductionist thinking does not explain where solutions come from.

Evidence

Conventional thinking does not explain where solutions come from.

Question

Can solution generation be attributed to whole-system/context modeling rather than to decomposition, and is that distinction empirically testable?

Source video
comparativeVerification not requested

Systems engineering decomposes into pure (cognitive skills and systems thinking) and applied (requirements, architecting, V&V) forms, analogous to pure and applied mathematics.

Evidence

Systems engineering is analogous to mathematics (pure and applied).

Source video

Connections

5