IBM Technology · Published 2026-08-20

AI Agents vs Business Rules: Which Should Make Decisions?

privateReview: unreviewedConfidence: high
business rulesAI agentsdeterministicprobabilistichybrid architecturebusiness-ruleshybrid-architectureGuardrailsHuman-in-the-loopagent-testingdecision-routing
Open on YouTube ↗

Summary

Overview

  • Speaker: Martin Keen
  • Channel: IBM Technology
  • Main topic: Comparing Business Rules Engines and AI Agents for automated decision making
  • Purpose: To educate developers and architects on the differences, trade-offs, and complementary nature of business rules versus AI agents for automated decision making. Martin Keen explains the differences between business rules engines (deterministic) and AI agents (probabilistic), detailing how each works, when to use them, and how a hybrid approach combining both can be utilized in complex enterprise systems.

Topic Map

Business Rules Engines

  • Explanation: Business rules engines run logic explicitly written down by humans, evaluating conditions and actions deterministically.
  • Key claims:
    • Business rules engines run explicit logic written by humans.
    • They are deterministic, meaning output is a fixed function of the input.
    • They provide consistency, predictability, and auditability.
  • Examples:
    • If order is less than 30 days old and item is not final sale, then approve refund.
  • Terminology:
    • business rules engine
    • deterministic
    • boolean logic
  • Why it matters: They are ideal for well-defined, structured, and predictable scenarios where exact compliance and auditability are required.

AI Agents

  • Explanation: AI agents are built using large language models, taking a goal, context, and tools to reason probabilistically over patterns.
  • Key claims:
    • AI agents use large language models that predict next tokens.
    • They are probabilistic, evaluating a probability distribution over possible responses.
    • They handle unstructured data, context, and judgment.
  • Examples:
    • Reviewing a customer return request with unstructured notes and photos of a damaged product.
  • Terminology:
    • AI agents
    • large language model
    • probabilistic
    • guardrails
    • human-in-the-loop
  • Why it matters: They enable automated decision-making in messy, unpredictable environments requiring judgment and tool calling.

Hybrid Architecture

  • Explanation: Real systems often combine rules and AI agents in a hybrid approach to leverage the strengths of both paradigms.
  • Key claims:
    • Hybrid approaches run rules first for cheap and quick checks.
    • Requests can escalate to AI agents when rules cannot make a definitive decision.
    • AI agent recommendations can be filtered through deterministic guardrails or human-in-the-loop review.
  • Examples:
    • Running a rules engine first, escalating to an AI agent for unstructured data, and evaluating output through guardrails.
  • Terminology:
    • hybrid approach
    • guardrails
    • human in the loop
  • Why it matters: It allows organizations to handle structured compliance efficiently while utilizing AI flexibility for complex, edge-case judgments.

Key Points

Deterministic vs Probabilistic

  • Explanation: Business rules are deterministic with fixed outputs for inputs, while AI agents are probabilistic with distributions over multiple outcomes.
  • Evidence: Rules engine evaluates exact boolean conditions; LLMs predict next tokens based on training patterns.
  • Practical implication: Rules provide predictable results for testing, whereas AI agents might return different outputs on repeated identical requests.

When to Use Business Rules

  • Explanation: Use rules when decisions are well-defined and inputs/outputs are structured and known in advance.
  • Evidence: Refund policies, loan eligibility checks, insurance pricing tiers.
  • Practical implication: Lower compute cost, complete auditability, and easy unit testing.

When to Use AI Agents

  • Explanation: Use agents when context is unstructured, messy, or unanticipated.
  • Evidence: Evaluating dumbbell return requests or reviewing images of defective products.
  • Practical implication: Enables automation where rigid rules would fail or require endless manual coding.

Frameworks, Models & Processes

Hybrid Decision Workflow

  • How it works: Requests enter a rules engine for fast deterministic filtering, escalate to AI agents for unstructured evaluation or tool calling, and pass through guardrails or human review before final decision.
  • Components:
    • Request
    • Rules Engine
    • AI Agent
    • Guardrails
    • Human In The Loop (HITL)
    • Final Decision
  • When to use: When handling high-volume requests that contain both structured compliance rules and unstructured judgment calls.

Examples & Case Studies

A customer requests a refund for a 12-day-old non-final-sale item.

  • Illustrates: Deterministic evaluation by a business rules engine.
  • Lesson: Rules engines quickly and accurately evaluate boolean conditions when facts are clean.

A customer requests a refund for heavy dumbbells with unstructured supporting text and photos.

  • Illustrates: Probabilistic judgment and tool usage by an AI agent.
  • Lesson: AI agents excel at parsing unstructured data and leveraging tools to determine outcomes requiring judgment.

Actionable Takeaways

  • Immediate:
    • Understand the fundamental difference between deterministic rules and probabilistic AI agents.
    • Identify whether your decision logic relies on structured data or requires subjective judgment.
  • Strategic:
    • Adopt a hybrid architecture that routes standard cases to rules and escalates complex edge cases to AI agents.
    • Implement guardrails and human-in-the-loop checks to govern AI agent outputs safely.
  • Questions to investigate:
    • Which processes in our organization currently require human judgment that could be augmented by AI agents?
    • How can we establish effective deterministic guardrails around probabilistic AI models?

Claims Worth Verifying

  • Business rules engines are entirely deterministic. (technical definition)
  • AI agents operate over probability distributions of possible responses. (machine learning principle)

Notable Quotes

"Have AI agents superseded business rules or do they both have their place?" (at 0:15) "The output is a fixed function of the input and business rules are deterministic." (at 4:18) "The two together can help make better decisions." (at 16:50)

Compressed Summary

  • Business rules engines execute explicit, deterministic logic for structured inputs.
  • AI agents use LLMs to reason probabilistically over goals, context, and tools for unstructured data.
  • Hybrid architectures combine rules for efficiency and AI agents for judgment, backed by guardrails and human oversight.
  • Keywords: business rules, ai agents, deterministic, probabilistic, hybrid architecture
  • Core insight: Business rules and AI agents are not mutually exclusive; combining deterministic rules with probabilistic AI agents creates robust hybrid decision systems.

Core insights

5
Architecturemedium noveltymoderate evidence

The decision system is modeled as a tiered cascade: a deterministic rules engine runs first as a cheap triage filter, and only cases the rules cannot definitively resolve escalate to a probabilistic agent, with the agent's output contained by deterministic guardrails before reaching a final decision. The architectural unit is not 'rules or agents' but 'rules, then agent, then guardrail'.

Why it matters

Most teams frame this as a binary technology choice. Framing it as a cascade moves the design question to the escalation boundary and lets high-volume structured traffic stay on the cheap deterministic path while only expensive probabilistic compute is spent on genuinely ambiguous cases.

Generalization

Applies to any decision pipeline mixing cheap deterministic logic with expensive learned inference: routing/triage tiers, model cascades, and tiered classification systems in general.

Hybrid approaches run rules first for cheap and quick checks.
Open source video
Requests can escalate to AI agents when rules cannot make a definitive decision.
Open source video
AI agent recommendations can be filtered through deterministic guardrails or human-in-the-loop review.
Open source video
Mental Modelmedium noveltymoderate evidence

Determinism is treated as a verification contract, not just a behavioral property: because a rules engine's output is a fixed function of its input, it supports predictable results for testing and easy unit testing, whereas an agent may return different outputs on repeated identical requests. Verification cost therefore differs structurally between the two tiers, not merely in degree.

Why it matters

It implies the test harness must be split by tier: exact-match assertions for the deterministic tier, and distributional/statistical or property-based evaluation for the probabilistic tier. Treating an agent like a function with a single expected output produces flaky CI rather than useful signal.

Generalization

Any system with a non-deterministic component needs a separate evaluation regime (repeat sampling, tolerance bands, rubric scoring) from its deterministic components.

Rules provide predictable results for testing, whereas AI agents might return different outputs on repeated identical requests.
Open source video
Lower compute cost, complete auditability, and easy unit testing.
Open source video
Mental Modelmedium noveltymoderate evidence

The routing criterion between the two tiers is explicitly a data-shape and definability test, not a task-importance or stakes test: structured, known-in-advance inputs and outputs go to rules; unstructured, messy, or unanticipated context goes to the agent. The stated reason to escalate is that rigid rules would 'fail or require endless manual coding'.

Why it matters

It gives a concrete, checkable routing predicate (is the input structured and the output space enumerable?) instead of a vague 'use AI where it adds value'. It also predicts where maintenance cost will land: every unanticipated edge case becomes a new rule.

Generalization

In hybrid systems generally, the right boundary between deterministic and learned components is the shape/definability of the input-output mapping, not the perceived importance of the decision.

Use agents when context is unstructured, messy, or unanticipated.
Open source video
Enables automation where rigid rules would fail or require endless manual coding.
Open source video
Mechanismmedium noveltymoderate evidence

Guardrails and human-in-the-loop review are positioned as a post-generation containment layer that sits between an agent's recommendation and the final decision — i.e., deterministic logic applied downstream of the probabilistic component, rather than validation baked into the agent itself.

Why it matters

It places the safety boundary outside the model and outside the agent's own reasoning, making it separately testable, auditable, and replaceable without retraining or re-prompting. It also creates a clean output contract between the probabilistic and deterministic domains.

Generalization

Constraining generative components by wrapping them in external deterministic validators/approval gates is more maintainable than relying on the model to police itself.

AI agent recommendations can be filtered through deterministic guardrails or human-in-the-loop review.
Open source video
How can we establish effective deterministic guardrails around probabilistic AI models?
Open source video
Failure Modemedium noveltyweak evidence

The failure mode of the deterministic tier is presented as unbounded rule growth: as cases become unanticipated, rigid rules require 'endless manual coding' rather than degrading gracefully. The failure mode of the agent tier is the mirror image — non-repeatable output — so each tier's weakness is precisely the other tier's strength.

Why it matters

It identifies where maintenance cost and reliability cost accumulate in each design, and implies the tier boundary should be drawn where the marginal cost of new rules exceeds the marginal cost of probabilistic judgment plus containment.

Generalization

Deterministic rule systems fail by combinatorial expansion; learned systems fail by variance. Hybrid designs exist to route each failure mode to the tier that can absorb it.

Enables automation where rigid rules would fail or require endless manual coding.
Open source video
AI agents might return different outputs on repeated identical requests.
Open source video

Deep dives

4

Escalation-boundary detection in rules-first cascades

Research question

How should a deterministic rules tier represent and return an explicit 'undetermined' outcome so that escalation to a probabilistic agent is triggered by design rather than by accident, and what is the false-escalation vs missed-escalation error profile of that design?

Why

The cascade's entire cost and quality profile is set by the escalation edge, yet the source only asserts that requests escalate 'when rules cannot make a definitive decision' without describing how a boolean engine expresses insufficiency. Without an explicit abstain state, escalation is either hardcoded per case or absent, and the tier boundary becomes untunable. A concrete abstain contract also makes escalation rate a first-class production metric for rule-set drift.

Hybrid approaches run rules first for cheap and quick checks.
Open source video
Requests can escalate to AI agents when rules cannot make a definitive decision.
Open source video
Enables automation where rigid rules would fail or require endless manual coding.
Open source video
Source video

Evaluation harness design for non-repeatable decision tiers

Research question

What evaluation methodology (repeat-sampling with tolerance bands, rubric scoring, property-based assertions) yields stable signal for a tier whose identical inputs may produce different outputs, and how does it compare to exact-match assertions on the same decision set?

Why

The source states non-repeatability as a practical implication but supplies no evaluation method. If teams carry exact-match assertions across the tier boundary they get flaky CI rather than signal, and the flakiness masks real regressions. A split test harness is the precondition for trusting any cascade that includes a probabilistic stage.

Rules provide predictable results for testing, whereas AI agents might return different outputs on repeated identical requests.
Open source video
Lower compute cost, complete auditability, and easy unit testing.
Open source video
AI agents might return different outputs on repeated identical requests.
Open source video
Source video

Guardrail construction as an external containment layer

Research question

Which guardrail strategies (schema/type validation, hard policy constraints, thresholded confidence, tool-call allowlists) provide the best coverage-to-false-rejection ratio when placed downstream of an agent's recommendation rather than inside its reasoning?

Why

The source poses effective deterministic guardrails around probabilistic models as an open question rather than a technique. Treating guardrails as an external, separately testable output contract — rather than validation baked into the prompt — determines whether the safety boundary can be audited, replaced, and regression-tested without touching the model. It also fixes where latency and human throughput costs land on the critical path.

AI agent recommendations can be filtered through deterministic guardrails or human-in-the-loop review.
Open source video
How can we establish effective deterministic guardrails around probabilistic AI models?
Open source video
Source video

The rule-count threshold where the deterministic tier stops being the cheap tier

Research question

At what rule-count, rule-churn rate, or case-diversity level does the maintenance cost of the deterministic tier exceed the inference-plus-containment cost of routing the same long tail to an agent?

Why

Both tiers are claimed to be cheaper, but the claims are conditional on case shape. Rules fail by combinatorial expansion; agents fail by variance. Without a measured crossover point, the tier boundary is drawn by intuition and drifts as edge cases accumulate, which is exactly the failure mode the hybrid design is meant to prevent.

Enables automation where rigid rules would fail or require endless manual coding.
Open source video
Lower compute cost, complete auditability, and easy unit testing.
Open source video
Use agents when context is unstructured, messy, or unanticipated.
Open source video
Source video

Article ideas

3

Your Test Suite Is Lying to You at the Tier Boundary

Teams that adopt a hybrid rules-plus-agent pipeline keep writing exact-match assertions for both tiers, which converts the agent tier's non-determinism into CI flake instead of signal; the harness must split into exact-match tests for the deterministic tier, distributional or rubric evaluation for the probabilistic tier, and dedicated escalation-path tests for the boundary, with escalation rate treated as a production metric for rule-set drift.

Angle

Arguing that verification cost differs structurally between tiers, not in degree, and that the flaky-CI symptom is a design smell pointing at a missing tier split.

Source video

Stop Asking Whether You Need AI Agents or Business Rules

The binary framing of 'agents vs rules' is the wrong decision to make; the correct artifact to design, instrument, and tune is the escalation boundary itself, and the right placement criterion is the shape and definability of the input-output mapping (structured and enumerable goes deterministic; unstructured and unanticipated goes probabilistic) rather than the stakes or importance of the decision.

Angle

Reframing the architecture question from technology selection to boundary design, and giving a checkable routing predicate instead of 'use AI where it adds value'.

Source video

Guardrails Belong Outside the Model

Positioning guardrails and human-in-the-loop review as post-generation containment — deterministic logic applied downstream of the agent's recommendation and upstream of the final decision — makes the safety boundary separately testable, auditable, and replaceable without retraining or re-prompting, whereas asking the model to police its own output collapses the control surface into the component it is supposed to constrain.

Angle

Argues that the safety boundary should be a clean output contract between the probabilistic and deterministic domains, and inventories the cost that containment adds to the critical path.

Source video

Project ideas

4

Abstain-and-Escalate Rules Kernel

gatehouse

Adding an explicit 'undetermined' return state to a boolean rules tier, plus a tunable escalation predicate, reduces missed-escalation error on ambiguous cases without materially inflating false escalations or total compute cost.

Proof of concept

Build a small refund-triage pipeline: a rules engine with three-valued return states (approve / deny / undetermined), a probabilistic agent stage that only receives undetermined cases, and a guardrail filter on agent output. Run it against a labeled case set spanning clean structured cases, messy unstructured cases, and adversarial near-misses.

Measurement

Escalation rate, false-escalation cost, missed-escalation error rate, and cost and latency per decision versus rules-only and agent-only routing on the same case set.

Source video

Tier-Split Eval Harness

beyond-evals

A test harness that uses repeated sampling with tolerance bands and rubric scoring for the probabilistic tier, plus exact-match assertions for the deterministic tier and explicit escalation-path tests, produces a lower flake rate and better regression-detection signal than exact-match assertions applied uniformly.

Proof of concept

Take one decision set and score it under three regimes: uniform exact-match assertions, tier-split exact-match plus repeat-sample tolerance bands, and tier-split plus rubric scoring. Track flakes, false regressions, and missed regressions across repeated runs and prompt/rule perturbations.

Measurement

CI flake rate, true-regression detection rate, and signal-to-noise ratio per regime; correlation between escalation rate and seeded rule-set drift.

Source video

Guardrail Strategy Bake-Off

gatehouse

Guardrail strategies differ sharply in coverage-to-false-rejection ratio, and schema/type validation combined with hard policy constraints outperforms thresholded confidence filtering on coverage while producing fewer false rejections.

Proof of concept

Wrap the same agent in four interchangeable guardrail layers (schema/type validation, hard policy constraints, thresholded confidence, tool-call allowlist) and replay a labeled set of agent recommendations, including deliberately malformed and out-of-policy outputs, through each layer independently and in combination.

Measurement

Coverage of unsafe or out-of-policy outputs, false-rejection rate on legitimate recommendations, added latency per decision, and human review volume when HITL is the final gate.

Source video

Rule-Growth Crossover Benchmark

movement-lab

There is a measurable rule-count and case-diversity threshold above which maintaining the deterministic tier costs more engineer-hours per case class than routing that long tail to an agent plus its containment layer.

Proof of concept

Simulate a decision domain with a growing long tail: incrementally add case classes and rules to a rules-only tier while routing an identical stream to an agent-plus-guardrail tier, logging authoring and maintenance effort, error rates, and per-decision compute for both.

Measurement

Engineer-hours per new case class, marginal cost per decision, accuracy, and auditability coverage as case diversity increases; identify the crossover point and its sensitivity to rule churn.

Source video

Architectural implications

3

Decision ownership moves from a single decision component to a multi-stage pipeline with an explicit escalation edge.

Before

One component owns the decision: either a rules engine or an agent.

After

Request -> Rules Engine -> AI Agent (on escalation) -> Guardrails -> Human In The Loop -> Final Decision.

Consequence

Compute spend becomes proportional to case ambiguity rather than to request volume, but the escalation criterion itself must be designed, instrumented, and tuned as a first-class artifact.

Source video

A deterministic validation boundary is introduced between the probabilistic component and the final decision.

Before

Agent output is consumed directly as the decision.

After

Agent output passes through deterministic guardrails and/or a human review stage before it can become the decision.

Consequence

Adds a stage to the critical path (latency, and human throughput where HITL fires), in exchange for an independently testable and auditable control surface outside the model.

Source video

Testing and observability requirements split by tier rather than remaining uniform.

Before

A single test suite asserting exact expected outputs for each decision case.

After

Exact-match unit tests for the rules tier; repeated-sample or rubric-based evaluation for the agent tier; escalation-path tests for the boundary.

Consequence

CI must tolerate and measure non-determinism, and the escalation rate becomes a first-class production metric for detecting rule-set drift.

Source video

Tradeoffs and failure modes

3

Deterministic rules tier

Benefit

Consistency, predictability, complete auditability, lower compute cost, and easy unit testing for structured compliance decisions.

Cost or risk

Brittleness in the long tail: unanticipated cases cannot be absorbed without writing more rules, leading to endless manual coding as context becomes messier.

Enables automation where rigid rules would fail or require endless manual coding.
Open source video
Source video

Probabilistic agent tier

Benefit

Handles unstructured data, context, judgment, and tool calling where rigid rules would fail.

Cost or risk

Repeated identical requests may yield different outputs, undermining reproducibility, straightforward testing, and deterministic audit trails.

AI agents might return different outputs on repeated identical requests.
Open source video
Source video

Guardrails and human-in-the-loop containment

Benefit

Provides a layer that can filter agent recommendations and prevent unreviewed probabilistic output from becoming the final decision.

Cost or risk

Adds stages between agent output and final decision, and the mechanism for building effective deterministic guardrails around probabilistic models is left unresolved by the source.

AI agent recommendations can be filtered through deterministic guardrails or human-in-the-loop review.
Open source video
Source video

Open questions

4

How is 'rules cannot make a definitive decision' actually detected and encoded at runtime?

Why unresolved

The summary asserts that requests escalate when rules cannot decide, but describes no mechanism for representing unknown/insufficient-information outcomes in a boolean rules engine.

Research direction

Prototype explicit abstain/undetermined return states in the rules tier and measure escalation rate, false-escalation cost, and missed-escalation error rate.

Source video

How can deterministic guardrails be constructed around probabilistic model outputs such that they are both effective and non-brittle?

Why unresolved

The source poses this as a question to investigate rather than describing a technique, schema, or validation pattern.

Research direction

Compare guardrail strategies (schema/type validation, hard policy constraints, thresholded confidence, tool-call allowlists) on coverage and false-rejection rate against a labeled case set.

Source video

What does an evaluation harness look like for a tier whose identical inputs may produce different outputs?

Why unresolved

The non-repeatability of agent outputs is stated as a practical implication, but no evaluation methodology is provided.

Research direction

Benchmark repeat-sampling with tolerance bands and rubric scoring versus exact-match assertions on the same decision set, tracking flake rate and signal quality.

Source video

Does combining rules with agents actually produce better decisions, or only cheaper ones?

Why unresolved

The claim that the two together help make better decisions is asserted in the source without a benchmark, control condition, or measured outcome.

Research direction

Run an A/B comparison of rules-only, agent-only, and hybrid pipelines on the same decision set, measuring accuracy, cost per decision, latency, and auditability.

Source video

Key claims

7
factualVerification needed

Business rules engines are deterministic: the output is a fixed function of the input.

Evidence

The output is a fixed function of the input and business rules are deterministic.

Question

Under what conditions does a real rules engine violate strict determinism — external data lookups, mutable fact state, time-of-evaluation, or conflicting/ordering-dependent rules?

Source video
factualVerification needed

AI agents are probabilistic, evaluating a probability distribution over possible responses.

Evidence

They are probabilistic, evaluating a probability distribution over possible responses.

Question

How much output variance actually occurs for repeated identical requests at typical temperature and tool-calling configurations?

Source video
causalVerification needed

Hybrid approaches run rules first for cheap and quick checks before escalating.

Evidence

Hybrid approaches run rules first for cheap and quick checks.

Question

What is the measured cost and latency delta per request between rules-first cascades and agent-first or agent-only routing on the same workload?

Source video
comparativeVerification needed

Rules-based decisions deliver lower compute cost, complete auditability, and easy unit testing relative to agent decisions.

Evidence

Lower compute cost, complete auditability, and easy unit testing.

Question

At what rule-count or rule-churn threshold does the maintenance cost of the deterministic tier overtake the inference and containment cost of the agent tier?

Source video
opinionVerification needed

Combining business rules and AI agents together helps make better decisions.

Evidence

The two together can help make better decisions.

Question

Is there any measured improvement in decision quality from the hybrid configuration, or is the gain limited to cost and coverage?

Source video
factualVerification not requested

AI agents are built on LLMs that predict the next token.

Evidence

AI agents use large language models that predict next tokens.

Source video
causalVerification needed

AI agents enable automation where rigid rules would fail or require endless manual coding.

Evidence

Enables automation where rigid rules would fail or require endless manual coding.

Question

How does maintenance effort scale with case diversity for a rules set versus an agent that handles the same long tail, measured in engineer-hours per case class?

Source video

Connections

4

Rules engineGuardrails

Both are deterministic control surfaces, but they operate at different points in the pipeline — the rules engine gates inputs before inference, guardrails gate outputs after inference.

Source video