The mathematics of AI uncertainty

Overview

• Speaker: Professor Hannah Fry and Zubin Ghahramani
• Channel: Google DeepMind
• Main topic: The mathematics of AI uncertainty and probabilistic machine learning
• Purpose: To educate viewers on the mathematical foundations of uncertainty in AI, the limitations of overconfident neural networks, and how probabilistic inference and Bayesian thinking can improve machine intelligence. Professor Hannah Fry interviews Zubin Ghahramani, VP of Research at Google DeepMind and Professor at Cambridge, discussing the importance of modeling uncertainty in artificial intelligence. They explore how traditional AI systems and large language models often lack self-doubt and overconfidently hallucinate incorrect answers, contrasting this with Bayesian reasoning, probabilistic inference, semantic entropy, and human cognitive traits. They also discuss weather forecasting models like GenCast, the history of neural networks and parallel computing, and why integrating uncertainty estimation is critical for building reliable, human-centric AI systems.

Topic Map

Introduction to AI Uncertainty and Self-Doubt

• Explanation: Discusses how current AI systems, including large language models, give absolute answers with unwavering authority even when they are wrong, lacking the fundamental human trait of self-doubt.
• Key claims:
  • AI systems usually give absolute answers with unswerving authority.
  • Today's AI seems to be missing a fundamental human trait: self-doubt.
  • Improving machine uncertainty may be one of the missing pieces in AI.
• Examples:
  • The Eiffel Tower in Las Vegas being a half-scale replica on the Vegas Strip.
• Terminology:
  • Large language models
  • Self-doubt
  • Uncertainty
• Why it matters: Understanding uncertainty is essential to prevent AI systems from being overconfidently wrong or easily fooled.

Decision Making Under Uncertainty

• Explanation: Zubin Ghahramani explains that decision-making is central to intelligence, and because our perception of the real world is limited, intelligent systems must represent, update, and make decisions under uncertainty.
• Key claims:
  • Decision making is one of the most important parts of intelligence.
  • Perception is limited, meaning we are always uncertain about the state of the real world.
  • To build an intelligent system, you need a system that can represent and update uncertainty to make good decisions.
• Examples:
  • Self-driving cars navigating complex real-world scenarios like hailstorms or sudden obstacles.
• Terminology:
  • Decision making
  • Uncertainty
  • Probability
• Why it matters: Real-world environments are inherently noisy and unpredictable, requiring AI to reason probabilistically rather than deterministically.

History of AI and Neural Networks

• Explanation: Zubin reflects on entering the AI field in the late 1980s when expert systems were dominant, parallel computers like the Connection Machine were emerging, and neural networks were considered counterculture.
• Key claims:
  • In the mid-to-late 1980s, the dominant paradigm in AI was expert systems, which were brittle.
  • Neural networks modeled after the human brain were much more flexible and could learn from data.
  • The Connection Machine had 65,000 processors used for coding parallel neural network language models.
• Examples:
  • Zubin's summer job reading and explaining parallel distributed processing books to computation linguist Arvin Joshi.
• Terminology:
  • Expert systems
  • Neural networks
  • Parallel distributed processing
  • Connection Machine
• Why it matters: Tracing the evolution of AI highlights how paradigms shifted from rigid rule-based systems to data-driven neural models.

Probabilistic Inference and Bayesian Thinking

• Explanation: Explains how Bayesian statistics and probabilistic inference provide a principled mathematical framework for updating beliefs based on evidence, linking human cognition and machine learning.
• Key claims:
  • Information is the reduction of uncertainty by a factor of two (bits).
  • Humans are actually quite bad at explicitly estimating probabilities, relying instead on mental shortcuts called heuristics.
  • Bayes' rule combines prior beliefs with likelihoods based on new evidence to produce posterior beliefs.
• Examples:
  • Detectives evaluating suspects using prior beliefs and evidence like murder weapons found in a pantry.
• Terminology:
  • Probabilistic inference
  • Bayes' rule
  • Prior beliefs
  • Posterior beliefs
  • Likelihood
  • Heuristics
  • Bit
• Why it matters: Bayesian probability theory offers the rigorous mathematical foundation needed to model uncertainty and update knowledge dynamically.

Adversarial Examples and Overconfidence

• Explanation: Discusses adversarial examples where minor, imperceptible changes to an input image cause a neural network to misclassify objects with high confidence.
• Key claims:
  • Modifying just a few pixels in an image of a school bus can cause a neural network to confidently classify it as a cheetah.
  • We care about correctness and confidence; we do not want systems that can be overconfidently wrong or easily fooled.
• Examples:
  • A school bus image modified slightly to be misclassified as a cheetah with 99% confidence.
• Terminology:
  • Adversarial example
  • Image classification
  • Overconfidence
• Why it matters: Highlighting overconfidence reveals that raw predictive accuracy alone is insufficient without calibrated confidence measures.

Advanced Weather Forecasting with GenCast

• Explanation: Introduces GenCast, Google DeepMind's advanced AI-based weather forecasting model that predicts weather up to 15 days in advance using diffusion models and ensemble forecasting.
• Key claims:
  • GenCast can predict weather conditions up to 15 days in advance in about 8 minutes rather than hours on supercomputers.
  • Diffusion models generate an ensemble of forecasts to capture the inherent uncertainty in weather systems.
  • Weather forecasting inherently requires probabilistic modeling due to chaotic dynamics and limited sensors.
• Examples:
  • Tracking tropical storms like Hurricane Melissa using ensemble forecasting.
• Terminology:
  • GenCast
  • Diffusion models
  • Ensemble forecasting
  • Chaos theory
• Why it matters: Shows practical, high-impact applications where probabilistic AI models outperform traditional supercomputing methods in speed and accuracy.

Continuous Learning and Energy Efficiency

• Explanation: Compares human and animal learning with AI models, highlighting challenges like catastrophic forgetting and energy efficiency.
• Key claims:
  • Human brains consume about 20 watts of energy, whereas training large models in data centers requires orders of magnitude more power.
  • Catastrophic forgetting occurs when training an AI model on new data overwrites existing parameters.
  • Bayesian updating theoretically avoids catastrophic forgetting, but continuous learning remains an open research challenge.
• Examples:
  • Comparing the energy consumption of a human brain to a lightbulb and massive data centers.
• Terminology:
  • Continuous learning
  • Catastrophic forgetting
  • Energy efficiency
  • Bayesian updating
• Why it matters: Addressing energy efficiency and continual learning is crucial for building sustainable, adaptive AI agents.

Semantic Entropy and Detecting Hallucinations

• Explanation: Explores semantic entropy as an uncertainty measure for large language models that groups different phrasings into shared meaning clusters to detect hallucinations.
• Key claims:
  • Semantic entropy measures uncertainty for LLMs by grouping different phrasings into shared meaning clusters.
  • Calculating entropy over ideas rather than exact wording helps detect hallucinations.
  • Combining models with explicit uncertainty quantification bridges the gap between raw pattern matching and reliable reasoning.
• Examples:
  • Asking an LLM a question and analyzing the variability across multiple generated samples to measure semantic entropy.
• Terminology:
  • Semantic entropy
  • Hallucination detection
  • Meaning clusters
  • Next-token prediction
• Why it matters: Provides a practical method to make large language models aware of their own uncertainty and reduce false outputs.

Key Points

AI models lack inherent self-doubt

• Explanation: Standard machine learning systems and large language models predict next tokens without maintaining an explicit probabilistic representation of their own uncertainty, leading to overconfident hallucinations.
• Evidence: LLMs state incorrect facts with absolute authority until challenged, after which they easily flip-flop.
• Practical implication: Deploying AI in high-stakes domains like medicine or autonomous driving requires explicit uncertainty quantification to avoid catastrophic failures.

Bayes' rule as the foundation for rational updating

• Explanation: Bayesian inference provides the mathematical framework for combining prior beliefs with likelihoods from new evidence to compute posterior probabilities.
• Evidence: Mathematical probability theory governs how information reduces uncertainty by factors of two (bits).
• Practical implication: AI systems should emulate Bayesian updating to properly weigh evidence and adjust confidence over time.

GenCast demonstrates the power of probabilistic diffusion models

• Explanation: By combining diffusion models with ensemble forecasting, Google DeepMind's GenCast forecasts weather up to 15 days in minutes while representing forecast uncertainty.
• Evidence: GenCast generates an ensemble of trajectory tracks for tropical storms in roughly 8 minutes.
• Practical implication: Complex real-world simulations benefit immensely from generative probabilistic models that account for chaotic dynamics.

The tension between scaling data and architectural innovation

• Explanation: While scaling data and compute has driven massive progress in LLMs, true intelligence requires architectural improvements like proper uncertainty handling and continual learning.
• Evidence: Current LLMs rely on next-token prediction and lack explicit calibration of their own confidence states.
• Practical implication: Future AI research must focus on energy efficiency, continuous learning, and calibrated uncertainty alongside pure scale.

Frameworks, Models & Processes

Bayesian Inference / Bayes' Rule

• How it works: Combines prior probability distributions with likelihoods of observed evidence to compute posterior probability distributions.
• Components:
  • Prior beliefs
  • Likelihood of evidence
  • Normalization factor
  • Posterior beliefs
• When to use: When making decisions under uncertainty, updating hypotheses as new data arrives, or modeling perception and learning.

Semantic Entropy

• How it works: Measures uncertainty in language models by generating multiple responses, grouping semantically equivalent phrases into meaning clusters, and calculating entropy across those clusters.
• Components:
  • Multiple model generations
  • Semantic clustering
  • Entropy calculation over ideas
• When to use: To detect hallucinations and quantify confidence in large language model outputs.

Examples & Case Studies

A neural network misclassifies a slightly modified image of a school bus as a cheetah with 99% confidence.

• Illustrates: Overconfidence and adversarial vulnerability in standard deep learning models.
• Lesson: Correctness alone is insufficient; models must also be calibrated and aware of what they do not know.

Zubin Ghahramani asked compute scientist Arvin Joshi for a summer job in 1986 and was given two books on parallel distributed processing to read and explain.

• Illustrates: The early days of neural network research when it was considered counterculture.
• Lesson: Foundational reading and parallel computing principles paved the way for the modern deep learning revolution.

GenCast forecasts weather up to 15 days in advance using diffusion models to produce an ensemble of possible trajectories.

• Illustrates: Applying probabilistic generative models to chaotic real-world forecasting tasks.
• Lesson: Uncertainty representation in weather modeling enables faster and more robust emergency planning decisions.

Actionable Takeaways

• Immediate:
  • Recognize that standard AI models output next tokens without explicit confidence calibration.
  • Understand the difference between aleatoric uncertainty (inherent randomness) and epistemic uncertainty (lack of knowledge).
• Strategic:
  • Incorporate Bayesian thinking and probabilistic inference into AI architecture design.
  • Prioritize research into energy-efficient learning, continual adaptation, and hallucination detection via semantic entropy.
• Questions to investigate:
  • How can large-scale transformer models be efficiently modified to output calibrated probabilistic uncertainty?
  • What new hardware architectures are needed to support probabilistic and sparse neural networks at scale?

Claims Worth Verifying

• The human brain consumes approximately 20 watts of power. (scientific fact)
• GenCast can predict weather conditions up to 15 days in advance in about 8 minutes. (technical benchmark)
• Daniel Kahneman and Amos Tversky showed humans rely on heuristics that cause systematic errors in probability estimation. (psychological research)

Notable Quotes

"If you ask an AI a question, it will usually give you an absolute answer with unwave-ring authority, even if that answer turns out to be wrong." (at 0:04) "If you think about intelligence, one of the most important parts of intelligence is decision-making." (at 2:00) "Information, a bit of information that we use in computer science, is the reduction of uncertainty by a factor of two." (at 7:30) "Daniel Kahneman and Amos Tversky showed humans don't form probabilistic beliefs using formal math. Instead, people rely on mental shortcuts called heuristics, which simplify complex judgments but cause systematic errors." (at 10:50) "Google DeepMind's advanced AI-based weather forecasting model that can predict weather conditions up to 15 days in advance." (at 44:16) "Generative AI algorithms that create realistic data—such as images, audio, or video—by learning to reverse a gradual noising process." (at 45:19) "An uncertainty measure for LLMs that groups different phrasings into shared meaning clusters, calculating entropy over ideas rather than exact wording to detect hallucinations." (at 38:39) "When an AI model trained on new data overwrites existing parameters, degrading or erasing its performance on previously learned tasks." (at 62:23) "Google DeepMind's AI system that predicts a protein's 3D structure from its amino acid sequence." (at 50:22)

Compressed Summary

• AI models currently lack explicit self-doubt and often hallucinate with high overconfidence.
• Bayesian inference and probabilistic reasoning provide the mathematical framework to manage uncertainty.
• GenCast uses diffusion models for fast, probabilistic ensemble weather forecasting.
• Semantic entropy measures meaning variance across LLM outputs to detect hallucinations.
• Future AI systems must balance computational scale with energy efficiency and calibrated uncertainty.
• Keywords: uncertainty, bayes, probability, entropy, neural-networks
• Core insight: Integrating explicit probabilistic reasoning and uncertainty estimation into AI architectures is essential for moving from brittle overconfident models to reliable, human-centric collaborators.