Jane Street · Published 2026-09-23

Teaching in the Age of AI: Learning Goals and the Goals of Learning

Open on YouTube ↗

Summary

Overview

  • Speaker: Aaron Bauer
  • Channel: Jane Street
  • Main topic: Software developer education and developer tooling in the age of AI
  • Purpose: To discuss the philosophy and practice of developer education at Jane Street, how teaching methods have evolved, and the impact of AI tools on learning goals and software engineering workflows. A conversation exploring how software developer education, specifically at Jane Street through programs like OCaml bootcamps and internships, intersects with the rise of AI tools and LLMs. The discussion covers the evolution of teaching methodologies, the balance between conceptual understanding and practical tools, the role of code reviews and editors as home bases, and the challenges AI introduces to learning goals and cognitive load.

Topic Map

Introduction and Background

  • Explanation: Aaron Bauer introduces his background, starting from childhood computer programming, attending the Summer Science Program in New Mexico, working as a TA on binary search trees, and pursuing graduate studies in human-computer problem-solving (studying Folding @ home) before teaching at Carlton College and joining Jane Street.
  • Key claims:
    • Computing was initially seen as a means to an end, beating slide rules.
    • Experiencing teaching as a TA on binary search trees sparked a passion for education.
  • Examples:
    • Writing Python code to compute asteroid orbits from telescope observations.
  • Terminology:
    • binary search trees
    • Folding @ home
    • human-computer problem-solving
  • Why it matters: Establishes the speaker's dual background in software engineering and education research.

The Flipped Classroom and Online Teaching

  • Explanation: Discussion on the transition to online teaching during the pandemic, the adoption of flipped classrooms, and the challenges of student engagement over video versus in-person interaction.
  • Key claims:
    • Text-based online AP computer science courses worked for some but failed many students lacking good internet or study habits.
    • Flipped classrooms use video lectures before class and interactive exercises during class.
    • Active learning and in-person engagement are critical for attention and retention.
  • Examples:
    • Teaching at Carlton College during spring 2020 through online quarters.
  • Terminology:
    • flipped classroom
    • active learning
    • synchronous classes
  • Why it matters: Highlights the limitations of passive video learning and the value of interactive, live instruction.

Note-Taking and Cognitive Processing

  • Explanation: Exploring the role of note-taking and writing code on paper as cognitive tools that aid retention and memory consolidation.
  • Key claims:
    • The act of writing notes down helps set information in memory in a qualitatively different way than just hearing it.
    • Writing code on paper forces programmers to think about what the machine is actually doing rather than arbitrarily perturbing code until it works.
  • Examples:
    • Writing scheme programs on paper with obfuscated names to force puzzle-solving.
  • Terminology:
    • cognitive psychology
    • retention
    • scheme programs
  • Why it matters: Demonstrates how removing digital tools can sometimes improve deep understanding and mental model building.

Jane Street's Education Philosophy

  • Explanation: Jane Street's approach to developer education, balancing the need to teach unique internal tech stacks, OCaml, Python, and trading concepts with practitioner-led teaching.
  • Key claims:
    • Teaching is an essential part of Jane Street's culture because internal tech stacks differ significantly from the outside world.
    • Educators at Jane Street are also practitioners who work in the ecosystem.
    • A 50/50 split between programming and educational work helps maintain practical relevance.
  • Examples:
    • Developing OCaml bootcamps and production bootcamps for new interns and software engineers.
  • Terminology:
    • developer educators
    • OCaml
    • tech stack
  • Why it matters: Explains why industry-embedded education is necessary for specialized engineering environments.

Editors as the Developer's Home Base

  • Explanation: The role of text editors (VS Code, Emacs, Neovim) in Jane Street's engineering workflow, code review culture, and how editor choices shape developer habits.
  • Key claims:
    • Jane Street treats the text editor as the software engineer's home base where all interactions, version control, and code reviews happen.
    • Code review (CR) in Jane Street happens directly within the editor using specialized formatting and inline comments.
    • Having a centralized editor workflow avoids jumping across multiple web apps and tools.
  • Examples:
    • Using Emacs, VS Code, and Neovim alongside custom tools like Iron.
  • Terminology:
    • VS Code
    • Emacs
    • Neovim
    • Iron
    • code review (CR)
  • Why it matters: Emphasizes the integration of development, review, and feedback loops within a unified environment.

AI in Software Engineering and Education

  • Explanation: Examining how the emergence of LLMs and agentic AI tools affects developer workflows, code generation, code review, and the foundational education of engineers.
  • Key claims:
    • LLMs create a new tension in education: if an LLM can write code instantly, what is the value of teaching people how to write that code?
    • Code review with LLMs becomes harder because reviewing generated code requires building a mental model without the incremental learning of writing it.
    • Over-reliance on LLMs without understanding can lead to subtle bugs and unmaintainable systems.
  • Examples:
    • Using Claude Code, GitHub Copilot, and custom LLM integrations for code generation and review.
  • Terminology:
    • LLMs
    • agentic AI
    • code review
    • cognitive load
  • Why it matters: Addresses the existential questions AI poses for software engineering proficiency and training programs.

Key Points

The Value of Struggling with Material

  • Explanation: Struggling through problems and writing code manually builds deep mental models that passive consumption or AI-generated solutions bypass.
  • Evidence: Cognitive psychology research on active learning and note-taking.
  • Practical implication: Educational programs must resist the temptation to make everything too easy or automated.

Editors as Integrated Workspaces

  • Explanation: Keeping development, version control, and code reviews inside a unified editor environment reduces context switching and improves developer flow.
  • Evidence: Jane Street's extensive use of Emacs and internal tools like Iron integrated directly into editors.
  • Practical implication: Tooling should unify workflows rather than fragmenting them across browsers and external apps.

The Challenge of AI-Generated Code in Education

  • Explanation: When AI writes code, learners miss the incremental feedback loop of catching syntax errors and debugging, which harms long-term retention.
  • Evidence: Observations from intern bootcamps where reliance on AI tools masks knowledge gaps.
  • Practical implication: Curricula must adapt to teach verification, debugging, and system design alongside AI utilization.

Frameworks, Models & Processes

Mastery Learning and Scaffolded Bootcamps

  • How it works: Students progress through structured exercises, getting feedback from mentors and automated systems, building up from basic syntax to full client-server apps.
  • Components:
    • OCaml bootcamp
    • Production bootcamp
    • Code review mentorship
    • Interactive exercises
  • When to use: Onboarding new software engineers and interns into specialized technology stacks.

Flipped Classroom Model

  • How it works: Lecture content is consumed prior to class via videos or readings, freeing up synchronous class time for interactive problem-solving and discussions.
  • Components:
    • Pre-class reading/video
    • In-class problem solving
    • Peer discussion
    • Instructor tutoring
  • When to use: Teaching complex technical subjects where active application is superior to passive listening.

Examples & Case Studies

Using summer science programs in high school to write Python code calculating asteroid orbits.

  • Illustrates: Early hooks into programming through real-world scientific applications.
  • Lesson: Combining domain science with programming creates highly engaging educational experiences.

Integrating code review comments directly inside the text editor rather than on a separate web interface.

  • Illustrates: Unified developer tooling minimizing context switching.
  • Lesson: Keeping developers in their primary environment fosters better collaboration and feedback loops.

Actionable Takeaways

  • Immediate:
    • Balance the use of AI coding assistants with foundational manual coding practice.
    • Design developer onboarding programs with structured mentorship and progressive bootcamps.
  • Strategic:
    • Recognize that AI changes the nature of code review from authoring verification to reviewing unfamiliar generated output.
    • Integrate development environments tightly to reduce context switching.
  • Questions to investigate:
    • How do we teach effective code review when LLMs generate large volumes of code?
    • What is the optimal balance between automated AI tutoring and human mentorship in technical bootcamps?

Claims Worth Verifying

  • Writing notes down by hand significantly improves long-term memory retention compared to typing or passive listening. (educational psychology)
  • Code review efficiency drops when reviewers must evaluate large, AI-generated code blocks without prior context. (software engineering productivity)

Notable Quotes

"The code that you don't have to write is the best code of all." (at 75:18) "Code review is hard. Writing code, you're sort of incrementally building your understanding." (at 92:22)

Compressed Summary

  • Software developer education must balance foundational manual practice with modern AI tools.
  • Jane Street utilizes intensive OCaml and production bootcamps combined with practitioner-led teaching.
  • Text editors serve as the central home base for development, version control, and inline code reviews.
  • AI code generation introduces new challenges for code review, comprehension, and cognitive load.
  • Keywords: education, ocaml, ai, code-review, tooling
  • Core insight: While AI tools accelerate code generation, developer education must still ensure engineers build deep conceptual models and strong code review skills through active struggle and practice.

Core insights

6
Mechanismhigh noveltymoderate evidence

Reviewing LLM-generated code is qualitatively harder than reviewing human-written code because the reviewer must reconstruct a mental model of the code without having experienced the incremental authoring process that normally builds that model.

Why it matters

It reframes AI-assisted development as a verification-burden shift: the author's cognitive savings are transferred to the reviewer, which has direct implications for PR/CR throughput, review latency, and where agentic systems should insert explanation or provenance artifacts.

Generalization

Any workflow where an automated system produces artifacts the human must validate (code, configs, plans) inverts the cost curve: generation gets cheap, comprehension and verification become the bottleneck.

Code review with LLMs becomes harder because reviewing generated code requires building a mental model without the incremental learning of writing it.
Open source video
Mechanismmedium noveltymoderate evidence

The incremental feedback loop of catching syntax errors and debugging while writing code is itself a learning mechanism; when AI writes the code, learners lose that loop and suffer reduced long-term retention.

Why it matters

Suggests that the error-correction cycle is not overhead to be optimized away but the substrate of understanding—implying that harnesses which fully hide errors or auto-fix silently may degrade operator/model grounding.

Generalization

Feedback loops in a learning or tuning process carry information that cannot be recovered by consuming the final artifact alone; removing them trades speed for fragile competence.

When AI writes code, learners miss the incremental feedback loop of catching syntax errors and debugging, which harms long-term retention.
Open source video
Practicemedium noveltyweak evidence

Removing the ability to run code (writing it on paper) forces the writer to reason about what the machine actually does instead of randomly perturbing code until it passes—an intentional friction that builds a correct mental model.

Why it matters

It is a concrete, cheap training technique for building mental models, and it suggests designing tooling that sometimes withholds execution or suggests predictions rather than answers.

Generalization

Deliberate friction or prediction-before-execution can improve model quality in any iterative system by forcing internal simulation instead of trial-and-error search.

Writing code on paper forces programmers to think about what the machine is actually doing rather than arbitrarily perturbing code until it works.
Open source video
Failure Modemedium noveltymoderate evidence

Over-reliance on LLM-generated code without understanding masks knowledge gaps and produces subtle bugs and unmaintainable systems, as observed in intern bootcamps.

Why it matters

It names a concrete failure mode—masked competence gaps and unmaintainable artifacts—that any team adopting code generation at scale must instrument for, since the symptoms (subtle bugs) surface late.

Generalization

Automation that hides a skill gap defers the cost: defects and maintenance debt accumulate invisibly until the automation is removed or its output must be debugged.

Over-reliance on LLMs without understanding can lead to subtle bugs and unmaintainable systems.
Open source video
Observations from intern bootcamps where reliance on AI tools masks knowledge gaps.
Open source video
Architecturemedium noveltymoderate evidence

Treating the editor as the single home base—where editing, version control, and code review (with inline comments) all happen—reduces context switching compared to fragmenting the loop across web apps and external tools.

Why it matters

It is an explicit architectural stance on consolidating the developer loop into one surface, which is directly relevant to where agentic interfaces (chat panes, review bots) should be embedded versus exposed as separate apps.

Generalization

Unifying workflows into one surface reduces context-switch cost; new capabilities should attach to the existing home base rather than spawning parallel tools.

Jane Street treats the text editor as the software engineer's home base where all interactions, version control, and code reviews happen.
Open source video
Having a centralized editor workflow avoids jumping across multiple web apps and tools.
Open source video
Practicelow noveltymoderate evidence

Practitioner-educators maintained a 50/50 split between programming and teaching work, which the speaker credits with keeping instruction practically relevant in a context where the internal tech stack differs sharply from the outside world.

Why it matters

It is a concrete staffing/organizational mechanism for keeping knowledge work grounded, applicable to how AI-engineering teams allocate time between building and teaching/documenting internal systems.

Generalization

Dual-role staffing (build half the time, teach/document the other half) keeps internal knowledge accurate because the teacher is still subject to the constraints the student will face.

A 50/50 split between programming and educational work helps maintain practical relevance.
Open source video
Teaching is an essential part of Jane Street's culture because internal tech stacks differ significantly from the outside world.
Open source video
Educators at Jane Street are also practitioners who work in the ecosystem.
Open source video

Deep dives

5

Verification burden and reviewer mental models in AI-assisted code review

Research question

What tooling and process interventions can restore or accelerate a reviewer's mental model of LLM-generated code, and how do they affect defect-detection rate, review latency, and reviewer confidence?

Why

Generation is cheap; verification becomes the critical path. Without compensation, review quality silently degrades and teams accumulate subtle bugs and unmaintainable systems.

Code review with LLMs becomes harder because reviewing generated code requires building a mental model without the incremental learning of writing it.
Open source video
Over-reliance on LLMs without understanding can lead to subtle bugs and unmaintainable systems.
Open source video
Source video

Instrumenting masked competence gaps from LLM over-reliance

Research question

Can instrumentation of AI reliance and periodic no-AI performance predict later defects and maintenance burden in production code?

Why

It names a concrete failure mode that surfaces late; early detection would let teams intervene before unmaintainable systems accumulate.

Over-reliance on LLMs without understanding can lead to subtle bugs and unmaintainable systems.
Open source video
Observations from intern bootcamps where reliance on AI tools masks knowledge gaps.
Open source video
Source video

The error-correction loop as a learning substrate in AI-assisted development

Research question

Does removing or hiding the incremental syntax-error and debugging loop reduce long-term retention and debugging ability, and can equivalent friction be reintroduced without sacrificing productivity?

Why

If feedback loops carry information not recoverable from final artifacts, agent harnesses that silently auto-fix may degrade grounding and produce fragile competence.

When AI writes code, learners miss the incremental feedback loop of catching syntax errors and debugging, which harms long-term retention.
Open source video
Writing code on paper forces programmers to think about what the machine is actually doing rather than arbitrarily perturbing code until it works.
Open source video
Source video

Editor-native agent placement and context-switch cost

Research question

Do agentic AI capabilities embedded in the editor reduce context-switch cost and review latency compared with equivalent standalone web-app agents?

Why

Determines where agentic interfaces should live relative to the developer's home base, with direct effects on review throughput and workflow fragmentation.

Jane Street treats the text editor as the software engineer's home base where all interactions, version control, and code reviews happen.
Open source video
Having a centralized editor workflow avoids jumping across multiple web apps and tools.
Open source video
Source video

Deliberate friction: prediction-before-execution in AI-era programming education

Research question

Does forcing prediction of program behavior before execution or AI assistance improve mental-model accuracy and delayed retention compared with immediate execution or AI assistance?

Why

It tests whether intentional friction is a general mechanism for building internal simulation rather than trial-and-error search.

Writing code on paper forces programmers to think about what the machine is actually doing rather than arbitrarily perturbing code until it works.
Open source video
The act of writing notes down helps set information in memory in a qualitatively different way than just hearing it.
Open source video
Source video

Article ideas

4

Generation Is Cheap; Verification Is the Job: Why AI Code Review Needs New Cognitive Tooling

AI-assisted development does not eliminate work; it transfers cognitive load from author to reviewer. Treating review as a secondary cost hides the real bottleneck and silently degrades defect detection. Teams should invest in review-time mental-model reconstruction—staged diffs, generated rationale, and prediction prompts—as first-class infrastructure.

Angle

Reframe the productivity debate around verification burden, with concrete metrics for review latency and defect detection.

Source video

The Editor Is the Agent's Home Base

Agentic AI capabilities should attach to the developer's existing home base rather than spawning parallel web apps. Centralizing editing, version control, review, and AI interactions reduces context-switch cost and enables review-native agent affordances; standalone agent dashboards fragment the loop.

Angle

Architectural argument comparing embedded editor agents against standalone web-app agents on context-switch cost and review latency.

Source video

Don't Automate the Feedback Loop: The Debugging Cycle Is the Curriculum

The syntax-error-and-debug loop is not overhead to be optimized away; it is the mechanism by which programmers build durable mental models. AI harnesses that hide errors, auto-fix silently, or generate before the learner predicts may make work faster while producing fragile competence. Educational and agentic tooling should preserve or reintroduce informative friction.

Angle

Argue against full automation of the error-correction cycle in training and agent design, using cognitive-load and retention evidence.

Source video

Masked Competence Gaps: The Hidden Maintenance Debt of LLM Reliance

Over-reliance on LLMs without understanding does not fail immediately; it defers cost into subtle bugs and unmaintainable systems. Teams adopting code generation at scale need instrumentation for masked knowledge gaps, because the symptoms surface late and look like ordinary maintenance. Curricula and tooling should measure no-AI capability alongside AI-assisted throughput.

Angle

Failure-mode analysis with an instrumentation proposal for detecting masked gaps before they become maintenance debt.

Source video

Project ideas

4

MentalModel Review Scaffold

gatehouse

A reviewer using an editor-native scaffold with staged diffs, generated rationale, and prediction prompts before test results detects more defects in LLM-generated code than a reviewer using a plain diff, without increasing review time.

Proof of concept

Editor plugin that, for each generated hunk, presents a concise rationale, asks the reviewer to predict behavior and failure modes, then reveals tests and actual execution.

Measurement

Defect detection rate, false positive rate, time-on-task, and comprehension quiz scores across reviewers on a corpus of LLM-generated diffs.

Source video

NoAI Probe

beyond-evals

Periodic no-AI capability probes combined with AI-reliance telemetry predict subsequent defect and maintenance rates in a codebase.

Proof of concept

Developer tool that logs AI usage per change, administers short no-AI tasks, and correlates a masked-gap score with future defect and churn metrics.

Measurement

Correlation between masked-gap score and future defect density, rework, and review comments, with lagged analysis.

Source video

PredictBeforeRun

movement-lab

Forcing learners to submit a prediction of program behavior before execution or AI assistance improves delayed retention and debugging transfer compared with immediate execution or AI assistance.

Proof of concept

Coding environment that withholds the run button and AI completions until a prediction is submitted, then shows actual behavior and difference.

Measurement

Delayed post-test retention, debugging task performance, prediction accuracy over time, compared to control with unrestricted execution and AI.

Source video

HomeBase Agent

new

Editor-native agent interactions reduce context-switch count and review latency compared with a functionally equivalent web-app agent.

Proof of concept

Same agent implemented as an editor pane and as a standalone web app; users complete code-change, review, and version-control tasks in each.

Measurement

Context switches, task completion time, review latency, subjective load, and error rate.

Source video

Architectural implications

2

Code review moves from a web app to inline within the developer's primary editor.

Before

Edit, version control, and code review are split across multiple tools and browser apps, forcing context switches between the authoring and reviewing surfaces.

After

All three activities live inside one editor surface, with customized formatting and inline comments during review.

Consequence

Reduced context switching and faster review loops; agentic review or generation capabilities likely belong as editor-native affordances rather than standalone dashboards.

Source video

The bottleneck in AI-assisted development shifts from code authoring to code verification and comprehension.

Before

Cost is dominated by writing code; the author builds a mental model as an inherent byproduct of the act of writing.

After

Generation is cheap, so the load moves to the reviewer, who must build the same mental model without the authoring process.

Consequence

Tooling and process must be redesigned to help reconstruct mental models of generated artifacts (explanations, provenance, staged diffs, prediction steps) or review quality silently degrades.

Source video

Tradeoffs and failure modes

4

Automating the learning process away

Benefit

AI-generated solutions make educational tasks faster and easier to complete.

Cost or risk

Bypasses the struggle and incremental feedback loop that build deep, retained mental models, weakening long-term competence.

Educational programs must resist the temptation to make everything too easy or automated.
Open source video
Source video

Code review of LLM output

Benefit

LLMs can generate code instantly, reducing authoring effort and time to a first draft.

Cost or risk

Reviewing generated code is harder because it requires building a mental model without the incremental learning of writing it, and unreviewed output risks subtle bugs and unmaintainable systems.

Code review with LLMs becomes harder because reviewing generated code requires building a mental model without the incremental learning of writing it.
Open source video
Source video

Passive video learning

Benefit

Recorded lectures scale and are watchable on demand.

Cost or risk

Fails many students and produces weaker engagement and retention than active, synchronous, in-person instruction.

Active learning and in-person engagement are critical for attention and retention.
Open source video
Source video

Removing execution while coding

Benefit

Forces reasoning about what the machine actually does rather than trial-and-error perturbation.

Cost or risk

Sacrifices the normal debugging and error-catching loop; may not transfer to production-style work that depends on that loop.

Writing code on paper forces programmers to think about what the machine is actually doing rather than arbitrarily perturbing code until it works.
Open source video
Source video

Open questions

4

What is the value of teaching people to write code when an LLM can write it instantly?

Why unresolved

The summary states the tension explicitly but does not state how Jane Street resolves it; curricula are described as needing to adapt without a settled answer being given.

Research direction

Design and measure alternative curricula that shift emphasis from authorship to verification, debugging, and system design, comparing retention and defect rates against traditional instruction.

Source video

How can a reviewer efficiently build a mental model of generated code without the incremental authoring process?

Why unresolved

The summary identifies this as a new source of difficulty in code review but provides no mechanism to compensate.

Research direction

Prototype review-time tooling (staged diffs, generated rationale, prediction prompts) and measure reviewer comprehension and defect-detection rates.

Source video

Where should agentic AI capabilities live relative to the developer's 'home base'?

Why unresolved

The summary documents the editor-as-home-base philosophy and mentions custom LLM integrations, but does not state whether agentic flows are embedded in the editor or kept external.

Research direction

Compare editor-native agent interaction against separate web-app agents on context-switch cost, review latency, and task completion.

Source video

Does deliberate friction (withholding execution) still produce better mental models when AI assistance is available?

Why unresolved

The paper-coding and note-taking claims come from cognitive psychology and classroom observation, not controlled AI-era studies.

Research direction

Run experiments contrasting prediction-before-execution against AI-assisted generation, measuring long-term retention and debugging ability.

Source video

Key claims

8
causalVerification needed

Reviewing LLM-generated code is harder than reviewing human-written code because it requires building a mental model without the incremental learning of writing it.

Evidence

Code review with LLMs becomes harder because reviewing generated code requires building a mental model without the incremental learning of writing it.

Question

Do reviewers detect fewer defects and take longer on LLM-generated diffs than on equivalent human-written diffs, controlling for size and complexity?

Source video
causalVerification needed

Learners relying on AI tools miss the incremental feedback loop of catching syntax errors and debugging, which harms long-term retention.

Evidence

When AI writes code, learners miss the incremental feedback loop of catching syntax errors and debugging, which harms long-term retention.

Question

Do learners who use AI code generation retain less than a control group on delayed assessments, and does the gap correlate with reduced error-correction activity?

Source video
causalVerification needed

Over-reliance on LLMs without understanding leads to subtle bugs and unmaintainable systems.

Evidence

Over-reliance on LLMs without understanding can lead to subtle bugs and unmaintainable systems.

Question

Does measured LLM reliance predict higher defect and maintenance rates in production code over time?

Source video
causalVerification needed

The act of writing notes down sets information in memory qualitatively differently than only hearing it.

Evidence

The act of writing notes down helps set information in memory in a qualitatively different way than just hearing it.

Question

In controlled studies, does note-writing produce measurably different retention than passive listening, and does the effect persist in AI-assisted note capture?

Source video
comparativeVerification needed

Active learning and in-person engagement are critical for attention and retention compared to passive video delivery.

Evidence

Active learning and in-person engagement are critical for attention and retention.

Question

How large is the retention delta between synchronous active instruction and recorded lectures across different student populations?

Source video
causalVerification not requested

Centralizing the developer workflow in a single editor avoids jumping across multiple web apps and tools.

Evidence

Having a centralized editor workflow avoids jumping across multiple web apps and tools.

Source video
opinionVerification needed

A 50/50 split between programming and educational work helps maintain practical relevance for practitioner-educators.

Evidence

A 50/50 split between programming and educational work helps maintain practical relevance.

Question

Does the practitioner-educator split measurably improve the currency and uptake of internal technical training versus dedicated full-time educators?

Source video
predictionVerification needed

Curricula must adapt to teach verification, debugging, and system design as AI takes over code authoring.

Evidence

Curricula must adapt to teach verification, debugging, and system design

Question

Do verification-and-debugging-centric curricula outperform authorship-centric ones when graduates work in AI-assisted environments?

Source video

Connections

4