Machine Learning Street Talk · Published 2026-09-11

What Building an AI Scientist Actually Requires Beyond Intelligence — Edward Hughes

privateReview: unreviewedConfidence: high
ai-scientistreplicationopen-endednessreinforcement-learningrubricslong-horizon-rlcredit-assignmentrubric-evaluationreward-hackingharness-engineeringGRPO
Open on YouTube ↗

Summary

Overview

  • Speaker: Edward Hughes
  • Channel: Machine Learning Street Talk
  • Main topic: Training AI agents to replicate scientific research and achieve open-endedness
  • Purpose: To explain the technical and philosophical foundations of training AI scientist systems for open-ended scientific discovery and paper replication. Edward Hughes, co-founder and chief scientist at Inherent, discusses the transition from individual AI intelligence to collective intelligence and the design of AI scientist systems. He covers the Faraday paper on training AI scientists to replicate machine learning and AI-for-science research papers using coding agents, GRPO, and automated rubric-based judges. The conversation explores open-endedness, the limits of optimization, evolutionary algorithms, and the importance of open-ended task spaces and evaluation in building autonomous scientific discovery agents.

Topic Map

From Individual Intelligence to Collective Intelligence

  • Explanation: The shift from single-agent capability to collective intelligence across multiple interacting agents and humans.
  • Key claims:
    • The next era is collective intelligence rather than individual intelligence.
    • AI by itself is meaningless without interaction with the wider ecosystem and scientific progress.
  • Examples:
    • Move 37 in Go was innovative without being creative in the true sense.
  • Terminology:
    • collective intelligence
    • recursive self-improvement
    • phase transition
  • Why it matters: Defines the paradigm shift required for autonomous AI systems to drive scientific progress.

Training AI Scientists to Replicate Research (The Faraday Paper)

  • Explanation: Introducing Faraday, a 27B parameter AI scientist agent that leverages coding agents as tools to replicate machine learning and AI-for-science papers.
  • Key claims:
    • Faraday outperforms frontier models like Claude and Codex on paper replication tasks.
    • Prompt optimization using automated judging loops improves agent performance on long-horizon tasks.
  • Examples:
    • Replicating 300 AI and ML papers published between 1990 and 2026.
  • Terminology:
    • Faraday
    • GRPO
    • per-task rubric generator
    • rubric-based judge
    • Replica task space
  • Why it matters: Demonstrates that open-weight models with proper scaffolding can perform rigorous scientific replication.

Open-Endedness and the Limits of Optimization

  • Explanation: Exploring the relationship between open-endedness, optimization, constraints, and why greatness cannot be planned.
  • Key claims:
    • Optimization is the wrong tool for creativity because it relies on known objectives.
    • Open-endedness requires moving from solving known problems to discovering new problems and questions.
    • Constraints act as a form of abstract knowledge that enables creative discovery.
  • Examples:
    • AlphaGo optimized within a fixed rule set, whereas open-ended systems discover new domains.
  • Terminology:
    • open-endedness
    • optimization
    • constraints
    • acceptation
    • observer-dependent novelty
  • Why it matters: Provides a theoretical foundation for building systems capable of superhuman open-ended innovation.

Organizational Design and Harness Engineering

  • Explanation: Drawing analogies between human organizational evolution, harness engineering, and AI agent architectures.
  • Key claims:
    • Harness engineering is crucial for guiding AI agents, but too much hardcoding limits generalization.
    • Recursive self-improvement should happen at the company level through interacting human-AI teams.
  • Examples:
    • Comparing the industrial revolution and electric dynamos to modern AI agent engineering.
  • Terminology:
    • harness engineering
    • Darwin Gödel Machine (DGM)
    • turn-level credit assignment
    • per-task rubric
  • Why it matters: Bridges machine learning agent design with organizational economics and evolutionary biology.

Key Points

Faraday Agent Architecture

  • Explanation: Faraday uses a Qwen 3.6-27B base model combined with Codex as a tool, optimized via Group Relative Policy Optimization (GRPO) with rubric-based judging.
  • Evidence: Performance evaluation charts on ML and AI-for-science held-out tasks.
  • Practical implication: Enables open-weight models to compete with or beat proprietary frontier models on complex scientific tasks.

The Role of Rubrics in Evaluation

  • Explanation: Instead of relying on single binary rewards or simple test cases, Faraday uses per-task rubric generators and LLM judges to assess visual fidelity, claim reproduction, implementation fidelity, experimental depth, and scientific integrity.
  • Evidence: Figure 3 showing higher human agreement and lower noise with rubric-based judging.
  • Practical implication: Reduces reward hacking and provides robust multi-dimensional evaluation for non-verifiable agent tasks.

Turn-Level Credit Assignment

  • Explanation: Assigning reward credit at the individual turn level rather than just the final rollout prevents credit assignment collapse in long-horizon RL tasks.
  • Evidence: Ablation plots showing stability improvements when turn-level credit assignment is used.
  • Practical implication: Stabilizes training for multi-turn coding and scientific experimentation agents.

Frameworks, Models & Processes

Faraday Replication Pipeline

  • How it works: Generates replication tasks from published papers, executes them using a coding agent in a container, and evaluates outputs using rubric-based judges.
  • Components:
    • Replica task space
    • Faraday policy with Codex tool
    • Per-task rubric generator
    • Rubric-based judge
    • GRPO optimizer
  • When to use: When training AI agents to execute complex, multi-step scientific paper replication tasks.

Open-Endedness Definition

  • How it works: Defines open-endedness as a system's ability to continually generate artifacts that are both novel and learnable to an observer.
  • Components:
    • System producing artifacts
    • Observer evaluating novelty
    • Observer evaluating learnability
  • When to use: When assessing whether an AI system is capable of unbounded cumulative discovery.

Examples & Case Studies

Faraday was evaluated on 68 held-out AI-for-science tasks against frontier models like Claude, Codex, and Qwen.

  • Illustrates: The effectiveness of training open-weight models with targeted tool use and rubric-based reinforcement learning.
  • Lesson: Specialized agent scaffolding and careful reward design can allow smaller open models to outperform larger closed models.

Dmitry Bahdanau added an attention mechanism to a machine translation model initially just as a heuristic component.

  • Illustrates: How foundational discoveries often start as seemingly minor architectural modifications that later become universal.
  • Lesson: True innovation often comes from breaking existing constraints in unexpected ways.

Actionable Takeaways

  • Immediate:
    • Rubric-based judges significantly reduce noise and hacking compared to binary verification.
    • Turn-level credit assignment is essential for stabilizing RL training on multi-turn coding agents.
    • Open-weight models like Qwen-27B can achieve top-tier scientific performance with proper harness engineering.
  • Strategic:
    • Open-endedness and discovery are more fundamental to AGI than static optimization against fixed benchmarks.
    • Collective intelligence and human-AI collaboration will dominate individual agent capabilities.
    • Scaling task diversity is more important than scaling model size alone for scientific discovery agents.
  • Questions to investigate:
    • How do we avoid good-hacking when scaling evaluation rubrics for autonomous discovery?
    • What is the equivalent of OKRs for recursive self-improving AI agent organizations?
    • How can open-ended agent systems autonomously discover valid scientific domains without human curation?

Claims Worth Verifying

  • Faraday outperforms Claude Opus 4.8 and GPT-5.5 on held-out paper replication tasks. (empirical)
  • GRPO stabilizes reinforcement learning by computing group-relative advantages without a separate value network. (technical)

Notable Quotes

"How do we start to build AI scientist systems that go beyond simply answering questions that we pose and start to ask the kinds of questions that lead to open-ended creative discovery?" (at 0:00) "The replicability of papers is a cornerstone of scientific knowledge, ensuring the reliability of existing results and providing a base for further experiments." (at 0:30) "That kind of open-ended invention is the mechanism by which human individuals and society at large accumulates new knowledge and technology." (at 151:50)

Compressed Summary

  • Faraday trains a 27B model to replicate scientific papers using coding agents and GRPO.
  • Per-task rubric generators and LLM judges replace brittle binary verification.
  • Turn-level credit assignment prevents RL collapse on long-horizon tasks.
  • Open-endedness requires systems to generate artifacts that are both novel and learnable to an observer.
  • Keywords: ai-scientist, replication, open-endedness, reinforcement-learning, rubrics
  • Core insight: Building an AI scientist requires moving beyond static optimization to open-ended task generation, rubric-based evaluation, and multi-turn credit assignment.

Core insights

6
Mechanismhigh noveltystrong evidence

Long-horizon multi-turn agent RL collapses under rollout-level credit assignment; assigning reward credit at the individual turn level is what stabilizes training for coding and scientific experimentation agents.

Why it matters

If credit must be resolved per turn rather than per episode, the reward pipeline and trace/logging infrastructure must capture and attribute per-turn signals, which changes how rollouts are stored, replayed, and optimized.

Generalization

Applies to any long-horizon agentic RL loop (coding agents, tool-use chains, multi-step research agents) where sparse terminal reward would otherwise be diluted across many turns.

Assigning reward credit at the individual turn level rather than just the final rollout prevents credit assignment collapse in long-horizon RL tasks.
Open source video
Ablation plots showing stability improvements when turn-level credit assignment is used.
Open source video
Mechanismhigh noveltystrong evidence

For non-verifiable agent tasks, replace single binary rewards with a per-task generated rubric scored by an LLM judge across multiple dimensions (visual fidelity, claim reproduction, implementation fidelity, experimental depth, scientific integrity).

Why it matters

This makes the evaluator a generated, task-specific artifact rather than a fixed test, so evaluation infrastructure must produce rubrics per task and treat rubric quality as a first-class engineering surface.

Generalization

Generalizes to any agent task lacking unit tests: report generation, data analysis, research replication, design work. The rubric generator plus judge pattern is a reusable evaluation abstraction.

Faraday uses per-task rubric generators and LLM judges to assess visual fidelity, claim reproduction, implementation fidelity, experimental depth, and scientific integrity.
Open source video
Figure 3 showing higher human agreement and lower noise with rubric-based judging.
Open source video
Reduces reward hacking and provides robust multi-dimensional evaluation for non-verifiable agent tasks.
Open source video
Empirical Resulthigh noveltymoderate evidence

A 27B open-weight policy paired with an external coding agent as a tool, tuned with GRPO over rubric rewards, beats proprietary frontier models on paper replication — capability came from scaffolding plus reward design, not parameter scale.

Why it matters

Reframes the build-vs-buy decision: specialized harnesses around small open models can be competitive on narrow, deeply-specified tasks, which changes cost, deployability, and fine-tuning economics.

Generalization

Suggests that for well-scoped, high-complexity domains, investing in harness and reward design may dominate investing in base-model size.

Faraday uses a Qwen 3.6-27B base model combined with Codex as a tool, optimized via Group Relative Policy Optimization (GRPO) with rubric-based judging.
Open source video
Specialized agent scaffolding and careful reward design can allow smaller open models to outperform larger closed models.
Open source video
Mental Modelmedium noveltymoderate evidence

Open-endedness is operationalized as a system's ability to continually generate artifacts that are both novel and learnable to an observer — novelty is observer-dependent, not intrinsic.

Why it matters

A testable definition makes 'open-ended' a measurable property tied to a specified observer, rather than a vague aspiration, which forces evaluation design to name the observer and the learnability criterion.

Generalization

Applies to benchmarking any discovery system, curriculum generator, or self-improvement loop where the candidate artifact stream must be graded against an evaluator.

Defines open-endedness as a system's ability to continually generate artifacts that are both novel and learnable to an observer.
Open source video
Open-endedness requires moving from solving known problems to discovering new problems and questions.
Open source video
Tradeoffmedium noveltymoderate evidence

Harness engineering faces a direct tension: hardcoded scaffolding is necessary to guide agents, but excess hardcoding suppresses generalization.

Why it matters

It gives a concrete failure mode for agent runtimes — a harness tuned to demo tasks can silently cap the policy's ceiling, so harness rigidity becomes a measured design variable, not a fixed assumption.

Generalization

Applies anywhere prompts, tool schemas, workflow graphs, or state machines are hand-authored around a model: each constraint is simultaneously guidance and a generalization tax.

Harness engineering is crucial for guiding AI agents, but too much hardcoding limits generalization.
Open source video
Mental Modelmedium noveltyweak evidence

Optimization is structurally the wrong tool for creative discovery because it presupposes known objectives; constraints, by contrast, function as compressible abstract knowledge that enables discovery without pre-specifying the target.

Why it matters

Argues against treating agent improvement as pure objective maximization and toward constraint-based search spaces — a different architecture for task generation and self-improvement.

Generalization

Relevant to curriculum design, task-space construction, and any self-improvement loop that risks optimizing toward a fixed objective while claiming to be open-ended.

Optimization is the wrong tool for creativity because it relies on known objectives.
Open source video
Constraints act as a form of abstract knowledge that enables creative discovery.
Open source video
AlphaGo optimized within a fixed rule set, whereas open-ended systems discover new domains.
Open source video

Deep dives

5

Mechanism of turn-level credit assignment in long-horizon agent RL

Research question

What specifically does turn-level credit assignment change — gradient variance across the trajectory, reward density per step, or optimizer dynamics — such that credit assignment collapse is prevented, and over what horizon lengths does the effect hold?

Why

If the fix is variance reduction, the requirement is a step-aware advantage estimator; if it is reward density, the requirement is a per-turn reward model or rubric. Those lead to different infrastructure: step-indexed traces and replay buffers versus a dense reward scorer. Without isolating the mechanism, teams will copy the surface recipe (GRPO plus turns) without the component that actually matters.

Assigning reward credit at the individual turn level rather than just the final rollout prevents credit assignment collapse in long-horizon RL tasks.
Open source video
Ablation plots showing stability improvements when turn-level credit assignment is used.
Open source video
What exactly makes turn-level credit assignment prevent collapse — is it variance reduction across turns, denser reward, or something else?
Open source video
Source video

Rubric judges under iterated optimization pressure

Research question

Do rubric-based LLM judges remain discriminative as the policy is trained against them, or does policy improvement toward the rubric surface cause human agreement to decay over training steps?

Why

The claim that rubric judging reduces reward hacking is measured on an untrained or lightly trained policy. If human agreement decays under iterated pressure, then rubric quality is not a one-time artifact but a continuously maintained, adversarially contested component, and any training pipeline built on rubric rewards needs a rubric-refresh loop with its own cadence and cost.

Reduces reward hacking and provides robust multi-dimensional evaluation for non-verifiable agent tasks.
Open source video
Figure 3 showing higher human agreement and lower noise with rubric-based judging.
Open source video
Do rubric-based judges remain discriminative as the policy improves and learns to satisfy the rubric surface rather than the underlying task?
Open source video
Source video

Transfer from paper replication to novel problem generation

Research question

Does training an agent to replicate published papers improve its ability to generate genuinely new hypotheses or new problem formulations, or does replication skill only demonstrate mastery of existing literature?

Why

The entire value proposition of an AI scientist rests on this transfer. If replication performance is orthogonal to discovery, then replication benchmarks are a proxy metric that can be maximized indefinitely without producing scientific output — a measurement trap for anyone funding or building on the replication task space.

Replicating 300 AI and ML papers published between 1990 and 2026.
Open source video
Replication and open-ended discovery are presented as distinct objectives, but the summary provides no evidence connecting replication performance to novel discovery.
Open source video
Open-endedness requires moving from solving known problems to discovering new problems and questions.
Open source video
Source video

Operationalizing observer-dependent novelty into a non-collapsing benchmark

Research question

Can novelty-plus-learnability relative to a named observer model be measured in a way that does not collapse into a fixed scalar objective the system can optimize against?

Why

The definition is testable in principle but brittle in practice: any measurable novelty score becomes an objective once a policy trains against it. Resolving how to specify the observer and rotate or hold it out determines whether open-endedness can be an evaluation axis at all, or whether it must remain descriptive.

Defines open-endedness as a system's ability to continually generate artifacts that are both novel and learnable to an observer.
Open source video
Can the observer-dependent novelty/learnability definition of open-endedness be operationalized into a benchmark that does not collapse to a fixed objective?
Open source video
Novelty and learnability are properties relative to a specified observer, making open-endedness measurable only against that observer.
Open source video
Source video

Harness rigidity as a measured generalization tax

Research question

Can the guidance-versus-generalization tradeoff of hand-authored scaffolding be quantified per constraint, such that each prompt, tool schema, or state-machine edge has a measurable effect on out-of-distribution task performance?

Why

Right now the tradeoff is stated qualitatively, which makes harness design a matter of taste. If constraint count or constraint type can be regressed against held-out task performance, harness engineering becomes an ablation discipline with a budget rather than an art, and demo-tuned harnesses become detectable before they cap the policy.

Harness engineering is crucial for guiding AI agents, but too much hardcoding limits generalization.
Open source video
It gives a concrete failure mode for agent runtimes — a harness tuned to demo tasks can silently cap the policy's ceiling, so harness rigidity becomes a measured design variable, not a fixed assumption.
Open source video
Hardcoded scaffolding trades guidance against generalization capacity.
Open source video
Source video

Article ideas

4

Capability Came From the Harness, Not the Parameters

The Faraday result reframes the build-versus-buy decision: on narrow, deeply-specified tasks, investment in harness design and reward shaping dominates investment in base-model scale, and the right comparison is not model versus model but system versus system.

Angle

Take the 27B-plus-external-tool configuration as evidence that the unit of competition has shifted from model to assembled system, then push on what that means for teams currently paying frontier-model margins for narrow workloads.

Source video

Your Evaluator Is a Pipeline Component Now

When evaluation is generated per task rather than fixed in advance, rubric quality becomes a first-class engineering surface with its own latency, cost, and failure modes, and treating evaluation as static infrastructure guarantees that reward hacking is discovered late and expensively.

Angle

Argue that the rubric generator plus judge pattern is a reusable abstraction for every domain without unit tests, then detail the operational consequences: per-task artifact production, judge drift monitoring, and rubric-refresh cadence.

Source video

Optimization Cannot Discover the Problem It Is Optimizing

Every system that claims open-endedness while training against a fixed objective is optimizing inside a known rule set, and constraints rather than objectives are the workable substitute for specifying search structure without specifying the target.

Angle

Use AlphaGo's fixed rule set as the limiting case, then press on the practical question the argument avoids: how constraint-based search spaces are constructed and how one detects that an 'open-ended' loop has quietly become an objective maximizer.

Source video

The Harness Is the Ceiling

Hardcoded scaffolding silently caps policy generalization, so harness rigidity should be treated as a measured design variable with an ablation budget rather than a fixed assumption baked in during the first demo.

Angle

Frame each hand-authored prompt, tool schema, and workflow edge as simultaneously guidance and generalization tax, and propose constraint-count regressions against held-out task performance as the diagnostic.

Source video

Project ideas

4

TurnCreditBench

movement-lab

Step-aware credit assignment reduces gradient variance and prevents training collapse relative to rollout-level credit, and the size of the advantage grows monotonically with task horizon length.

Proof of concept

Train the same policy on identical long-horizon agentic traces under three credit schemes — rollout-level, turn-level, and an intermediate discounted scheme — sweeping horizon length from short to long while logging per-step advantage statistics, gradient variance, and held-out task success.

Measurement

Gradient variance per update, collapse incidence (success rate falling to baseline mid-run), and final held-out task success as a function of horizon length across the three schemes.

Source video

RubricRot

beyond-evals

LLM judge agreement with human raters decays measurably over successive rounds of policy training against rubric rewards, and the decay rate depends on rubric granularity.

Proof of concept

Run iterated training rounds where a policy optimizes against a fixed generated rubric while periodically re-scoring a frozen sample of trajectories with human raters and with freshly regenerated rubrics; track both agreement curves against training step.

Measurement

Human-judge agreement coefficient per training round, gap between fixed-rubric and regenerated-rubric scores on the same trajectories, and steps until the gap exceeds a pre-registered threshold.

Source video

RubricGen Service

beyond-evals

Runtime-generated per-task multi-dimensional rubrics, scored by an LLM judge, correlate with human assessment better and are less exploitable than a single binary outcome reward on non-verifiable agent tasks.

Proof of concept

Build a pipeline that ingests an agent trace and a task description, emits a task-specific rubric across the five dimensions used in the Faraday configuration, scores the trace, and exposes rubric provenance and versioning so downstream training can pin or rotate rubric revisions.

Measurement

Correlation with human scores versus a binary-reward baseline, rank correlation under adversarial trajectories written to satisfy the rubric without doing the task, and per-rubric generation cost and latency.

Source video

Replication-to-Discovery Transfer Suite

new

Replication performance on held-out papers does not predict performance on tasks requiring a novel hypothesis or a new problem formulation; the two capabilities are measurably dissociated.

Proof of concept

Take agents at several checkpoints of replication training and evaluate each on a paired task set: a replication task and a matched novel-problem-generation task drawn from the same subfield, scoring both with the same rubric dimensions plus a novelty assessment by a held-out observer model.

Measurement

Rank correlation between replication score and novel-generation score across checkpoints and subfields; evidence of dissociation where replication improves while novel generation stays flat or regresses.

Source video

Architectural implications

3

Evaluation is generated per task rather than fixed in advance.

Before

Evaluation is a static benchmark or unit-test suite written by humans before the agent runs.

After

A rubric generator produces task-specific criteria at runtime, and an LLM judge scores the trajectory against them.

Consequence

Evaluation becomes a pipeline component with its own quality, latency, and cost profile, and rubric quality becomes a tunable lever for reducing reward hacking.

Source video

Reward attribution moves from the rollout boundary into the trajectory.

Before

One terminal reward per episode, assigned at the end of the rollout.

After

Per-turn credit assigned across the trajectory, requiring traces to carry per-step reward signals.

Consequence

Rollout storage, replay, and optimizer integration must be step-aware; training stability depends on this plumbing.

Source video

Model capability is decomposed into a policy plus an external tool rather than absorbed into weights.

Before

Frontier capability assumed to come from a larger proprietary base model.

After

A smaller open-weight policy calls an external coding agent as a tool and is tuned on task-specific reward.

Consequence

Enables open-weight deployment and fine-tuning control, and shifts the optimization target from scale to harness and reward design.

Source video

Tradeoffs and failure modes

4

Harness engineering rigidity

Benefit

Harness engineering is crucial for guiding AI agents.

Cost or risk

Too much hardcoding limits generalization.

Harness engineering is crucial for guiding AI agents, but too much hardcoding limits generalization.
Open source video
Source video

Optimization as a discovery mechanism

Benefit

Optimization works well inside a fixed rule set, as with AlphaGo.

Cost or risk

It relies on known objectives, so it cannot drive discovery of new problems or domains.

Optimization is the wrong tool for creativity because it relies on known objectives.
Open source video
Source video

Rubric-based judging

Benefit

Reduces reward hacking, lowers noise, and yields higher human agreement than binary verification.

Cost or risk

It is applied to non-verifiable agent tasks, so evaluation rests on generated rubrics and LLM judges rather than ground truth.

Reduces reward hacking and provides robust multi-dimensional evaluation for non-verifiable agent tasks.
Open source video
Source video

Recursive self-improvement locus

Benefit

Placing recursive self-improvement at the company level through interacting human-AI teams broadens the improvement surface beyond a single model.

Cost or risk

Depends on organizational coordination instead of a single verifiable agent-level loop.

Recursive self-improvement should happen at the company level through interacting human-AI teams.
Open source video
Source video

Open questions

4

What exactly makes turn-level credit assignment prevent collapse — is it variance reduction across turns, denser reward, or something else?

Why unresolved

The summary reports the ablation result but not the underlying mechanism or the alternative credit-assignment schemes it was compared against.

Research direction

Ablate turn-level versus rollout-level versus intermediate credit-assignment schemes across horizon lengths and measure gradient variance and stability.

Source video

Do rubric-based judges remain discriminative as the policy improves and learns to satisfy the rubric surface rather than the underlying task?

Why unresolved

The summary claims reduced reward hacking but does not report judge behavior under iterated optimization pressure.

Research direction

Run adversarial or iterated-training studies pitting rubric judges against policies trained against them, tracking human agreement over training time.

Source video

Does the ability to replicate published papers transfer to producing genuinely new problems and domains, or does it only demonstrate mastery of existing literature?

Why unresolved

Replication and open-ended discovery are presented as distinct objectives, but the summary provides no evidence connecting replication performance to novel discovery.

Research direction

Evaluate replication-trained agents on tasks requiring novel hypotheses or new problem generation, and test whether replication skill predicts them.

Source video

Can the observer-dependent novelty/learnability definition of open-endedness be operationalized into a benchmark that does not collapse to a fixed objective?

Why unresolved

The definition names an observer but the summary does not specify how novelty or learnability are measured or who the observer is in practice.

Research direction

Build an evaluation harness with a specified observer model and test whether measured open-endedness predicts downstream discovery outcomes.

Source video

Key claims

7
comparativeVerification needed

Faraday, a 27B open-weight agent using Codex as a tool, outperforms frontier models on paper replication tasks.

Evidence

Faraday outperforms frontier models like Claude and Codex on paper replication tasks.

Question

On which held-out tasks, with what scoring, and by what margin does Faraday outperform Claude and Codex?

Source video
comparativeVerification needed

Rubric-based judging achieves higher human agreement and lower noise than alternative evaluation approaches.

Evidence

Figure 3 showing higher human agreement and lower noise with rubric-based judging.

Question

What was the human-agreement baseline, sample size, and inter-rater protocol for Figure 3?

Source video
causalVerification needed

Turn-level credit assignment prevents credit assignment collapse in long-horizon RL tasks.

Evidence

Assigning reward credit at the individual turn level rather than just the final rollout prevents credit assignment collapse in long-horizon RL tasks.

Question

What does the ablation show about collapse without turn-level credit assignment, and over what task horizons?

Source video
factualVerification needed

Faraday was trained and evaluated over a large corpus of AI/ML papers with a held-out AI-for-science set.

Evidence

Replicating 300 AI and ML papers published between 1990 and 2026.

Question

How were the 300 replication tasks constructed, and how were the 68 held-out tasks selected to avoid contamination?

Source video
causalVerification needed

Constraints act as compressible abstract knowledge that enables creative discovery, whereas optimization presupposes known objectives.

Evidence

Constraints act as a form of abstract knowledge that enables creative discovery.

Question

Is there a demonstrable causal link between constraint structure and measured creative output, or is this a conceptual claim?

Source video
opinionVerification needed

Open-endedness should be defined as continually generating artifacts that are both novel and learnable to an observer.

Evidence

Defines open-endedness as a system's ability to continually generate artifacts that are both novel and learnable to an observer.

Question

Does this definition discriminate between systems that accumulate discovery and those that merely produce diverse output?

Source video
opinionVerification not requested

Recursive self-improvement should occur at the company level through interacting human-AI teams rather than within a single agent.

Evidence

Recursive self-improvement should happen at the company level through interacting human-AI teams.

Source video

Connections

5