Jiho Sim

Accepted as a poster at the AI for Science workshop, ICML 2026 (Track 3: AI Scientist Proposal Competition). Changhee Cho*, YeJun Ji*, Su-Young Jin*, Sumin Lee*, Jiho Sim* (Seoul National University).
*Equal contribution.

Paper (PDF) Poster (PDF)

We’re actively developing this, and I’d welcome any discussion on similar projects or AI4Science topics — feel free to reach out!

Background

This project started as our entry to the AI Scientist Competition at the ICML 2026 AI for Science workshop’s Track3. The paper contains details on our proposal of an AI Scientist that can come up with novel but grounded hypotheses. It details how autonomous the system is, how the agents are wired together, how we’d evaluate it, and how humans stay in the loop.

It’s since moved into active development, so the design here reflects where the system is headed.

Abstract

AI scientists are advancing fast, but most systems tangle evidence management with reasoning, which limits hypothesis generation to narrow, prior-driven contexts. We propose the Evidence-Grounded AI Scientist (EGAS), which strictly separates evidence curation from reasoning. A Context Optimizer curates and grounds evidence without interpreting it, letting a Tree-of-Thoughts Hypothesis Generator explore diverse reasoning paths and produce novel, falsifiable hypotheses. Experimental outcomes feed back into the system, and a Diagnostician flags anomalous signals before they corrupt downstream interpretation. This growing body of validated evidence is meant to improve later ideation cycles. As a proof of concept, we develop EGAS as an integrated member of a drug discovery team, spanning target suggestion through wet-lab validation.

Motivation

The bottleneck is ideation, not execution. Because LLMs are trained on published literature, their priors favor likely continuations of existing knowledge over novel connections, and retrieval augmentation inherits that bias: the same prior shapes both what gets retrieved and how it’s weighted. Treating hypothesis generation as a single-pass task leaves the whole process leaning on the model’s prior. EGAS addresses this structurally rather than through prompting alone.

Architecture

Two principles drive the design: evidence curation is separated from reasoning so the prior can’t bias the evidence base, and ideation is treated as a grounded search rather than one-shot generation. The pipeline has three interconnected stages.

• Context Optimizer (CO). Acts as a Librarian. It collects, grounds, and indexes evidence but never interprets, summarizes, or ranks it. Evidence is stored as atomic claims with provenance pointers, reachable only through a stateless retrieval interface. The separation keeps the evidence base from narrowing the reasoning space, and makes any failure traceable to either the evidence layer or the reasoning layer. CO is append-only, so it doubles as long-term memory that compounds over time.
• Hypothesis Generator (HG). A grounded Tree-of-Thoughts search over CO’s evidence, driven by three modules: an Explorer that generates and expands hypotheses, a State Estimator that evaluates them, and a Router that picks the next action. Diversity is structurally induced: HG partitions evidence into distinct subsets and explores each through a different reasoning mode, so sibling hypotheses differ along interpretable dimensions rather than through random sampling. Evaluation is two-level: node-level checks that a hypothesis is evidence-backed and testable; trajectory-level checks that the step producing it was justified. Those signals let the Router prune unsound claims, expand promising-but-flawed ones, or graduate sound hypotheses.
• Collaborative discovery loop. A graduated hypothesis goes to an Experiment Designer, which picks an In Silico or Wet-Lab verification module. The In Silico Scientist runs computational pipelines; the Wet-Lab Assistant writes reproducible, version-controlled protocols. Before results enter the database, a Diagnostician intercepts anomalous signals, traces root causes, and proposes protocol revisions. A Knowledge Consolidator then records all findings, including negative results and diagnostic reports, back into CO for future reasoning.

Human oversight stays central throughout. Domain experts curate retained evidence and choose which hypotheses to test, controlling what the AI sees independently of how it reasons.

Evaluation roadmap

The primary test uses a knowledge cutoff: CO’s retrieval is restricted to pre-cutoff evidence, and we ask whether EGAS can reproduce discoveries established in the literature afterward, rather than regressing toward its prior. Beyond final hypothesis quality, the search process itself is assessed. CO’s retrieval and grounding, and HG’s reasoning-step quality, are measured on existing biomedical benchmarks; the Diagnostician on custom anomaly-detection datasets; and wet-lab validation expands progressively from narrow candidate checks to end-to-end therapeutic discovery, testing whether accumulating evidence actually improves later ideation.