LLM-AutoSciLab: Closed-Loop Scientific Law Discovery

Abstract

Scientific discovery is a closed-loop process where hypotheses guide data acquisition and observations refine the hypothesis space. Yet most approaches reduce discovery to supervised learning over fixed datasets, where limited observations can support multiple plausible mechanisms that fit locally but fail to generalize. Thus, the key challenge is selecting informative observations to resolve uncertainty, shifting the focus from static inference to adaptive data acquisition.

To address this, we propose LLM-AutoSciLab, a closed-loop framework that couples hypothesis generation with hypothesis-conditioned experiment selection and mechanism refinement. Rather than fitting models to passively collected data, LLM-AutoSciLab iteratively proposes plausible hypotheses, selects informative experiments to distinguish or refine them, and updates its state using the resulting evidence.

To evaluate dynamic, closed-loop scientific discovery with active data acquisition, we introduce ActiveSciBench, comprising (i) ActiveSciBench-Chem (57 enzyme-kinetics domains) and (ii) ActiveSciBench-GRN (45 gene-regulatory-network tasks). Across NewtonBench, ActiveSciBench-Chem, and ActiveSciBench-GRN, LLM-AutoSciLab outperforms prior methods, achieving 67.6% and 35.1% symbolic accuracy and 31.1% exact graph recovery. Hypothesis-guided experimentation is 2–5× more sample-efficient than the strongest competing baselines.

Method Overview

LLM-AutoSciLab models scientific discovery as active hypothesis-conditioned experiment design — iteratively generating hypotheses, selecting maximally informative experiments, and refining mechanisms using observed evidence.

LLM-AutoSciLab: hypothesis generation, experiment selection, and confidence feedback loop

Figure 1. Overview of LLM-AutoSciLab. (A) An LLM generates candidate hypotheses from observations and memory. (B) Experiments are selected in regions of maximal hypothesis disagreement. (C) Candidates are refined via domain-specific optimization, with a bootstrap confidence gate filtering brittle hypotheses.

Step A

Hypothesis Generation

A smaller LLM generates diverse candidate hypotheses conditioned on accumulated observations and memory. A larger LLM synthesizes these into a structured primary hypothesis, alternatives, and diagnostic search regions — without premature collapse.

Step B

Hypothesis-Conditioned Experiment Selection

Acquisition targets regions of maximal disagreement among competing hypotheses. In Disambiguate mode, experiments separate competing mechanisms; in Refine mode, they improve parameterization of a stable candidate.

Step C

Optimization & Confidence Feedback

Each hypothesis is refit on the augmented dataset. A bootstrap confidence gate filters unstable candidates. The refined mechanism is written back to memory, informing subsequent hypothesis generation and acquisition.

Key Design Choices

Asymmetric LLM roles — small model for diversity, large model for synthesis

Disagreement-driven acquisition — selects experiments that separate competing laws

Bootstrap confidence gate — filters brittle hypotheses under adaptive data

Representation-agnostic — handles equations, graphs, and open-ended mechanisms

ActiveSciBench

A new benchmark suite for active, budget-constrained scientific discovery where the learner must jointly design experiments, identify relevant variables, and recover true mechanisms.

Figure 2. Overview of ActiveSciBench. (a) ActiveSciBench-Chem: symbolic enzyme rate law recovery with distractor variables and budget-limited oracle access. (b) ActiveSciBench-GRN: signed directed gene regulatory graph inference from perturbation-response experiments.

Physics

NewtonBench

Symbolic equation recovery across 11 physics domains including gravity, magnetic force, Fourier's law, Snell's law, radioactive decay, and harmonic oscillators. Uses counterfactual, memorization-resistant law variants with interactive oracle probing.

11 domains

Budget B=20

Equation recovery

Chemistry

ActiveSciBench-Chem

Enzyme kinetics rate law discovery across 57 domains covering Michaelis-Menten kinetics, competitive inhibition, substrate inhibition, Arrhenius temperature dependence, pH effects, and bisubstrate catalysis. Designed with distractor variables and anti-alignment to defeat classical Bayesian optimization.

57 domains

Budget B=60

Distractor variables

Biology

ActiveSciBench-GRN

Signed causal gene regulatory graph inference across 45 tasks. The learner must jointly identify topology, sign, and effective dynamics from sparse perturbation-response data. Motifs span feedforward loops, feedback, toggle circuits, and repression chains.

45 tasks

Budget B=20

Causal graph recovery

Benchmark Comparison

ActiveSciBench is the first suite requiring active exploration, interventional design, physical grounding, variable selection, and memorization-free evaluation simultaneously.

Benchmark	Active Exploration	Interventional	Physical Grounding	Variable Selection	Memorization-Free	Discovery Target
Passive law benchmarks	✗	✗	✓	✗	✗	Closed-form law
LLM equation benchmarks	✗	✗	✗	✗	✓	Out-of-distribution
NewtonBench	✓	✗	✗	✗	✓	Counterfactual law
Optimization benchmarks	✓	✓	✓	✗	✓	Condition optimization
GRN inference benchmarks	✗	✓	✓	✗	✓	GRN inference
ActiveSciBench (Ours)	✓	✓	✓	✓	✓	Active mechanism discovery

Results

LLM-AutoSciLab outperforms all baselines across three benchmarks, achieving state-of-the-art symbolic accuracy and graph recovery with substantially fewer experiments.

NewtonBench

SA = Symbolic Accuracy (%), Ex. = Exact Accuracy (%), RMSLE = Root Mean Squared Log Error. Budget B=20. Overall across Easy/Medium/Hard difficulty levels.

Method	ED	SA↑	Ex.↑	RMSLE↓
PySR	✗	24.07	74.54	0.182
Bayesian Opt.	✓	24.54	68.52	0.334
Bayesian Exp. Design	✓	11.11	63.86	0.577
LLM-only	✓	6.48	7.41	5.039
Code-assisted LLM	✓	7.41	10.19	4.912
LLM-AutoSciLab (Ours)	✓	67.60	81.50	0.150

ActiveSciBench-Chem

SA = Symbolic Accuracy (%), Ex. = Exact Accuracy (%), RMSLE = Root Mean Squared Log Error. Budget B=60. Overall across Simple/Composite domains.

Method	ED	SA↑	Ex.↑	RMSLE↓
PySR	✗	7.89	15.79	1.212
Bayesian Opt.	✓	5.26	6.98	1.960
Bayesian Exp. Design	✓	31.58	39.47	0.249
LLM-only	✓	0.00	0.00	0.764
Code-assisted LLM	✓	0.00	0.00	0.824
LLM-AutoSciLab (Ours)	✓	35.09	50.88	0.189

ActiveSciBench-GRN

F1 = Edge F1 (%), Ex. = Exact Graph Accuracy (%), Sign = Sign Accuracy (%). Budget B=20. Overall across Easy/Medium/Hard difficulty levels.

Method	ED	F1↑	Ex.↑	Sign↑
GENIE3	✗	38.60	0.00	79.07
GIES	✗	56.27	6.67	85.78
NOTEARS	✗	27.60	2.22	58.15
Random + GraphFit	✓	35.56	2.22	84.44
Uncertainty + GraphFit	✓	50.10	4.44	88.15
LLM-only	✓	50.41	0.00	79.44
Code-assisted LLM	✓	54.67	0.00	85.98
LLM-AutoSciLab (Ours)	✓	72.49	31.11	98.15

Analysis

Sample efficiency curves, ablation studies, and noise robustness experiments confirm the contribution of each component.

Sample Efficiency

Hypothesis-guided experimentation is 2–5× more sample-efficient than the strongest competing baselines, reaching equivalent performance with a fraction of the oracle calls.

Ablation Study

Removing hypothesis-conditioned acquisition, the bootstrap confidence gate, or memory each leads to a significant performance drop, confirming the necessity of each component.

Noise Robustness

NewtonBench

ActiveSciBench-Chem

ActiveSciBench-GRN

LLM-AutoSciLab remains robust under increasing oracle noise, substantially outperforming baselines even at high noise levels.

Qualitative Results — GRN Case Study

Qualitative GRN case study: activation chain motif

A qualitative case study on an activation chain GRN motif, showing how LLM-AutoSciLab progressively refines its hypothesis towards the correct signed causal graph across budget iterations.