AI for Evidence Generation · Systematic Reviews

LEADS: A Foundation Model for Human–AI Collaboration in Medical Literature Mining

Zifeng Wang, Lang Cao, Qiao Jin, Joey Chan, Nicholas Wan, Behdad Afzali, Hyun-Jin Cho, Chang-In Choi, Mehdi Emamverdi, Manjot K. Gill, Sun-Hyung Kim, Yijia Li, Yi Liu, Yiming Luo, Hanley Ong, Justin F. Rousseau, Irfan Sheikh, Jenny J. Wei, Ziyang Xu, Christopher M. Zallek, Kyungsang Kim, Yifan Peng, Zhiyong Lu, Jimeng Sun

Nature Communications 2025

PDF Journal

These slides were generated with the help of AI and may contain errors.

01Motivation

Systematic reviews synthesise evidence but demand months of expert labour

A systematic review turns thousands of studies into one answer — and can take a team the better part of a year.
General LLMs offer help, but progress has been held back by the absence of training data built for the task.
The work is not one job but three: searching, screening, and extracting data from studies.

02Method

LEADS is trained on 633,759 samples curated from real systematic reviews

LEADSInstruct is assembled from 21,335 systematic reviews, 453,625 involved citations and 27,015 trial registries.
Those sources become instructions for the three tasks a reviewer actually performs.

Data collection and task formulation for LEADSInstruct — Data collection and task formulation: real reviews and their citations become training data for study search, screening and data extraction. Fig. 1a–b.

03Result

LEADS retrieves more target studies than GPT-4o in publication and trial search

On both publication and trial search, LEADS recovers more of the target studies than GPT-4o.
The margin is widest on trial search, where general models struggle most.

Recall against the number of target studies a review must find, for publication search (left) and trial search (right). Fig. 2d–e.

04Result

LEADS achieves higher screening recall than dense retrieval and Mistral-7B

Against dense retrieval and Mistral-7B, LEADS recovers more of the target studies at every difficulty level.
The margin holds in the hardest bucket, where more than 30 studies must be found.

Recall@50 in study screening, grouped by how many target studies a review must find. Fig. 3c.

On the head-to-head recall table, GPT-4o edges LEADS at Recall@20 and Recall@100; LEADS leads at Recall@10 and Recall@50.

05Result

LEADS outperforms four frontier LLMs across all data extraction tasks

Across all four extraction tasks, LEADS leads GPT-4o, GPT-3.5, Haiku and Mistral under manual evaluation.
The largest margin is on participant statistics, the task general models handle worst.

Accuracy across the four extraction tasks under manual evaluation, against GPT-4o, GPT-3.5, Haiku and Mistral. Fig. 4a&n — Accuracy across the four extraction tasks under manual evaluation, against GPT-4o, GPT-3.5, Haiku and Mistral. Fig. 4b.

06Result

Experts using LEADS achieve higher recall and accuracy in less time

Measured where it counts — a user study with 16 clinicians and researchers across 14 institutions.
The gain shows up as human–AI collaboration: experts stay in the loop and still move faster.

0.81

screening recall
vs 0.78 without

20.8%

less time spent
on screening

0.85

extraction accuracy
vs 0.80 without

26.9%

less time spent
on extraction

07Summary

LEADS at a glance

Background	Systematic reviews are the backbone of evidence-based medicine.
Problem	They are slow, and generic LLMs lack data built for the task.
Approach	A foundation model trained on 633,759 samples from real reviews, trials and registries.
Results	Beats four frontier LLMs across six tasks; experts gain recall and save ~21–27% of their time.
Conclusion	Domain data, not model scale, is what makes literature mining work.