GeneAgent
AI for Evidence Utilization · Verify

GeneAgent: self-verification language agent for gene-set analysis using domain databases

Zhizheng Wang*, Qiao Jin*, Chih-Hsuan Wei, Shubo Tian, Po-Ting Lai, Qingqing Zhu, Chi-Ping Day, Christina Ross, Robert Leaman, Zhiyong Lu
* equal contribution
Nature Methods 2025
PDF Journal
These slides were generated with the help of AI and may contain errors.
01Motivation

Gene-set analysis is where LLMs hallucinate

  • Naming the biological process a set of genes shares is routine but high-stakes.
  • Exactly the open-ended task where LLMs confidently name a function the evidence can’t support.
02Method

An agent that verifies itself against databases

  • Drafts an answer, then loops through self-verification and modification.
  • Autonomously queries curated databases (GO, KEGG, Reactome, HPO, WikiPathways, MSigDB).
  • Confirms or rejects each claim before finalizing.
The GeneAgent self-verification loop.
The GeneAgent self-verification loop. Fig. 1.
03Setup

1,106 gene sets from diverse sources

DatasetSetsGenes / setAvg. genesResourceReleased
GO1,0003–45648.3Literal curationNov 2023
NeST505–32319.0ProteomicsApr 2024
MSigDB564–200112.0Molecular functionMay 2023
All1,1063–45650.7——
Gene sets for the systematic benchmark, plus seven novel mouse-melanoma sets for the real-world case study. Source: Table 1.
04Result

GeneAgent names biological processes more accurately than GPT-4

  • Compared against the strong GPT-4 gene-set baseline of Hu et al., 2025.
  • Beats GPT-4 on GO, NeST, and MSigDB.
GeneAgentGPT-4GO · GeneAgent: 0.710.71GO · GPT-4: 0.690.69GONeST · GeneAgent: 0.760.76NeST · GPT-4: 0.710.71NeSTMSigDB · GeneAgent: 0.740.74MSigDB · GPT-4: 0.720.72MSigDBName similarity (MedCPT)
MedCPT semantic similarity of generated names to ground truth. Source: Fig. 2b.
05Result

92% of self-verification decisions are correct

SupportPartialRefuteUnknown
Correct801427192.4%
Partially correct60105.3%
Incorrect21002.3%
GeneAgent’s decisionAccuracy
Manual audit of GeneAgent’s accept/reject calls against human judgment (cell counts). 92.4% of decisions agree with experts. Source: Fig. 3e.
06Recognition

Highlighted by NIH

nih.gov News Releases · July 28, 2025.
nih.gov News Releases · July 28, 2025.
07Impact

Adopted for real gene-set analysis

08Summary

GeneAgent at a glance

BackgroundInterpreting a gene set means naming the biological process its genes share.
ProblemGene-set analysis is exactly where LLMs hallucinate plausible-but-wrong functions.
ApproachAn LLM agent that self-verifies each claim against curated biology databases.
ResultsBeats GPT-4 on 1,106 gene sets; 92% of its verification calls agree with experts.