i-MedRAG
AI for Evidence Utilization · Summarize

Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions

Guangzhi Xiong*, Qiao Jin*, Xiao Wang, Minjia Zhang, Zhiyong Lu, Aidong Zhang
* equal contribution
PSB 2025
PDF Proceedings
These slides were generated with the help of AI and may contain errors.
01Motivation

One-shot retrieval fails on multi-hop questions

  • Standard RAG retrieves once, using the original question.
  • It fails when the first query returns nothing useful.
  • Or when the answer needs several facts chained together that no single query can surface.
02Method

i-MedRAG = MedRAG + iterative follow-up questions

  • Where MedRAG retrieves once, i-MedRAG runs several rounds.
  • Each round auto-generates follow-up questions from what it has found so far.
  • The growing information-seeking history supports multi-hop reasoning.
MedRAG (one round) vs. i-MedRAG (iterative follow-ups): each round asks new questions from what it h
MedRAG (one round) vs. i-MedRAG (iterative follow-ups): each round asks new questions from what it has found. Fig. 2.
03Result

Agentic search reaches the zero-shot frontier

  • Plotted against prior methods over time, zero-shot i-MedRAG lands at the top on MedQA.
  • The gain comes purely from iterative search — with no additional training.
Progress on MedQA over time: iterative, agentic search (“Ours”) reaches the top of the z
Progress on MedQA over time: iterative, agentic search (“Ours”) reaches the top of the zero-shot frontier. Fig. 1.
04Result

69.7% on MedQA, zero-shot — beating fine-tuned methods

MethodKnowledgeSettingAccuracy
Chain of Thought–zero-shot50.8%
Knowledge Solveryeszero-shot58.4%
CoT + Self-Consistency–zero-shot61.3%
MedAgents–zero-shot64.1%
LLM + Medical Textbookyeszero-shot65.0%
MedRAGyeszero-shot66.6%
MedAdapter–fine-tuned68.7%
i-MedRAG (ours)yeszero-shot69.7%
MedQA accuracy with GPT-3.5. i-MedRAG leads all prompt-engineering methods and beats fine-tuned baselines. Source: Table 1.
05Result

Beats MedRAG and chain-of-thought, across models

  • The gains hold across backbones — Llama-3.1-8B, not just GPT-3.5.
  • i-MedRAG (colored) stays above chain-of-thought and MedRAG (grey), most clearly on MedQA.
Accuracy vs. iterations on MedQA and MMLU-Med, for GPT-3.5 (top) and Llama-3.1-8B (bottom). Source:
Accuracy vs. iterations on MedQA and MMLU-Med, for GPT-3.5 (top) and Llama-3.1-8B (bottom). Source: Fig. 3.
06Impact

Adopted as an agentic-retrieval method

  • Reused as a component — SEMA-RAG (ACL) builds its multi-round retrieval on i-MedRAG.
  • A standard zero-shot baseline in new medical-RAG methods — RGAR, RAR² (EMNLP’25) & WWW’25’s From Retrieval to Generation.
  • Shipped as the agentic mode of the MedRAG toolkit (570+ stars).
  • Motivated our own RAG-Gym, which trains search agents with process supervision.
07Summary

i-MedRAG at a glance

BackgroundMany clinical questions need several linked facts, not one lookup.
ProblemOne-shot retrieval fails on these multi-hop questions.
ApproachMedRAG plus iterative, auto-generated follow-up questions.
Results69.7% zero-shot on MedQA — beating fine-tuned baselines.