AI for Evidence Utilization · Summarize

MedCite: Can Language Models Generate Verifiable Text for Medicine?

Xiao Wang, Mengjue Tan, Qiao Jin, Guangzhi Xiong, Yu Hu, Aidong Zhang, Zhiyong Lu, Minjia Zhang

ACL Findings 2025

These slides were generated with the help of AI and may contain errors.

01Motivation

AI medical citations: can we trust them?

When a medical AI cites, one question decides trust: does the cited article really support the statement?
Existing systems answer without citations, or attach references that may not back the claim.

02Method

A citation seeker re-retrieves and reranks the sources behind each generated statement.
An attribution judge then checks whether each cited article actually supports its claim.

03Result

Method	Accuracy	MAUVE	ROUGE-L	Cite recall	Cite prec.
Non-RAG	71.4	53.2	18.1	59.1	52.9
RAG	82.9	52.2	14.8	49.0	42.8
RAG w. Oracle	94.3	63.5	20.6	57.5	43.2

Grounding the answer in retrieved evidence (and an oracle upper bound) raises answer quality. Source: Table 2.

04Result

Citation quality by attribution strategy (answer accuracy 83.3 for all). Re-retrieval + LLM reranking gives the best precision. Source: Table 3.

05Result

Configuration	Accuracy	MAUVE	ROUGE-L	Cite recall	Cite prec.
Non-parametric (single-pass)	82.9	52.2	14.8	49.0	42.8
Hybrid (multi-pass)	83.3	59.2	16.8	65.7	47.7

The multi-pass hybrid design lifts citation recall and precision at comparable answer quality. Source: Table 4.

06Summary

Background	Medical AI answers increasingly arrive with supporting citations.
Problem	But can we trust them — do the citations actually support the claim?
Approach	End-to-end citation generation plus an automatic attribution judge.
Results	Re-retrieval with LLM reranking gives the best citation precision; the judge matches expert annotators.
Conclusion	A benchmark + metric for the “verify” step of trustworthy medical AI.