AI for Evidence Utilization · Retrieve

MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs

Qiao Jin, Won Kim, Qingyu Chen, Donald C. Comeau, Lana Yeganova, John Wilbur, Zhiyong Lu

Bioinformatics 2023

PDF Journal

These slides were generated with the help of AI and may contain errors.

01Motivation

Matching terms, not their meaning

PubMed’s Best Match learns to rank from 100+ features — but none capture meaning.
Search “lead heart damage” — meaning the metal — and the top hits use “lead” as a verb instead.

02Method

MedCPT is a family of three contrastively pre-trained encoders.
A query encoder and an article encoder retrieve; a cross-encoder re-ranks the top hits.

03Method

Supervised by 255M real user clicks from PubMed search logs.
Contrastive learning pulls relevant pairs together in embedding space.
The cross-encoder re-ranker is then trained on hard negatives from the retriever.

04Result

Evaluated zero-shot on the biomedical subtasks of BEIR.
MedCPT (330M) beats general retrievers up to ~15× larger — including Google’s 4.8B GTR-XXL.

Retriever	Size	TREC-COVID	NFCorpus	BioASQ	SciFact	Avg.
BM25	—	0.656	0.325	0.465	0.665	0.454
docT5query	220M	0.713	0.328	0.431	0.675	0.462
ColBERT	110M	0.677	0.305	0.474	0.671	0.454
Google GTR-XXL	4.8B	0.501	0.342	0.324	0.662	0.398
MedCPT	330M	0.709	0.355	0.553	0.761	0.510

Zero-shot nDCG@10 on biomedical BEIR subtasks. Source: Table 1.

05Result

RELISH scores how well a model finds articles related to a given paper.
MedCPT’s document encoder tops every baseline — including PubMed’s own PMRA related-articles algorithm.

Method	MAP@10	NDCG@10	Avg.
BM25	86.72	87.39	87.21
PMRA	87.57	88.40	88.40
BioSentVec	88.10	87.76	88.29
PubMedBERT	81.07	82.39	82.26
SciNCL	92.74	91.91	92.52
MedCPT (DEnc)	93.99	93.12	93.72

Article-similarity re-ranking on RELISH (%). MedCPT’s document encoder leads. Source: Table 2.

06Result

MedCPT’s embeddings capture sentence-level meaning, not just document relevance.
Its query encoder best matches human similarity judgments on BIOSSES and MedSTS.

Pearson correlation with human sentence-similarity judgments (BIOSSES, MedSTS). MedCPT’s query encoder leads on both. Source: Table 3.

07Impact

Over 8M downloads on Hugging Face — a de-facto default biomedical retriever.
Its cross-encoder re-ranks candidate cases in DeepRare, the Nature rare-disease diagnosis agent (2025).
Others adopt it off-the-shelf as their retriever — e.g. Self-BioRAG, OLAPH and AAAI’s EMS-QA.
The most-used retriever across surveyed medical-RAG systems (Yale survey, 2025).
Powers NCBI LitSense 2.0 — semantic search over 38M abstracts, serving millions of requests.
The embedding model behind much of our own work — TrialGPT, MedRAG, i-MedRAG, MedCite, Med-V1, GeneAgent, AgentMD, and BriefContext.

08Summary

Background	Biomedical search should match meaning, not just keywords.
Problem	Lexical search (PubMed Best Match) misses semantically related papers.
Approach	Three encoders — retrieve, then re-rank — trained on 255M PubMed click pairs.
Results	State-of-the-art zero-shot biomedical retrieval & sentence similarity.
Impact	8M+ downloads; powers NCBI LitSense 2.0; the most-used retriever in medical RAG.