PMC-Patients
AI for Evidence Generation · Case Reports

A Large-scale Dataset of Patient Summaries for Retrieval-based Clinical Decision Support

Zhengyun Zhao*, Qiao Jin*, Fangyuan Chen, Tuorui Peng, Sheng Yu
* equal contribution
Scientific Data 2023
PDF Journal
These slides were generated with the help of AI and may contain errors.
01Motivation

Case reports are rich evidence machines can’t search

What case reports capture
  • They sit near the base of the evidence pyramid.
  • Yet they record individual patients, rare diseases, and unusual presentations the higher tiers miss.
Why they’re unusable
  • The evidence is scattered, unstructured narrative.
  • Making it searchable needs a large, structured, openly licensed collection — which did not exist.
02Method

167k patient summaries, mined at scale

  • 167k patient summaries extracted from PubMed Central case reports.
  • 3.1M patient–article and 293k patient–patient links, mined from citation signals.
Each patient summary is linked to relevant PubMed articles and to similar patients.
Each patient summary is linked to relevant PubMed articles and to similar patients. Fig. 1.
03Method

A repeatable extraction pipeline

  • Patient summaries identified from PMC Open Access case reports via extraction triggers.
  • Filtered by length, language, and demographics.
  • Yields 167k patient-note candidates.
The collection pipeline.
The collection pipeline. Fig. 3.
04Benchmark

Two retrieval tasks for decision support

  • Patient-to-article (PAR) — find the literature relevant to a patient.
  • Patient-to-patient (PPR) — find similar patients.
  • Both scored against the mined relevance annotations.
The ReCDS benchmark: two retrieval tasks over PMC-Patients.
The ReCDS benchmark: two retrieval tasks over PMC-Patients. Fig. 2.
05Result

Large, open — with clear headroom

  • An order of magnitude beyond earlier case-report collections.
  • Released with a public leaderboard; strong retrievers still leave much signal untapped.
RetrieverPAR · MRRPPR · MRR
BM25 (sparse)18.722.9
MedCPT13.113.7
PubMedBERT19.819.4
BioLinkBERT19.121.2
SPECTER17.915.1
RRF (fusion)29.927.8
Retrieval quality (MRR×100) on patient-to-article (PAR) and patient-to-patient (PPR). Source: Table 4.
06Impact

A reference corpus across top-tier medical AI

07Summary

PMC-Patients at a glance

BackgroundMillions of published case reports describe real patients and their care.
ProblemThat case-report evidence is rich but unstructured and unsearchable.
ApproachMined 167k patient summaries + 3.1M patient–article links from PMC.
ResultsThe ReCDS benchmark for patient-to-article & patient-to-patient retrieval.
ImpactA reference corpus — from Nature’s DeepRare to our MedCalc-Bench.