1
Unrealistic targets
Choosing from supplied options differs from producing an answer in clinical practice.
Evaluate realistic outputs
AI for Evidence-Based Medicine
AI for Evidence Generation
How can AI help produce the evidence at each tier of the pyramid?
AI for Evidence Utilization
How can AI fully utilize existing evidence?
AI for Evidence Generation and Utilization
How can AI generate — and make use of — medical evidence?
Systematic Reviews
Clinical Trials
Case Reports
Retrieve
Summarize
Verify
Agentic orchestration
Medical AI Evaluation
How do we know when medical AI is good enough to trust?
routinely measured
rarely measured
Medical AI Evaluation
MCQ accuracy leaves two evaluation gaps
2
Unassessed process
Even a correct result on a realistic task can hide a flawed process.
Correct outcome ≠ trustworthy process
Realistic targets address the first gap. Process evaluation addresses the second.