LADER
AI for Evidence Utilization · Retrieve

LADER: Log-Augmented DEnse Retrieval for Biomedical Literature Search

Qiao Jin, Andrew Shin, Zhiyong Lu
SIGIR 2023
PDF Proceedings
These slides were generated with the help of AI and may contain errors.
01Motivation

Click logs are a free relevance signal retrievers ignore

The cost of dense retrieval
  • Dense retrievers match by meaning — but need costly labeled pairs to fine-tune.
The unused signal
  • Search engines log exactly which article users click for each query.
  • That relevance signal sits unused.
02Method

Borrow what past users clicked, at inference time

  • For a new query, find similar past queries and aggregate their clicked documents.
  • Blend that with the retriever’s own scores — no extra training.
An incoming query borrows the documents previous users clicked for similar past queries.
An incoming query borrows the documents previous users clicked for similar past queries. Fig. 1.
03Method

A plug-in over any dense retriever

  • The query embedding is matched to documents via maximum inner-product search.
  • A parallel branch aggregates click-log evidence; the two scores are combined.
The architecture: dense similarity blended with click-log evidence.
The architecture: dense similarity blended with click-log evidence. Fig. 2.
04Result

New state of the art on TripClick

  • Measured on TripClick, a benchmark of 5.2M real health-search click logs.
  • Full LADER leads on HEAD, TORSO, and TAIL query slices.
ModelHEADTORSOTAIL
BM250.1400.2060.267
ConvKNRM0.1980.2430.271
TK (best baseline)0.2080.2720.295
LADER w/o log-aug0.2470.2410.260
LADER w/o dense0.3240.2660.232
LADER (full)0.3380.3030.310
NDCG@10 on TripClick by query frequency. Source: Table 1.
05Result

Gains transfer from head to tail queries

  • Retrieval quality rises with log size across all query slices.
  • Head queries gain most; torso and tail improve too.
NDCG@10 vs. the fraction of the 685k-query log used, for HEAD, TORSO, and TAIL queries. Source: Fig.
NDCG@10 vs. the fraction of the 685k-query log used, for HEAD, TORSO, and TAIL queries. Source: Fig. 3.
06Summary

LADER at a glance

BackgroundNeural retrievers need relevance labels that biomedical search largely lacks.
ProblemYet users’ search click logs are a free relevance signal retrievers ignore.
ApproachBorrow documents past users clicked for similar queries — no training.
ResultsNew state of the art on TripClick, across head, torso & tail queries.
ConclusionClick logs alone rival trained retrievers — a precursor to the log-driven training behind our MedCPT.