AIM — Artificial Intelligence in Medicine — In Vorbereitung
DEMNE: Determination of Extraction Method for Named Entity
Longeac C. et al.
Manuskript zum DEMNE-Kaskadenframework, den fünf Korpus-Charakterisierungsmetriken zur Paradigmenauswahl und den Kalibrierungsergebnissen der Grid-Search über drei französische Onkologie-Korpora.
Results, at a glance
DEMNE learns, per entity, whether to route it to rules, a lightweight model, or an LLM. Three views: how well it generalises from one clinical domain to another, its per-corpus performance in oncology, and its margin over fixed-tier strategies.
Figure 3
Generalisation across clinical domains
lowhigh
Each row is a configuration optimised on one domain; each column, the domain it is evaluated on. Darker cells mean higher routing concordance. The diagonal dominates — every configuration is strongest on its own domain — yet the off-diagonal cells stay high: the learned routing rules transfer from one clinical corpus to another.
Table 5
Per-corpus performance — oncology
Corpus
Concordance
Cost-score
Cantemist
1.000
1.000
ESMO 2025
1.000
1.000
Oncolog-IA
0.863
0.842
Global (in-domain)
0.954
0.947
Cross-validation (LOCO)
0.866
0.837
Protocol B, oncology domain. Concordance = share of entities routed to the optimal tier; cost-score = quality weighted by compute cost. On Cantemist and ESMO 2025 routing is perfect; under leave-one-corpus-out cross-validation (LOCO) it holds at 0.866.
Baselines
Why adaptive routing
Two naive strategies: route everything to rules, or everything to the LLM. Rules alone are fast but miss the hard cases (0.729); the LLM alone is costly and surprisingly low-concordance (0.148). DEMNE's adaptive routing reaches 0.954 concordance at a 0.947 cost-score.