AIM — Artificial Intelligence in Medicine — Dalam persiapan
DEMNE: Determination of Extraction Method for Named Entity
Longeac C. dkk.
Naskah yang merinci kerangka kerja cascade DEMNE, lima metrik karakterisasi korpus yang digunakan untuk pemilihan paradigma, dan hasil kalibrasi grid search pada tiga korpus onkologi Prancis.
Results, at a glance
DEMNE learns, per entity, whether to route it to rules, a lightweight model, or an LLM. Three views: how well it generalises from one clinical domain to another, its per-corpus performance in oncology, and its margin over fixed-tier strategies.
Figure 3
Generalisation across clinical domains
lowhigh
Each row is a configuration optimised on one domain; each column, the domain it is evaluated on. Darker cells mean higher routing concordance. The diagonal dominates — every configuration is strongest on its own domain — yet the off-diagonal cells stay high: the learned routing rules transfer from one clinical corpus to another.
Table 5
Per-corpus performance — oncology
Corpus
Concordance
Cost-score
Cantemist
1.000
1.000
ESMO 2025
1.000
1.000
Oncolog-IA
0.863
0.842
Global (in-domain)
0.954
0.947
Cross-validation (LOCO)
0.866
0.837
Protocol B, oncology domain. Concordance = share of entities routed to the optimal tier; cost-score = quality weighted by compute cost. On Cantemist and ESMO 2025 routing is perfect; under leave-one-corpus-out cross-validation (LOCO) it holds at 0.866.
Baselines
Why adaptive routing
Two naive strategies: route everything to rules, or everything to the LLM. Rules alone are fast but miss the hard cases (0.729); the LLM alone is costly and surprisingly low-concordance (0.148). DEMNE's adaptive routing reaches 0.954 concordance at a 0.947 cost-score.