A Benchmark Evaluation of Clinical Named Entity Recognition in French - Laboratoire Interdisciplinaire des Sciences du Numérique Access content directly
Conference Papers Year : 2024

A Benchmark Evaluation of Clinical Named Entity Recognition in French


Background: Transformer-based language models have shown strong performance on many Natural Language Processing (NLP) tasks. Masked Language Models (MLMs) attract sustained interest because they can be adapted to different languages and sub-domains through training or fine-tuning on specific corpora while remaining lighter than modern Large Language Models (LLMs). Recently, several MLMs have been released for the biomedical domain in French, and experiments suggest that they outperform standard French counterparts. However, no systematic evaluation comparing all models on the same corpora is available. Objective: This paper presents an evaluation of masked language models for biomedical French on the task of clinical named entity recognition. Material and methods: We evaluate biomedical models CamemBERT-bio and DrBERT and compare them to standard French models CamemBERT, FlauBERT and FrALBERT as well as multilingual mBERT using three publically available corpora for clinical named entity recognition in French. The evaluation set-up relies on gold-standard corpora as released by the corpus developers. Results: Results suggest that CamemBERT-bio outperforms DrBERT consistently while FlauBERT offers competitive performance and FrAlBERT achieves the lowest carbon footprint. Conclusion: This is the first benchmark evaluation of biomedical masked language models for French clinical entity recognition that compares model performance consistently on nested entity recognition using metrics covering performance and environmental impact.
Fichier principal
Vignette du fichier
final.pdf (178.32 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04523267 , version 1 (27-03-2024)


Public Domain


  • HAL Id : hal-04523267 , version 1


Nesrine Bannour, Christophe Servan, Aurélie Névéol, Xavier Tannier. A Benchmark Evaluation of Clinical Named Entity Recognition in French. The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), May 2024, Torino, Italy. ⟨hal-04523267⟩
33 View
35 Download


Gmail Facebook X LinkedIn More