Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case - Laboratoire Interdisciplinaire des Sciences du Numérique Access content directly
Conference Papers Year : 2023

Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case

Abstract

In order to foster inclusive education, automatic systems that can adapt textbooks to make them accessible to children with Developmental Coordination Disorder (DCD) are necessary. In this context, we propose a task to classify exercises according to their DCD adaptation type. We introduce a challenging exercise dataset extracted from French textbooks, with two major difficulties: limited and unbalanced, noisy data. To set a baseline on the dataset, we use state-of-the-art models combined through early and late fusion techniques to take advantage of text and vision/layout modalities. Our approach achieves an overall accuracy of 0.802. However, the experiments show the difficulty of the task, especially for minority classes, where the accuracy drops to 0.583.
Fichier principal
Vignette du fichier
Lincker_Noisy-and-Unbalanced-Multimodal-Document-Classification.pdf (850.58 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
licence : CC BY - Attribution

Dates and versions

hal-04221023 , version 1 (28-09-2023)

Licence

Attribution

Identifiers

Cite

Élise Lincker, Camille Guinaudeau, Olivier Pons, Jérôme Dupire, Céline Hudelot, et al.. Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case. 20th International Conference on Content-based Multimedia Indexing (CBMI 2023), Sep 2023, Orléans, France. ⟨10.1145/3617233.3617239⟩. ⟨hal-04221023⟩
114 View
97 Download

Altmetric

Share

Gmail Facebook X LinkedIn More