Empirical Study of Diachronic Word Embeddings for Scarce Data - Laboratoire Interdisciplinaire des Sciences du Numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2019

Empirical Study of Diachronic Word Embeddings for Scarce Data

Résumé

Word meaning change can be inferred from drifts of time-varying word embeddings. However, temporal data may be too sparse to build robust word embeddings and to discriminate significant drifts from noise. In this paper, we compare three models to learn diachronic word embeddings on scarce data: incremental updating of a Skip-Gram from Kim et al. (2014), dynamic filtering from Bamler and Mandt (2017), and dynamic Bernoulli embeddings from Rudolph and Blei (2018). In particular, we study the performance of different initialisation schemes and emphasise what characteristics of each model are more suitable to data scarcity, relying on the distribution of detected drifts. Finally, we regularise the loss of these models to better adapt to scarce data.
Fichier principal
Vignette du fichier
R19-1092.pdf (392.61 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte
licence
Copyright (Tous droits réservés)

Dates et versions

hal-04421484 , version 1 (27-01-2024)

Licence

Copyright (Tous droits réservés)

Identifiants

Citer

Syrielle Montariol, Alexandre Allauzen. Empirical Study of Diachronic Word Embeddings for Scarce Data. Recent Advances in Natural Language Processing, Sep 2019, Varna, Bulgaria. pp.795-803, ⟨10.26615/978-954-452-056-4_092⟩. ⟨hal-04421484⟩
8 Consultations
11 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More