Automatic Segmentation of Sign Language into Subtitle-Units - Laboratoire Interdisciplinaire des Sciences du Numérique Access content directly
Conference Papers Year : 2020

Automatic Segmentation of Sign Language into Subtitle-Units

Abstract

We present baseline results for a new task of automatic segmentation of Sign Language video into sentence-like units. We use a corpus of natural Sign Language video with accurately aligned subtitles to train a spatio-temporal graph convolutional network with a BiLSTM on 2D skeleton data to automatically detect the temporal boundaries of subtitles. In doing so, we segment Sign Language video into subtitle-units that can be translated into phrases in a written language. We achieve a ROC-AUC statistic of 0.87 at the frame level and 92% label accuracy within a time margin of 0.6s of the true labels.
Fichier principal
Vignette du fichier
nf7881.pdf (2.49 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03098684 , version 1 (05-01-2021)

Identifiers

Cite

Hannah Bull, Michèle Gouiffès, Annelies Braffort. Automatic Segmentation of Sign Language into Subtitle-Units. Sign Language Recognition, Translation & Production workshop, Aug 2020, Glasgow ( virtual ), United Kingdom. ⟨10.1007/978-3-030-66096-3_14⟩. ⟨hal-03098684⟩
150 View
301 Download

Altmetric

Share

Gmail Facebook X LinkedIn More