Seeing, listening, drawing: interferences between sensorimotor modalities in the use of a tablet musical interface - Laboratoire Interdisciplinaire des Sciences du Numérique Access content directly
Journal Articles ACM Transactions on Applied Perception Year : 2016

Seeing, listening, drawing: interferences between sensorimotor modalities in the use of a tablet musical interface

Abstract

Audio, visual, and proprioceptive actions are involved when manipulating a graphic tablet musical interface. Previous works suggested a possible dominance of the visual over the auditory modality in this situation. The main goal of the present study is to examine the interferences between these modalities in visual, audio, and audiovisual target acquisition tasks. Experiments are based on a movement replication paradigm, where a subject controls a cursor on a screen or the pitch of a synthesized sound by changing the stylus position on a covered graphic tablet. The experiments consisted of the following tasks: (1) a target acquisition task that was aimed at a visual target (reaching a cue with the cursor displayed on a screen), an audio target (reaching a reference note by changing the pitch of the sound played in headsets), or an audiovisual target, and (2) the replication of the target acquisition movement in the opposite direction. In the return phase, visual and audio feedback were suppressed. Different gain factors perturbed the relationships among the stylus movements, visual cursor movements, and audio pitch movements. The deviations between acquisition and return movements were analyzed. The results showed that hand amplitudes varied in accordance with visual, audio, and audiovisual perturbed gains, showing a larger effect for the visual modality. This indicates that visual, audio, and audiovisual actions interfered with the motor modality and confirms the spatial representation of pitch reported in previous studies. In the audiovisual situation, vision dominated over audition, as the latter had no significant influence on motor movement. Consequently, visual feedback is helpful for musical targeting of pitch on a graphic tablet, at least during the learning phase of the instrument. This result is linked to the underlying spatial organization of pitch perception. Finally, this work brings a complementary approach to previous studies showing that audition may dominate over vision for other aspects of musical sound (e.g., timing, rhythm, and timbre).
Fichier principal
Vignette du fichier
TAP-2016-a10-Perrotin-dAlessandro.pdf (661.23 Ko) Télécharger le fichier
Origin : Publication funded by an institution
Loading...

Dates and versions

hal-01672241 , version 1 (23-01-2018)

Identifiers

Cite

Olivier Perrotin, Christophe d'Alessandro. Seeing, listening, drawing: interferences between sensorimotor modalities in the use of a tablet musical interface. ACM Transactions on Applied Perception, 2016, 14 (2), pp.1 - 19. ⟨10.1145/2990501⟩. ⟨hal-01672241⟩
118 View
120 Download

Altmetric

Share

Gmail Facebook X LinkedIn More