Perception and performance of human body motion in synchrony with music

Perception and performance of human body motion in synchrony with music

This work received the award IGSA by CRBLM (Center for Research on Brain, Language and Music). It’s a collaboration between Ramona Kaiser (PhD Candidate, psychologist) and myself.

::: abstract :::

Synchronization between people is a crucial non-verbal communication skill in human, specially for skilled motor performers as athletes, musicians and dancers. Very little is known about the cognitive and neural mechanisms that allow us to perceive, predict and synchronize with other’s movements. Music is an ideal stimulus to study this question, because it is an ecologically valid real-world stimulus. It strongly evokes natural body motion, and there is substantioal research on social and communicative music aspects (Godoy, R.I and Leman, M. (Org), 2010, Routledge).

However, several critical questions remain unanswered: Which auditory cues underlie the ability to synchronize with music? How do humans perceive the synchrony between somebody’s motion and music? Our work aims at understanding, firstly, how people move in synchrony with sounds and music; and secondly, how people perceive the synchrony between the body movements of other people and auditory events.

The combination of infrared-based motion capture with inertial sensing helps analyzing how people move in synchrony with sounds. From these recordings, we can generate and modify displays of these body movements to study the effects of specific motion cues on the perception of synchrony and cross-modal binding.

Our project thus involves two lines of inquiry, one being development of new methods for motion analysis, the other being the evaluation of synchrony perception. That is, a collaboration between engineering and psychology.

Video credits
subject: signed permission for disclosure | editing: Carolina Brum Medeiros, under CC license