TY - JOUR
T1 - A multimodal system for gesture recognition in interactive music performance
AU - Overholt, Dan
AU - Thompson, John
AU - Putnam, Lance
AU - Bell, Bo
AU - Kleban, Jim
AU - Sturm, Bob
AU - Kuchera-Morin, Jo Ann
PY - 2009/12
Y1 - 2009/12
N2 - A multimodal system was presented for gesture recognition in untethered interactive flute performance. A performer's discrete cues were identified remotely, using computer vision, audio analysis, and electric-field sensing. Continuous expressive gestures were captured in musical performance. Cues and gestures that were typical among performers were used to allow the performer to naturally communicate with an interactive music system in the same way that they communicated with another performer. The system featured custom-designed electronics and software that performed real-time spectral transformation of audio from the flute. The approach also used non-contact sensors, microphones, cameras, and electric-field sensors embedded in a music stand, called the Multimodal Music Stand System (MMSS). The multimodal array of untethered sensors contained within the music stand, providing data to an analysis system that identified a set of predetermined gestures as discrete cues.
AB - A multimodal system was presented for gesture recognition in untethered interactive flute performance. A performer's discrete cues were identified remotely, using computer vision, audio analysis, and electric-field sensing. Continuous expressive gestures were captured in musical performance. Cues and gestures that were typical among performers were used to allow the performer to naturally communicate with an interactive music system in the same way that they communicated with another performer. The system featured custom-designed electronics and software that performed real-time spectral transformation of audio from the flute. The approach also used non-contact sensors, microphones, cameras, and electric-field sensors embedded in a music stand, called the Multimodal Music Stand System (MMSS). The multimodal array of untethered sensors contained within the music stand, providing data to an analysis system that identified a set of predetermined gestures as discrete cues.
UR - http://www.scopus.com/inward/record.url?scp=73649119197&partnerID=8YFLogxK
U2 - 10.1162/comj.2009.33.4.69
DO - 10.1162/comj.2009.33.4.69
M3 - Article
AN - SCOPUS:73649119197
SN - 0148-9267
VL - 33
SP - 69
EP - 82
JO - Computer Music Journal
JF - Computer Music Journal
IS - 4
ER -