Dancing on a piano with projection mapping
Xiao Xiao from MIT Media Lab has a beautiful series of experiments fusing piano playing with projection mapping. Shown above, Andante, uses projection mapping and a player piano to create a compelling illusion where virtual characters create music by dancing on a piano. The thesis of the work explores the connection between the rhythm of walking and the rhythm of music.
“BASED ON A VIEW OF MUSIC PEDAGOGY THAT EMPHASIZES EXPRESSIVE, FULL-BODY COMMUNICATION EARLY IN THE LEARNING PROCESS, ANDANTE PROMOTES AN UNDERSTANDING OF THE MUSIC ROOTED IN THE BODY, TAKING ADVANTAGE OF WALKING AS ONE OF THE MOST FUNDAMENTAL HUMAN RHYTHMS.”
MirrorFugue (above) is the ultimate player piano. After recording a performer playing the piano, a projector displays the performer’s hands on the piano keys and projects the performer onto a screen. The end result is a player piano that can record and replay duets between virtual and real world performers.
“THE PIANIST IS BUT AN ILLUSION OF LIGHT AND MIRRORS, A GHOST AT ONCE PRESENT AND ABSENT”.
Perpetual Canon (above) shoots circles up in the air as each key is depressed, in a mesmerizing visualization. What is unique about Perpetual Canon compared to other piano projection mapping visualizations, is that when the circles fall back down the piano repeats the note. The piano is thus augmented into a completely different playing experience.
“INSPIRED BY THE INTRICATE CONTRAPUNTAL MUSIC OF J.S. BACH, I WANTED TO CREATE A MACHINE THAT TRANSFORMS ANY PLAYING INTO A CANON. EACH NOTE PLAYED BY A PIANIST SHOOTS UP AS A DIGITAL “CANON BALL” AND ECHOES THE ORIGINAL NOTE WITH THE SAME INTENSITY AS IT FALLS BACK DOWN INTO THE KEY.”
This work was done in the Tangible Media Group led by Hiroshi Ishii.You can see Xiao Xiao’s full thesis presentation below.
“BASED ON A VIEW OF MUSIC PEDAGOGY THAT EMPHASIZES EXPRESSIVE, FULL-BODY COMMUNICATION EARLY IN THE LEARNING PROCESS, ANDANTE PROMOTES AN UNDERSTANDING OF THE MUSIC ROOTED IN THE BODY, TAKING ADVANTAGE OF WALKING AS ONE OF THE MOST FUNDAMENTAL HUMAN RHYTHMS.”
“THE PIANIST IS BUT AN ILLUSION OF LIGHT AND MIRRORS, A GHOST AT ONCE PRESENT AND ABSENT”.
“INSPIRED BY THE INTRICATE CONTRAPUNTAL MUSIC OF J.S. BACH, I WANTED TO CREATE A MACHINE THAT TRANSFORMS ANY PLAYING INTO A CANON. EACH NOTE PLAYED BY A PIANIST SHOOTS UP AS A DIGITAL “CANON BALL” AND ECHOES THE ORIGINAL NOTE WITH THE SAME INTENSITY AS IT FALLS BACK DOWN INTO THE KEY.”
“THIS THESIS INTRODUCES AND EXAMINES METHODS FOR THE CAPTURE AND REPRODUCTION OF MUSIC ON THE PIANO THAT MAINTAINS A TIGHT COUPLING BETWEEN THE BODY AND THE PHYSICAL INSTRUMENT. MUSIC PLAYING ORIGINATES IN THE BODY, WHICH PLAYS A CENTRAL ROLE IN BOTH PERFORMANCE AND PEDAGOGY. HOWEVER, TECHNOLOGY FOR THE CAPTURE AND REPRODUCTION OF MUSIC HAS LARGELY IGNORED THE INTRICATE RELATIONSHIP BETWEEN THE SOUND, THE BODY AND THE INSTRUMENT IN THE MAKING AND APPRECIATION OF MUSIC.”
Follow us on Facebook
