On 22 November 2017, a group of people descended on a concert hall in Tokyo. Staff from Yahama, a representative from the Tokyo University of the Arts, as well as world-renowned dancer Kaiji Moriyama, were there. Also in attendance was the Berlin Philharmonic Orchestra Scharoun Ensemble. The centrepiece of the stage was a Yamaha CFX Concert Grand Piano, worth in excess of USD150k. The event was special and was like none other. In this concert, named Mai Hi Ten Yu, instead of having a pianist play at the piano, Moriyama would dance on stage and have his movements translated into musical notes on the instrument.
This was a concert for the public as much as it was an experiment for Yamaha in the field of artificial intelligence. For years, scientists have been exploring AI, creating newer and better machines to mimic human thinking in the hope of breaking new ground that will make life more convenient for humans. Today, artificial intelligence has made its foray into the world of music. What is interesting is that the musical instrument is not a digital keyboard that would presumably be easy to hook up to a robot. Rather, this is a classical, acoustic piano that produces sound with hammers - more specificially, a Yamaha Disklavier player piano, in this performance.
On the day of the concert, engineers from Yahama hooked up four types of sensors onto Moriyama’s body. They attached the sensors onto the dancer’s back, wrists and feet. At the other end of the wires was a wireless transmitter attached to his belt. As Moriyama writhed and twisted his agile body on the stage, his movements were recorded by the sensors and transmitted as wireless data signals to an AI receiver in the control room. The AI robot instantaneously selected suitable melodies based on a pre-loaded database of MIDI music. The resulting notes were then sent to the CFX Concert Grand, which automatically hit the hammers that produced music. Accompanying the sounds was the music from the Berlin-based orchestra that shared the stage with Moriyama.
While most observers would assume that Moriyama was only in charge of the beat, it turned out that the dancer was also able to create different tunes of the music, by subtly adjusting the balance of his body. This certainly is interesting from the perspective of science and artificial intelligence. Given what Yamaha and Moriyama achieved on that night in November 2017, one can certainly imagine that the realms of AI and music creation are limitless! As Yamaha put it, “this performance represents steady progress in the pursuit of new forms of artistic expression.”
