"Flow Following" not "Score Following"

I just put up a YouTube video from one of the demonstrations I did in Berlin recently.  It's basically an "automatic accompanist" thing, which, musically doesn't interest me at all; why would you not want to use a human pianist?   I made this demo in the summer at IRCAM during my residency in Paris, just to illustrate that there maybe an alternative to the tradition of "score following" in computer music, where people have been developing sophisticated systems to detect pitch, beat, rhythm etc. in order for computer to "follow" the human player.  Although "score following" doesn't interest me as a musical tool in interactive music, I wanted to show that it is the "musical flow" which is more crucial to performance than notes and beats themselves.

When I thought about it, I just imagined how do we, in fact, performers "follow" the other player during performance.  We do listen to the pitch, beat etc., but we don't really try to match note by note without the "flow" of the music.  We are following the "flow" of music that notes and beats and the rest of it "fit" inside.

So my approach was, to create the "flow" using the bowing motion sensor, in a very "violin" way, by simply tracking a "sustaining" motion, which carves the "flow" of the phrases.  The result, with this very simple "one trick pony" approach, was surprisingly accurate.   Today people still ask "How are you tracking this?" even though I repeatedly say there is no "tracking" or "score following" involved.


In the MaxMSP program I have used some of the "common sense" of performance, such as setting the the minimum and maximum tempo.  No sane performer would slow down or speed up beyond the acceptable musical limits, for example :)   So no matter how I try to speed up or slow down using the bowing "sustaining" motion, there is a "limit".  In this video therefore, I am simply controlling the tempo within those limits.

The mathematics behind the system was created by Nicolas Rasamimanana of the Realtime Musical Interactions Team at IRCAM.  We worked together to come up with the correct calibration which was the most time consuming part.  Nicolas is also a violinist, so it was so easy for him to understand what I wanted.  When I first approached him with this idea, what he said was, "But we already have that function, although we never thought of using it this way!"  There is however, a hand-drawn calibration table I made inside the program where I listened and tried so many times to get the scaling just right.  And that really is user-specific.

For those who are in computer music, in this demo I am running MaxMSP inside Ableton LIVE's Max for LIVE, using API to simply control the tempo of the quantized MIDI piano sequence.    By the way, the piece I'm playing for this demo is "Eu Te Amo" by Tom Jobim and Chico Buarque.  It is a very soothing music perfect for me trying over and over again calibrating :)