Besides the creation of MIDI files, the other important function of Helm is to follow along in the score as it ``listens'' to the MIDI data from a performer's keyboard, transmitting a retuned version of this MIDI data to a MIDI synthesizer in real-time. There are five major software components that are used to accomplish score following. The first two are the SDL parser and score serializer already discussed. The third creates an input/output (I/O) map file that describes what the follower is looking for and what it should translate its findings into. Another way of describing this file is a map from the untuned MIDI data of the score (what it will be receiving from the performer) to the tuned MIDI data of the score (what it will be sending to the synthesizer). The fourth and fifth components are the only part of Helm that are not written in Java; they are written in C++ [25] because they need to access operating system calls that deal with MIDI input and output. The fourth major component of the software is the program that reads in the I/O map file and actually performs the score following and remapping. This program is built upon the fifth and final major component, the Rogus McBogus C++ MIDI Library [6].
The serial score to I/O map converter works as follows. There is no timing information in the I/O map file; the score is just segmented into clumps of inputs (Note Ons and Offs) that appear simultaneous according to the score, along with the output they are supposed to map to. This timelessness is due to the fact that the score follower that uses this information is naive and cannot take advantage of timing information. The MIDI output information does not include channel assignment: that is done later by the score follower itself.
Figure 4.2 contains a fragment of an I/O map file for the phrase of Bach introduced in Figures (removed due to latex2html problem) and 4.1. A clump is a pair of NOF/NON lines. A NOF line lists the MIDI note numbers of any Note Offs expected from the keyboard around the time of the clump. A NON line lists comma-separated I/O pairs for any Note Ons expected from the keyboard around the time of the clump. The I/O pairs consist of three numbers: the input Note On expected from the keyboard, and the corresponding Note On and Pitch Bend values to be sent out as a result.
So, for example, the second clump contains the Note Offs expected from the first chord (74, 70, 65, and 46) and the mappings for the Note Ons of the second chord (70, 67, 63, and 51). Because the second chord is tuned to 12TET as per the underscores in the input file (see Figure 4.1), the Note Ons of the second chord map to the same Note On value with a Pitch Bend of zero, i.e. the I/O pairs are all of the form X (X 0).
As was already mentioned, the score follower is naive. In fact, it is not so much a score follower as an interactive score player. It will never get lost because it basically uses the keyboard input to step through the score. In fact, the major problem with the so-called score follower is that it is easy for the performer to get lost, whereas in real score followers, the problem is just the opposite: it is easy for the computer to get lost! It was decided that since this project was about tuning, not score following, a naive score follower that worked well enough to serve as a proof of principle would suffice.
The score follower has three modes of operation: performance mode, note mode, and chord mode. In performance and note mode, the program uses only the note on information in the I/O map file. When a key is released, it looks up the most recent Note On it sent as a result of a press of that key and sends an appropriate Note Off. Thus the duration of notes is entirely controlled by the performer. In performance mode, the only way to cause a note to be played on the synth is to press a key that corresponds to a Note On in the current clump. Recall that a clump is a bunch of inputs that appear simultaneous in the score and the outputs that they should produce. When all the keys in the current clump have been pressed, the program moves its ``clump pointer'' to the next clump.
Note mode is the same as performance mode except that input from any key will be accepted, and the clump will map it to whatever output appears first in the clump. The ordering of outputs in the clump is by part number. So, for example, in performance of a score with part numbering as in Figure 4.1, repeatedly pressing a single key will step through all the notes of the piece with chords being stepped through from top down. To play the piece polyphonically, all that is necessary is to perform the rhythm correctly since which key is depressed does not matter. Chord mode plays out an entire clump (Note On and Note Off outputs) every time it receives a key press. All of the modes map the input velocity through to the output velocity, giving the performer the normal expressive control of a MIDI keyboard. In the case of chord mode, all the notes of a chord have the same velocity as the input keypress that triggered it.
The score following software is implemented on top of the Rogus McBogus C++ MIDI Library [6], written by the author and Patrick Pelletier. It was written primarily to support the MIDI programming required in the Brain Opera [15] project, an interactive multimedia performance piece.
