A collaboration with Prof. Neil Gershenfeld, led to cluster-based embedding modeling, a novel approach to the inference of non-linear dynamical systems from the observation of time series, and for which a patent was issued.

My Ph.D. dissertation, which I defended in October '96, investigated means of applying these principles to the inference of dynamical sound synthesis algorithms from recorded wave-forms, where the control set of the virtual instrument is perceptually meaningful. This research went even further suggesting a specific scheme, "Psymbesis", an implemented and fully functional illustration of this novel approach to sound synthesis.

I scaled down the slides I prepared for my defense and cleaned up some of my defense notes. The result is my "Cyber-defense". Note that my defense insisted upon the "modeling" part of my research and it didn't cover the "musical gesture" part of my work in detail.

Alternatively, here is a Pdf version of my dissertation (1,850 kbytes).

Computer music is not the artistic expression of an exclusive set of composers that it used to be. Musicians and composers have grown to expect much more from electronics and computers than their ability to create "out-of-this-world" sounds for a tape piece. Silicon has already made its way on stage, in real-time musical environments, and computer music has evolved from being an abstract layer of sound to a substitute for real instruments and musicians. Within the past three decades, an eclectic set of tools for sound analysis and synthesis have been developed without ever leading to a general scheme which would highlight the issues, the difficulties and the justifications associated with a specific approach. A rush in the direction of incremental improvement of existing techniques has traditionally distinguished analysis and synthesis.

Rather than confining ourselves to one of these arbitrarily exclusive tasks, we are pursuing an ambitious dream from a radically new perspective. The ultimate goal of this research is to infer virtual instruments from the observation of a real instrument without any strong pre-conception about the model's architecture. Ideally, the original observation should be a simple audio recording. We also want for our inferred virtual instruments to exhibit physically realistic behaviors. Finally, we want the nature of the virtual instrument's control to be universal and perceptually meaningful.

For this purpose, our investigation falls naturally into three steps. We first identify a set of perceptually meaningful musical gestures which can be extracted from an audio stream. In the case of a monochromatic sound, we discuss the definition and the estimation of loudness, pitch contour, noisiness and brightness. The second step is to investigate means by which a physically meaningful model can be inferred from observed data. While doing so, we introduce embedding modeling as our general philosophy and reduce modeling to the characterization of prediction surfaces. We also suggest some general purpose interpretations, including an original cluster-weighted modeling technique. Finally, our third and last step is to suggest a strategy for applying such modeling ideas to musical audio streams parametrized by the perceptually meaningful musical gestures that we previously identified. We present pitch synchronous embedding synthesis (or Psymbesis), a novel approach to the inference of a virtual instrument, as a working sound synthesis algorithm and an interpretation of these suggestions.

Psymbesis was designed specifically around musical instrument modeling but the general philosophy of embedding modeling extends beyond the field of computer music. We expect that the introduction of embedding modeling will provide signal modeling with a new perspective, relaxing the constraint of linearity and filling the present gap between physical models and standard signal processing.