ISCA Workshop on
Statistical And Perceptual Audition

Makuhari, Japan
September 25, 2010

Informed Source Separation of Orchestra and Soloist using Masking and Unmasking

Yushen Han, Christopher Raphael

School of Informatics and Computing, Indiana University Bloomington

A novel technique of unmasking to repair the degradation in sources separated by spectrogram masking is proposed. Our approach is based on explicit knowledge of the musical audio at note level from a score-audio alignment, which we termed Informed Source Separation (ISS). Such knowledge allows the spectrogram energy to be decomposed into note-based models. We assume that a spectrogram mask for the solo is obtained and focus on the problem of repairing the degraded audio. We evaluate the spectrogram as well as the harmonic structure of the music: we either search for unmasked (orchestra) partials of the orchestra to be transposed onto a masked (solo) region or reshape a solo partial with phase and amplitude imputed from unmasked regions. We describe a Kalman smoothing technique to decouple the phase and amplitude of a musical partial that enables the modification to the spectrogram. Audio examples from a piano concerto are available for evaluation.

Index Terms: musical audio source separation BSS score following spectrogram masking Kalman phase estimation

Full Paper

Bibliographic reference.  Han, Yushen / Raphael, Christopher (2010): "Informed source separation of orchestra and soloist using masking and unmasking", In SAPA-2010, 31-36.