Conditioned Source Separation by Attentively Aggregating Frequency Transformations With Self-Conditioning

Woosung Choi, Yeong Seok Jeong, Jinsung Kim, Jaehwa Chung, Soonyoung Jung, Joshua D. Reiss

Research output: Contribution to journalArticlepeer-review

Abstract

Label-conditioned source separation extracts the target source, specified by an input symbol, from an input mixture track. A recently proposed label-conditioned source separation model called Latent Source Attentive Frequency Transformation (LaSAFT)–Gated Point-Wise Convolutional Modulation (GPoCM)–Net introduced a block for latent source analysis called LaSAFT. Employing LaSAFT blocks, it established state-of-the-art performance on several tasks of the MUSDB18 benchmark. This paper enhances the LaSAFT block by exploiting a self-conditioning method. Whereas the existing method only cares about the symbolic relationships between the target source symbol and latent sources, ignoring audio content, the new approach also considers audio content. The enhanced block computes the attention mask conditioning on the label and the input audio feature map. Here, it is shown that the conditioned U-Net employing the enhanced LaSAFT blocks outperforms the previous model. It is also shown that the present model performs the audio-query–based separation with a slight modification.

Original languageEnglish
Pages (from-to)661-673
Number of pages13
JournalAES: Journal of the Audio Engineering Society
Volume70
Issue number9
DOIs
Publication statusPublished - 2022 Sep

ASJC Scopus subject areas

  • Engineering(all)
  • Music

Fingerprint

Dive into the research topics of 'Conditioned Source Separation by Attentively Aggregating Frequency Transformations With Self-Conditioning'. Together they form a unique fingerprint.

Cite this