We propose a differentiable WORLD synthesizer and demonstrate its use in end-to-end audio style transfer tasks such as (singing) voice conversion and the DDSP timbre transfer task. Our baseline differentiable synthesizer has no model parameters, yet it yields adequate synthesis quality. We extend the baseline synthesizer by appending lightweight black-box postnets applying further processing in order to improve fidelity. An alternative differentiable approach relies on the extraction of the source excitation spectrum directly, and results in improved naturalness, albeit for a narrower class of style transfer applications. The acoustic feature parameterization used by our approaches has the benefit that it naturally disentangles pitch and timbral information so that they can be modeled separately. Moreover, as there exists a robust means of estimating these acoustic features from monophonic audio sources, it enables new training configurations and allows for parameter loss terms to be added to an end-to-end objective function, which can help convergence and/or further stabilize (adversarial) training.
(See document for exact affiliation information.)
https://aes2.org/publications/elibrary-page/?id=22073
(1751KB)
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member Join the AES. If you need to check your member status, login to the Member Portal.
