Singing voice conversion (SVC) converts audio sung by a source voice into the timbre of a target voice without changing the linguistic content or melody of the sung utterance. The popular SVC paradigm is to extract content features from the source and condition on timbre features extracted from the target to generate converted audio. In recent years, great strides have been made in SVC with advancements in content representations. However, what constitutes an accurate representation of timbre is still an emerging topic. Currently, most SVC systems use a single speaker embedding to represent one singer, but this disregards the large variations in timbre characteristics even for the same person when singing in different techniques and vocal ranges. In this paper, we propose a cross-attention mechanism to generate frame-wise speaker timbre embeddings informed by linguistic content and melody; we aim to generate timbre features that more accurately reflect how the voice changes with different vocal ranges and enunciations at different times. Objective metrics and subjective listening tests of our converted singing audio show improvements in enunciation clarity and voice similarity, while expressiveness is a more elusive challenge.
(See document for exact affiliation information.)
https://aes2.org/publications/elibrary-page/?id=23012
(364KB)
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member Join the AES. If you need to check your member status, login to the Member Portal.
