A multichannel extension to the RVQGAN neural coding method is proposed, and realized for data-driven compression of third-order Ambisonics audio. The input- and output layers of the generator and discriminator models are modified to accept multichannel input. We also propose a loss function which accounts for spatial perception in immersive reproduction, and transfer learning from single-channel models. Listening test results with 7.1.4 immersive playback show that the proposed extension is suitable for coding ambient scene-based, third-order (16-channel) Ambisonics content with good quality at 16 kbps when trained and tested on the EigenScape database. The model presented is the first neural codec dedicated to immersive audio to the authors knowledge and has potential applications for learning other types of content and multichannel formats.
(See document for exact affiliation information.)
https://aes2.org/publications/elibrary-page/?id=22992
(376KB)
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member Join the AES. If you need to check your member status, login to the Member Portal.
