This work introduces a real-time, human-in-the-loop (HITL) audio mixer that integrates reinforcement learning from human feedback (RLHF) and neural source separation to achieve adaptive, personalized audio mixes. Existing automated mixers primarily address the optimization of technical parameters but often cannot understand the nuanced artistic preferences critical to high-quality audio production. In order to close this gap, the framework given combines ongoing user feedback with a reinforcement learning agents decision making process, where it is trained with Proximal Policy Optimization (PPO). By employing Demucs in neural source separation, the system obtains precise stem isolation, enhancing mixing controllability and flexibility. Explicit ratings as well as implicit behavioral signals are used as iterative user feedback, which guides the agent towards enhancing its mixing strategy with the passage of time. Experimental tests, using signal-to-distortion ratio (SDR) measurements and user satisfaction surveys, demonstrate significant technical quality and creative match enhancement. This method significantly bridges the gap between automation and artistic control and offers an adaptive and flexible solution for audio engineers and music producers seeking efficiency as well as individuality in audio mixing.
(See document for exact affiliation information.)
https://aes2.org/publications/elibrary-page/?id=23022
(2563KB)
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member Join the AES. If you need to check your member status, login to the Member Portal.
