Modelling analogue distortion replicates the warmth, character, and nonlinear harmonics of classic gear often used in audio production, to be incorporated into modern, digital workflows. Creating such models can be challenging as it requires accurately capturing complex nonlinearities and dynamic responses while while ensuring real-time processing. We evaluate the performance of various transfer learning methods for improved modelling of analogue distortion effects. These approaches take into account the choice of pretrained model, the choice of dataset for fine-tuning, and the method in which the knowledge is transferred from source to target task. We compare the impact of using a source effect closest to the target one for pretraining, determined either via feature extraction or using domain knowledge, to the use of a "general model". We evaluate how to transfer the knowledge from source to target task by assessing which weights to freeze and what data to use for the downstream training. Each approach is assessed on two architectures applied to three example distortion modelling tasks. All models obtained via transfer learning are compared to a baseline model trained from scratch with random initialisation to gauge the gain in accuracy due to the transfer. Results show that transfer learning can help improve model performance and reduce training times when the appropriate transfer approach is used.
(See document for exact affiliation information.)
https://aes2.org/publications/elibrary-page/?id=22997
(550KB)
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member Join the AES. If you need to check your member status, login to the Member Portal.
