Video Game Music (VGM) is essential to video games. In particular, preparing a suitable VGM is required to raise the quality of the game. However, it is not easy to retrieve a VGM that has both the worldview of the game and the atmosphere of the scene at the same time. Therefore, we propose a system to retrieve VGM considering these conditions. This system receives three inputs. (1) Source VGM MP3: MP3 of VGM to be adopted in the created game, (2) Source VGM Tag: Tag of the scene using the adopted VGM, (3) Target VGM Tag: Tag of another scene to add VGM. First, the variational auto-encoder (VAE) converts Source VGM MP3 into the latent variable z. Second, let Pz be the set of latent variables of the VGM corresponding to Source VGM Tag. Let pc be the center of Pz. Similarly, let Qz be the set of latent variables of the VGM corresponding to Target VGM Tag. Let qc be the center of Qz. Third, let z? be the latent variable of Target VGM MP3. The system assumes that z ? pc = z? ? qc. Therefore, z? is computed by z? = z +qc ? pc. Finally, the system retrieves the VGM with latent variables close to z?. We conducted a preliminary experiment to confirm that VAE can train on this system. As a result of training VAE, the loss function of the training data continued decreasing, but the loss function of the test data stopped decreasing. Therefore, VAE likely caused overtraining. It is necessary to tune the VAE to the task. As a result of the restoration, VAE restored the timbre, not the tempo. It is necessary to use robust audio features for tempo.
(See document for exact affiliation information.)
https://aes2.org/publications/elibrary-page/?id=22397
(449KB)
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member Join the AES. If you need to check your member status, login to the Member Portal.
