This paper describes the design and implementation strategies proposed in the development of a synthesizer that allows the multimodal exploration of the Ca II triplet (CaT) stellar spectra library from the Spanish Virtual Observatory (SVO). The prototype reacts "on the fly" to provide harmonic accompaniment for each note of the piano keyboard, generated from real sky objects data using deep learning. A two-layer autoencoder architecture reduces each stellar spectrum to a six-note MIDI chord. The algorithm looks for the root note of the resulting "stellar chords" matching the user pressed piano key note, and generates the sonification as function of its encoded latent vector and the number of absorption lines detected in the spectrum. The UI provides the name of the object, the spectrum, and the accuracy of the auditory representation via R2 calculated between the original and the decoded spectrum. The proposal introduces a musical extension of the sonification of astronomical data to be used in sound design and musical composition for the creation of complex timbres and textures, and in educational and outreach activities to bring astronomy concepts closer to all audiences using the engagement potential of sound and music.
(See document for exact affiliation information.)
https://aes2.org/publications/elibrary-page/?id=22537
(811KB)
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member Join the AES. If you need to check your member status, login to the Member Portal.
