Auditory Augmented Reality (AAR) is an emerging technology that enhances user experiences by overlaying spatially appropriate sounds onto a real environment and gives the impression of virtual sounds coexisting with the physical world. A major challenge in achieving a plausible real-time AAR experience is to seamlessly align virtual soundscapes with real-world environments. To address this, we previously introduced the Augmented reality Room Acoustic Estimator (ARAE). However, the earlier versions of the ARAE encountered difficulties in appropriately replicating the room acoustic behavior due to inadequate rendering of early reflections. To overcome these challenges, we implemented two innovative techniques: mesh segmentation to identify dominant acoustic elements and material estimation through deep learning. Our pilot measurement demonstrated that the proposed method better replicates the sound behavior of the physical environment than the one without material estimation in terms of its frequency characteristics, although some discrepancies were also noted. Future developments will concentrate on customizing the material estimation datasets with the intention of inferring real-world acoustic conditions and optimizing the proposed method through subjective evaluations.
(See document for exact affiliation information.)
https://aes2.org/publications/elibrary-page/?id=22404
(4228KB)
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member Join the AES. If you need to check your member status, login to the Member Portal.
