Head-worn devices equipped with microphone arrays and cameras can be used to capture the experience from a user’s perspective and reproduce it in virtual, mixed, or augmented reality. A concept that has recently introduced is to present the video capture as a 2D video screen augmented into the real-world environment through a mixed reality headset. This study presents such a system for reproducing audio and video capture from glasses arrays as a video “augment” along with binaural audio. Results of an initial listening experiment are presented, evaluating different state-of-the-art methods for binaural rendering. A stereo rendering through virtual loudspeakers attached to the video "augment” is compared with head-locked and world-locked binaural syntheses based on a binaural beamforming approach. The results suggest that listeners rated beamforming-based reproduction higher than stereo rendering. World-locked rendering was not rated significantly better than the head-locked version.
(See document for exact affiliation information.)
https://aes2.org/publications/elibrary-page/?id=22924
(798KB)
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member Join the AES. If you need to check your member status, login to the Member Portal.
