Project Overview
Microsoft Soundscape was an innovative project designed to help individuals with low vision or blindness navigate their surroundings using 3D audio. Developed in collaboration with Microsoft Research, the app provided an enhanced sense of spatial awareness by delivering auditory cues about nearby locations.
Client
Microsoft Research
Role
Interaction designer
Tools
Sketch, Adobe Premiere, Adobe Audition, Hype4, Web technologies (HTML+CSS+JS)
Project duration
3 year, 4 month
(Jan 2017 – May 2020)
The MS Soundscape’s journey
The project was guided by a team committed to making navigation more intuitive. Instead of replacing traditional mobility tools, Soundscape complemented users’ existing skills by providing contextual audio information without overwhelming them.
Designing an Audio-First Experience
Creating an audio-based augmented reality (AR) experience came with unique challenges. The team experimented with non-visual prototypes, sound cues, and binaural audio mockups to refine how information was presented. They considered real-world scenarios like walking down streets, using public transport, and navigating indoor spaces.
Wayfinding and audio-based augmented reality
The Soundscape team consisted of individuals with diverse backgrounds. One of the first challenges in the project was to have a good understanding about wayfinding concepts. Under the guidance of Arturo Toledo from Pixelspace, these conversations took shape, while I observed, analyzed, and documented key insights.
To ensure this knowledge was preserved, I created diagrams, slides, maps, and other materials. Some of these were later printed and showcased at Microsoft’s offices.
Designing audio-based AR experience concepts
We iterated multiple times to find the right balance between contextual information, the app’s sound elements, and the physical environment. We considered scenarios such as walking along a street, utilizing public transport, or being indoors during the experience.
Testing Audio-based AR experience concepts
Prototyping tools are unable to work correctly with iOS VoiceOver, even nowadays. However, I knew HTML and ARIA properties could be an option since I created websites from scratch in the past.
We tested the app’s layout and messages using this method. To reduce time, I used Tumult Hype 4’s accessibility panel.
I created binaural audio to test the 3D audio experience and connected the phone to a sampler. So, when users interacted with a UI element, such as a button, with the sampler, the team manually reproduced the specific audio directly to the headphones. Additionally, a speaker was used to hear what the user was listening to through the headphones.
By doing this, users could test both the audio experience and perceive the app using VoiceOver at the same time.
Expanding the Soundscape identity
Launch and Legacy
Microsoft Soundscape officially launched in 2019 to positive reception, winning two awards and receiving praise from users and reviewers. However, Microsoft discontinued support in 2022, transitioning the technology to an open-source project. Fortunately, the Soundscape Community, including former team members, continues to develop and improve the platform.
Even though the original app is no longer available, Microsoft Soundscape paved the way for new innovations in accessible technology. Its impact on mobility and independence for visually impaired users remains a testament to the power of 3D audio in making the world more inclusive.
I found this project very enjoyable. I appreciate working with such dedicated people and being part of a project that had many challenges. Still, it was well-received in the end.
Check another project
MyRX: Vaccine Appointment Hub
During the COVID lockdown, myRX enabled remote scheduling of COVID tests and vaccines to reduce the spread of the virus.