microsoft / spatialaudio-unity Goto Github PK

This repository provides plugins, tools and samples for integrating spatial audio and acoustics into your Unity 3D applications and games.

License: MIT License

CMake 0.06% C++ 99.54% C 0.18% MATLAB 0.03% Python 0.19%

unity nuget audio spatial-audio hololens 3daudio acoustics

spatialaudio-unity's Introduction

Introduction

This repository provides plugins and tools for integrating spatial audio into your Unity 3D applications and games

A cross-platform spatializer plugin (supported on Windows and Android) that uses highly efficient spatial audio DSP processing algorithms.
A sample Unity application that demonstrates cross-platform spatializer plugin configuration and usage.

Getting started with Spatial Audio for Unity

Cloning this repository is not required to start using the Microsoft Spatializer in your Unity project. Visit the documentation for instructions on integrating the Microsoft Spatializer into your Unity project. For a more in-depth exploration of spatial audio, check out the learning module. If you'd like to build the plugin yourself, see below.

Choosing the right spatializer

With requirements and features evolving over time, there are now 3 different Unity spatializer plugins available from Microsoft. Here's a brief description of their differences which can help decide the right plugin for a project.

Microsoft Spatializer v2

This is the latest highly optimized cross-platform spatializer plugin for Windows and Android built from this repository. Although this plugin is currenly in the pre-release phase, it's being actively developed and recommended for any new projects, especially those that need to support both Windows and Android. This plugin uses the latest DSP engine that is highly optimized for both memory and CPU and fits well into Unity's audio engine architecture.

Microsoft Spatializer v1

While it is recommended to switch over to the latest cross-platform spatializer plugin, the previous HoloLens 2 specific spatializer plugin version with hardware offload suppport, remains available on GitHub releases and on a UPM feed via Mixed Reality Feature Tool. This plugin can be useful for any HoloLens 2 specific projects where it can reduce the CPU usage by leveraging offloaded spatial audio DSP. This plugin utilizes Windows Spatial Audio Platform APIs that prevent the processed audio signal to flow back into Unity's audio engine which can make it cumbersome for supporting some audio design features, such as adding an environmental reverb to the spatial audio mix.

Unity MS-HRTF Plugin

This is the original spatializer plugin which is shared for historical purposes. This plugin does not utilize the multi-source mixer plugin which leads to higher compute overhead than newer plugin offerings.

Required Software

Windows SDK 18362+	Visual Studio 2019	CMake	Unity 2019+	Python 3+	Node.js	Android NDK
Windows 10 May 2019 Update SDK to build the spatializer plugin.	Visual Studio is used for code editing, deploying and building UWP app packages	CMake is required for generating Visual Studio 2019 projects	Unity 2019 is required to build the spatializer plugin package. Plugin can be used on Unity 2018 LTS and higher versions.	Helper scripts for build and packaging use Python 3 and higher.	For UPM packaging.	Required for building Android binaries.

Branch Guide

This repository follows the GitFlow branching model.
Master branch is used for building release candidates and official releases. Direct pull requests into master are not allowed.
Develop branch is used for staging ongoing work for the next official release and merged with master after extensive review and testing. Direct pull requests into develop branch are not allowed.
Use feature branches to bring up individual features. Once a feature is ready and tested, use a pull request to merge it into the develop branch.

Clone the Repository

git clone https://github.com/microsoft/spatialaudio-unity.git --recurse-submodules

If you forget to include submodules when cloning, add them with git submodule update --init --recursive

Build Status

Build	Branch	Status
Release	master
Validation	develop

Local Build

Launch "Developer Command Prompt for Visual Studio 2019".
Switch directory to the root of your Git enlistment.
Run the CMake script to generate Visual Studio 2019 projects: python3 Tools\runcmake.py
Run the build script to build all flavors: python3 Tools\build.py
To generate the Unity package: python3 Tools\unity_package.py -u "c:\Program Files\Unity\Hub\Editor\2020.3.2f1\Editor" -v 2.0.0
To generate the UPM package: python3 Tools\upm_package.py -v 2.0.0

Artifacts

Build produces UPM and Unity asset packages
Unity asset package is available under releases tab
Unity asset packages are also available on a UPM feed via Microsoft Mixed Reality Feature Tool

spatialaudio-unity's People

Contributors

Stargazers

Watchers

spatialaudio-unity's Issues

Please update repository readme to reference the Mixed Reality Feature Tool

Per Unity's Terms of Service, Microsoft Mixed Reality UPM packages are currently not distributed via scoped registry. Please update the instructions on the readme to reference the Mixed Reality Feature Tool for acqusition.

Download location for MRFT is https://aka.ms/mrfeaturetool
Here is an image of the spatializer in the tool UI

package update

Will there be a package update in the near future or is this project dead?

I reported an issue over a year ago and it got closed with the notice that it will be fixed with the next update, but this never happened (#83).

Plugin package is missing symbols

Need to add PDBs to the plugin package to help with diagnosability.

v1.0.36 isn't available in the Github releases page but is available on Nuget.

Latest release on Nuget is v.1.0.36: https://www.nuget.org/packages/Microsoft.SpatialAudio.Spatializer.Unity/
Latest release on Github is v1.0.18: https://github.com/microsoft/spatialaudio-unity/releases

Is Nuget the direction that this project is heading? It would be great if the Unity packages could be kept in lock-step

UWP binaries are built incorrectly

CMake project generation needs to take the right flags for UWP configs. Right now UWP plugins fail to load because of non-UWP CRT linkage.

Sample is missing NuGet binaries

The sample is missing NuGet for Unity binaries, so doesn't load the Nuget packages for MRTK and the spatializer on project start.

Sample project doesn't support latest Unity LTS (2019.4)

Sample project can't be auto-ugraded by Unity to current Unity LTS (2019.4) because MRTK version used in sample project doesn't support Unity 2019.4.

Build status badges aren't properly displayed

Claims master branch never built, and image for develop branch is not showing for me.

Spatial Sound in Unity Microsoft.SpatialAudio.Spatializer.Unity Requires Dependencies

I think the documentation [here: https://docs.microsoft.com/en-us/windows/mixed-reality/spatial-sound-in-unity] should be updated to include any required dependencies that the user needs to install. I thought this plugin was independent.

Got this error:
Plugins: Failed to load 'Assets/Packages/Microsoft.SpatialAudio.Spatializer.Unity.1.0.0/Plugins/x86_64/AudioPluginMicrosoftSpatializer.dll' because one or more of its dependencies could not be loaded.

Unity app crashes at invalid_pointer_read_c0000005_audiopluginmicrosoftspatializer.dll!unitygetaudioeffectdefinitions

We publish a Hololens2 Unity app that uses Microsoft audio spatializer. And in the device portal, we receives many reports about application crashes at invalid_pointer_read_c0000005_audiopluginmicrosoftspatializer.dll!unitygetaudioeffectdefinitions. The following is the callstack (not very helpful since it has no symbol)

Any ideas how to resolve the problem?

AudioSource GetOutputData and GetSpectrumData Issues

Describe the bug

Output and spectrum data do not exist while running on HoloLens 2.

Sample of how we're using the two methods:

audioSource.GetOutputData(audioSamples, 0);
audioSource.GetSpectrumData(audioSpectrum, 0, FFTWindow.BlackmanHarris);

To reproduce

Steps to reproduce the behavior:

Have an AudioSource that's playing looping audio with all spatializer settings enabled.
Write a simple script that computes DB, like so: https://answers.unity.com/questions/157940/getoutputdata-and-getspectrumdata-they-represent-t.html
Verify this works with MS HRTF Spatializer on HoloLens 2.
Swap to Microsoft Spatializer and redeploy.
See that the results now on HoloLens 2 are not consistent.

Expected behavior

Behavior should be consistent with MS HRTF Spatializer and what's being seen in the Unity editor.

Your setup

Unity Version 2019.2.21f1
Microsoft Spatializer v1.0.0

Target platform

HoloLens 2
WMR immersive

Can't change spatial sound format once activated

When using the Microsoft Spatializer on desktop, I sometimes want to change my spatial sound mode between Windows Sonic, Dolby Atmos, and DTS. I've seen that once I play any audio in one of these modes, I need to close and reopen Unity to change to another mode. For example:
Have spatial sound mode set to Windows Sonic for Headphones.
Play some spatial audio in Unity. Works!
Change spatial sound mode to Dolby Atmos for Home Theater.
Play some spatial audio in Unity. Doesn't work.
Close Unity.
Open Unity.
Play some spatial audio in Unity. Works!

It would be nice if the Microsoft Spatializer was updated to handle these mode switches automatically without needing to restart Unity.

Binaries for x86 incorrectly built as x64

VS2019 generator, unlike VS2017 and prior, defaults to host architecture rather than Win32 (x86). CMake generator scripts need to change to address this.

Incorrect plugin import settings for standalone x86_64

In the latest version 2.0.37 the import settings for x86_64 HrtfDsp.dll are not configured correctly - x64 should be checked in Platform settings for standalone builds. At the moment both available options (x86, x64) are unchecked.

Missing mainfast?

When I try to add the code into unity via package manager and the git link I receive this error

[Package Manager Window] Cannot perform upm operation: Unable to add package [https://github.com/microsoft/spatialaudio-unity.git]:
[https://github.com/microsoft/spatialaudio-unity.git] does not point to a valid package. No package manifest was found. [NotFound].

I'm using Windows 11 and unity 2021.2.19