How unity-technologies/nativeaudioplugins works
1. This is a Unity-based audio plugin demo workspace, not a single app. The repository combines a Unity project, native C++ DSP/plugin sources, and custom editor GUIs for Unity audio effects.
2. The top-level architecture has three major layers:
• Unity runtime content under Assets/
• Native audio plugin implementations under NativeCode/
• Custom inspector/editor tooling under GUICode/
These layers are connected through Unity’s native audio plugin API.
3. The Unity project is organized as a set of demo scenes in Assets/scenes, with matching audio mixer assets in Assets/mixers. Each scene is a self-contained showcase for one plugin or audio technique, such as convolution reverb, granulation, spatialization, vocoding, pitch detection, speaker routing, teleport, and weather/vehicle effects.
4. Runtime behavior in Unity is driven by C# scripts in Assets/scripts. These scripts handle demo setup and parameter control, including microphone capture, sample upload, audio recording, camera/player movement, UI text, playback position display, physics-driven audio triggers, and per-demo modulation logic. They are the bridge between scene objects and the audio plugins.
5. Audio content lives in Assets/sounds, including source clips, impulse responses, and effect materials. These files feed the demos as input signals, control material, or convolution IRs, depending on the scene. Assets/materials and Assets/physics materials support visual/physical demo setups rather than the audio DSP itself.
6. The native DSP layer is in NativeCode/. This is the core of the SDK: each Plugin_*.cpp file implements one effect or instrument, such as equalizer, granulator, reverb, spatializer, vocoder, wah-wah, pitch detector, routing, and synth modules. AudioPluginInterface.h and AudioPluginUtil.* provide the shared plugin API and helpers.
7. NativeCode/PluginList.h is the integration point that enumerates available native plugins for the demo build. Architecturally, this is the registry that connects Unity-side demo scenes to specific native effect implementations.
8. Several plugins are designed around real-time audio input/output rather than static clips. The MicrophoneFeed, AudioRecorder, GranulatorUploadSample, and ConvolutionReverbUploadIR scripts indicate a data path where audio can be captured, transformed, and uploaded into the native plugin state during play mode.
9. The “teleport” demo is the main boundary-crossing scenario. It demonstrates audio transport between Unity and external applications through NativeCode/TeleportDemo.cpp and the Audio Unit plugin in NativeCode/Xcode/AUTeleport. This is the only part of the repo that depends on outside audio applications and Apple Core Audio utility classes.
10. Platform packaging is explicit. Assets/Plugins contains prebuilt native binaries and symbols for Windows/UWP/macOS, while NativeCode/VisualStudio, NativeCode/UWP, and NativeCode/Xcode contain build projects for the same plugin family on each platform. The repo therefore supports both shipping binaries and rebuilding the native layer from source.
11. GUICode/AudioPluginDemoGUI contains custom editor windows and inspectors for plugin parameters and meters. These GUI classes are not the runtime audio path; they are editor-side tooling that communicates with plugin parameters and provides visual controls such as scopes, meters, equalizers, and multiband displays.
12. The included PDFs in Documentation/ explain the SDKs this repo builds against: the Native Audio Plugin SDK, audio spatializer SDK, and ambisonic audio material. They are reference docs for the plugin interfaces used by the native layer and the demo scenes.
13. Overall data flow is: Unity scene assets and scripts instantiate demo objects, send audio or parameter data into the native plugin layer, the plugin processes audio in real time, and the results are routed back through Unity’s audio mixer/output. The teleport demo extends that flow to external processes via an additional audio transport boundary.
Main entry points: PluginList.h Plugin registry · NativeCode Core effects · AudioPluginInterface.h Plugin API · scripts Demo scripts
Components 16
Unity runtime
Scenes, scripts, audio content, and mixer assets that drive the demos in play mode.
- Demo scenes · Self-contained showcases for reverb, granulation, spatialization, vocoding, pitch detection, routing, teleport, and related effects.
Assets/scenes - Demo scripts · Scene setup, parameter control, capture/upload, movement, and per-demo modulation logic.
Assets/scripts - Mixers · Matching mixer assets used by the demo scenes for signal routing and output presentation.
Assets/mixers - Audio assets · Clips, impulse responses, and other audio materials consumed by the demos.
Assets/sounds
Native DSP
C++ audio plugin implementations and shared SDK glue used by Unity through the native audio plugin API.
- Plugin API · Shared native audio plugin interface used by all effect implementations.
NativeCode/AudioPluginInterface.h - Shared utils · Common helpers and utility code for native plugin implementation details.
NativeCode/AudioPluginUtil.cpp - Plugin registry · Enumerates the native plugins available to the demo build and links Unity demos to implementations.
NativeCode/PluginList.h - Core effects · Main processing plugins such as convolution reverb, equalizer, granulator, modal filters, multiband, spatializer, vocoder, and wah-wah.
NativeCode - Meters and analysis · Correlation, loudness, oscilloscope, pitch detection, and related analysis plugins.
NativeCode - Generators and routing · Impact, impulse, noise, synth, routing, widening, and modulation-oriented plugins.
NativeCode - Teleport stack · Teleport demo transport path spanning Unity, native C++, and the external Audio Unit companion.
NativeCode/TeleportDemo.cpp - AU companion · macOS Audio Unit companion used by the teleport demo for cross-application audio transport.
NativeCode/Xcode/AUTeleport
Editor GUI
Unity editor-side plugin inspectors, meters, and custom parameter controls.
- Custom GUIs · Custom inspectors and editor windows for meters, scopes, equalizers, and plugin parameter controls.
GUICode/AudioPluginDemoGUI
Packaging and docs
Platform-specific binaries, build projects, and reference documentation for the plugin SDKs.
- Prebuilt plugins · Shipping binaries and symbols for supported Unity target platforms.
Assets/Plugins - SDK docs · Documentation for the Native Audio Plugin SDK, Audio Spatializer SDK, and Ambisonic audio material.
Documentation - Build projects · Visual Studio, UWP, and Xcode build projects for rebuilding the native plugin family.
NativeCode
Connections 21
- Demo scenes → Demo scripts: drives · Scenes instantiate scripts that control demo setup and runtime behavior.
- Demo scripts → Plugin registry: binds · Scripts route parameters and uploaded audio into the registered native plugins.
- Audio assets → Demo scripts: feeds · Source clips and impulse responses are consumed or uploaded by runtime scripts.
- Mixers → Demo scenes: mixes · Mixer assets define scene-specific routing and presentation.
- Plugin registry → Core effects: registers · Registry exposes the effect plugins to the Unity demo build.
- Plugin registry → Meters and analysis: registers · Registry exposes analysis and metering plugins.
- Plugin registry → Generators and routing: registers · Registry exposes generators and routing-oriented plugins.
- Plugin API → Core effects: implements · Effect plugins are built against the shared native audio plugin interface.
- Plugin API → Meters and analysis: implements · Analysis plugins are built against the shared native audio plugin interface.
- Plugin API → Generators and routing: implements · Source and routing plugins share the common plugin API.
- Shared utils → Core effects: supports · Shared helpers reduce duplication across DSP implementations.
- Shared utils → Generators and routing: supports · Shared helpers also support generators and routing plugins.
- Demo scripts → Teleport stack: uploads · Teleport-related runtime code drives transport and I/O coordination.
- Teleport stack → AU companion: bridges · Teleport demo extends to an external Audio Unit companion on macOS.
- Custom GUIs → Meters and analysis: inspects · Custom editor windows render analysis views and meters.
- Custom GUIs → Core effects: controls · Editor-side inspectors expose effect parameters and displays.
- Prebuilt plugins → Core effects: ships · Prebuilt binaries package the native effect implementations for Unity.
- Prebuilt plugins → Meters and analysis: ships · Prebuilt binaries package analysis plugins for supported platforms.
- Build projects → Plugin registry: builds · Platform-specific projects compile the same plugin family from source.
- SDK docs → Plugin API: describes · Reference material documents the native plugin SDK interface.
- SDK docs → Teleport stack: describes · Documentation covers the spatializer and ambisonic dependencies used by the demo stack.