How oculus-samples/unity-theworldbeyond works
This project is a comprehensive Unity sample built for mixed reality on the Meta (Oculus) platform. It demonstrates how to create a dynamic, scene-driven virtual experience that adapts in real time to a user’s physical environment using features such as NavMesh navigation, dynamic object spawning, voice control, passthrough rendering, interaction (including hand tracking via the Interaction SDK) and audio spatialization. The project's purpose is to show how to integrate various Presence Platform features into an immersive experience (“The World Beyond”), and the file tree reflects a typical Unity-based XR experience with an extensive set of assets, scenes, scripts, and supporting metadata.
Main Components
• Scene Management & Core Experience:
• TheWorldBeyond.unity is the core scene.
• WorldBeyondManager (from Assets/TheWorldBeyond/Scripts/GameManagement/WorldBeyondManager.cs) acts as the central controller (similar to a main() function) coordinating the entire experience.
• Environment & Dynamic Scene Construction:
• Environment assets and prefabs support the virtual world (Assets/TheWorldBeyond/Environment/…).
• Room and floor generation from the Scene API are managed by classes such as VirtualRoom and WorldBeyondEnvironment.
• NavMesh generation and on-the-fly obstacle setup are key for Oppy’s movement (dynamic obstacles built from user room geometry).
• Character and Interaction Components:
• Oppy: All assets and scripts for the character named “Oppy” (located under Assets/TheWorldBeyond/Scripts/Characters/Oppy) manage its animations, movement, and interactions.
• MultiToy: A unique tool that attaches to a hand/controller (Assets/TheWorldBeyond/Scripts/Toy/MultiToy.cs), ensuring only one instance ever exists.
• Interaction SDK integration: Parts of the system (e.g., OVRInteraction rig, hands behaviors) that provide hand-tracking and direct manipulation components.
• Audio & Spatialization:
• Audio features are managed via the Oculus AudioManager and Spatializer (referenced by ProjectsSettings and Assets/TheWorldBeyond/Audio/WorldBeyondAudioMixer.mixer).
• Scripts under Audio (Assets/TheWorldBeyond/Scripts/Audio) and sound entry prefabs detail how sound is modified based on scene occlusion and room openness.
• Passthrough & Rendering Techniques:
• Techniques include Depth Test (using DepthOnly materials) and custom Shader Masking (all shaders labeled with “Passthrough”).
• These techniques are integrated into scene objects to blend virtual objects with a live video passthrough from the real-world.
• Voice & Wit Integration:
• Voice commands and transcription are integrated through the Voice SDK, handled primarily by WitConnector (Assets/TheWorldBeyond/Scripts/Wit).
• The system reacts to voice inputs by triggering animations or behaviors on Oppy.
• Auxiliary Sample Scenes:
• Several smaller sample scenes (e.g., Passthrough Pet, Passthrough Room, Virtual Frames, Voice Transcription) offer focused demonstrations of specific features. These can be represented as subsystems or modules illustrating different use cases of the overall architecture.
Relationships and Interactions
• WorldBeyondManager acts as the orchestrator, initializing the dynamic environment, handling room surface detection (through OVRSceneManager and Scene API), and coordinating the NavMesh, spawning of objects, and passthrough effects.
• Components such as VirtualRoom, WorldBeyondEnvironment, and SceneMesher work together to generate the virtual floor, walls, and obstacle layout based on real-world inputs.
• MultiToy’s single-instance pattern allows dynamic attachment to player controllers; it interacts with hand tracking and Input management.
• The passthrough rendering module (combining shader masking and depth test approaches) directly affects how both the virtual and real environments are composited, linking with the scene rendering workflow.
Architectural Patterns & Design Principles
• Modular Design: The system is broken into clear modules (Scene Management, Environment Generation, Character Control, Audio, Passthrough, Voice) which can be understood and modified independently.
• Data Flow: Real-world sensor input (via OVRSceneManager) feeds into environment calculation modules (VirtualRoom, SceneMesher) which then relay updated information (e.g., walkable areas) to the NavMesh and object spawners.
• Event-Driven Components: Changes such as wall openings or voice commands trigger events that are processed by corresponding managers (e.g., AudioManager, WitConnector).
Technologies, Frameworks & Libraries
• Unity Engine 2020.3.18f1 or higher.
• Oculus Interaction/Presence Platform.
• Unity NavMesh for dynamic navigation.
• Voice SDK (Wit-based) for voice interactions.
• Oculus AudioManager and Spatializer.
• Custom shader techniques for passthrough masking.
• Text Mesh Pro for UI and text overlays.