# oculus-samples/unity-theworldbeyond architecture

> Architecture diagram of the GitHub repository oculus-samples/unity-theworldbeyond, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oculus-samples/unity-theworldbeyond
- Repository: https://github.com/oculus-samples/unity-theworldbeyond

## Overview

This project is a comprehensive Unity sample built for mixed reality on the Meta (Oculus) platform. It demonstrates how to create a dynamic, scene-driven virtual experience that adapts in real time to a user’s physical environment using features such as NavMesh navigation, dynamic object spawning, voice control, passthrough rendering, interaction (including hand tracking via the Interaction SDK) and audio spatialization. The project's purpose is to show how to integrate various Presence Platform features into an immersive experience (“The World Beyond”), and the file tree reflects a typical Unity-based XR experience with an extensive set of assets, scenes, scripts, and supporting metadata.

##### Main Components
- Scene Management & Core Experience:
- TheWorldBeyond.unity is the core scene.
- WorldBeyondManager (from Assets/TheWorldBeyond/Scripts/GameManagement/WorldBeyondManager.cs) acts as the central controller (similar to a main() function) coordinating the entire experience.
- Environment & Dynamic Scene Construction:
- Environment assets and prefabs support the virtual world (Assets/TheWorldBeyond/Environment/…).
- Room and floor generation from the Scene API are managed by classes such as VirtualRoom and WorldBeyondEnvironment.
- NavMesh generation and on-the-fly obstacle setup are key for Oppy’s movement (dynamic obstacles built from user room geometry).
- Character and Interaction Components:
- Oppy: All assets and scripts for the character named “Oppy” (located under Assets/TheWorldBeyond/Scripts/Characters/Oppy) manage its animations, movement, and interactions.
- MultiToy: A unique tool that attaches to a hand/controller (Assets/TheWorldBeyond/Scripts/Toy/MultiToy.cs), ensuring only one instance ever exists.
- Interaction SDK integration: Parts of the system (e.g., OVRInteraction rig, hands behaviors) that provide hand-tracking and direct manipulation components.
- Audio & Spatialization:
- Audio features are managed via the Oculus AudioManager and Spatializer (referenced by ProjectsSettings and Assets/TheWorldBeyond/Audio/WorldBeyondAudioMixer.mixer).
- Scripts under Audio (Assets/TheWorldBeyond/Scripts/Audio) and sound entry prefabs detail how sound is modified based on scene occlusion and room openness.
- Passthrough & Rendering Techniques:
- Techniques include Depth Test (using DepthOnly materials) and custom Shader Masking (all shaders labeled with “Passthrough”).
- These techniques are integrated into scene objects to blend virtual objects with a live video passthrough from the real-world.
- Voice & Wit Integration:
- Voice commands and transcription are integrated through the Voice SDK, handled primarily by WitConnector (Assets/TheWorldBeyond/Scripts/Wit).
- The system reacts to voice inputs by triggering animations or behaviors on Oppy.
- Auxiliary Sample Scenes:
- Several smaller sample scenes (e.g., Passthrough Pet, Passthrough Room, Virtual Frames, Voice Transcription) offer focused demonstrations of specific features. These can be represented as subsystems or modules illustrating different use cases of the overall architecture.

##### Relationships and Interactions
- WorldBeyondManager acts as the orchestrator, initializing the dynamic environment, handling room surface detection (through OVRSceneManager and Scene API), and coordinating the NavMesh, spawning of objects, and passthrough effects.
- Components such as VirtualRoom, WorldBeyondEnvironment, and SceneMesher work together to generate the virtual floor, walls, and obstacle layout based on real-world inputs.
- MultiToy’s single-instance pattern allows dynamic attachment to player controllers; it interacts with hand tracking and Input management.
- The passthrough rendering module (combining shader masking and depth test approaches) directly affects how both the virtual and real environments are composited, linking with the scene rendering workflow.

##### Architectural Patterns & Design Principles
- Modular Design: The system is broken into clear modules (Scene Management, Environment Generation, Character Control, Audio, Passthrough, Voice) which can be understood and modified independently.
- Data Flow: Real-world sensor input (via OVRSceneManager) feeds into environment calculation modules (VirtualRoom, SceneMesher) which then relay updated information (e.g., walkable areas) to the NavMesh and object spawners.
- Event-Driven Components: Changes such as wall openings or voice commands trigger events that are processed by corresponding managers (e.g., AudioManager, WitConnector).

##### Technologies, Frameworks & Libraries
- Unity Engine 2020.3.18f1 or higher.
- Oculus Interaction/Presence Platform.
- Unity NavMesh for dynamic navigation.
- Voice SDK (Wit-based) for voice interactions.
- Oculus AudioManager and Spatializer.
- Custom shader techniques for passthrough masking.
- Text Mesh Pro for UI and text overlays.

## Diagram (Mermaid)

```mermaid
graph TD

    %% Application Logic Layer
    subgraph "Application Logic Layer"
        WBManager["WorldBeyondManager (Main Controller)"]:::applogic
    end

    %% Scene & Environment Layer
    subgraph "Scene & Environment"
        Scene["TheWorldBeyond Scene"]:::scene
        EnvAssets["Environment Assets & Prefabs"]:::environment
        VirtualRoom["Virtual Room (Room Generation)"]:::environment
        WBEnvironment["WorldBeyond Environment (Scene Construction)"]:::environment
        NavMesh["NavMesh Navigation"]:::environment
    end

    %% Character & Interaction Layer
    subgraph "Character & Interaction"
        Oppy["Oppy (Character)"]:::character
        MultiToy["MultiToy (Hand/Tool Interaction)"]:::character
    end

    %% Service Layer
    subgraph "Service Layer"
        AudioManager["Audio Manager"]:::service
        AudioMixer["Audio Mixer"]:::service
        WitConnector["Wit Connector (Voice SDK)"]:::service
    end

    %% Passthrough Rendering Layer
    subgraph "Passthrough Rendering"
        PassthroughShader["Passthrough Shader"]:::passthrough
        DepthOnly["DepthOnly Material"]:::passthrough
    end

    %% Auxiliary Sample Scenes Layer
    subgraph "Auxiliary Sample Scenes"
        PassthroughPet["Passthrough Pet"]:::samples
        PassthroughRoom["Passthrough Room"]:::samples
        VirtualFrames["Virtual Frames"]:::samples
        VoiceTranscription["Voice Transcription"]:::samples
    end

    %% Connections from Application Logic to Scene & Environment
    WBManager -->|"initEnvironment"| EnvAssets
    WBManager -->|"configureRoom"| VirtualRoom
    WBManager -->|"setupScene"| WBEnvironment
    WBManager -->|"setupNavMesh"| NavMesh
    EnvAssets -->|"provides assets for"| Scene

    %% Connections to Character & Interaction
    WBManager -->|"controls"| Oppy
    MultiToy -->|"attaches to"| Oppy

    %% Service Layer interactions
    EnvAssets -->|"triggers audio updates"| AudioManager
    EnvAssets -->|"triggers audio updates"| AudioMixer
    WitConnector -->|"voice commands"| Oppy

    %% Passthrough Rendering interactions
    EnvAssets -->|"applies shader masking"| PassthroughShader
    EnvAssets -->|"applies depth testing"| DepthOnly

    %% Auxiliary Sample Scenes activation
    WBManager -->|"demo launch"| PassthroughPet
    WBManager -->|"demo launch"| PassthroughRoom
    WBManager -->|"demo launch"| VirtualFrames
    WBManager -->|"demo launch"| VoiceTranscription

    %% Click events (Component Mapping)
    click Scene "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scenes/TheWorldBeyond.unity"
    click WBManager "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scripts/GameManagement/WorldBeyondManager.cs"
    click EnvAssets "https://github.com/oculus-samples/unity-theworldbeyond/tree/main/Assets/TheWorldBeyond/Environment"
    click VirtualRoom "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scripts/Environment/RoomEnvironment/VirtualRoom.cs"
    click WBEnvironment "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scripts/Environment/BackgroundEnvironment/WorldBeyondEnvironment.cs"
    click NavMesh "https://github.com/oculus-samples/unity-theworldbeyond/tree/main/Assets/NavMeshComponents"
    click Oppy "https://github.com/oculus-samples/unity-theworldbeyond/tree/main/Assets/TheWorldBeyond/Scripts/Characters/Oppy"
    click MultiToy "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scripts/Toy/MultiToy.cs"
    click AudioManager "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scripts/Audio/AudioManager.cs"
    click AudioMixer "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Audio/WorldBeyondAudioMixer.mixer"
    click PassthroughShader "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Environment/Shaders/PassthroughWall.shader"
    click DepthOnly "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/MultiToy/Materials/DepthOnly.shader"
    click WitConnector "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scripts/Wit/WitConnector.cs"
    click PassthroughPet "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scenes/PassthroughPet.unity"
    click PassthroughRoom "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scenes/PassthroughRoom.unity"
    click VirtualFrames "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scenes/VirtualFrames.unity"
    click VoiceTranscription "https://github.com/oculus-samples/unity-theworldbeyond/blob/main/Assets/TheWorldBeyond/Scenes/VoiceTranscription.unity"

    %% Styles
    classDef applogic fill:#DDEBF7,stroke:#1F4E78,stroke-width:2px;
    classDef scene fill:#FCE4D6,stroke:#953734,stroke-width:2px;
    classDef environment fill:#E2EFDA,stroke:#38761D,stroke-width:2px;
    classDef character fill:#FCE5CD,stroke:#BF9000,stroke-width:2px;
    classDef service fill:#E4DFEC,stroke:#674EA7,stroke-width:2px;
    classDef passthrough fill:#FFF2CC,stroke:#BF9000,stroke-width:2px;
    classDef samples fill:#D9E1F2,stroke:#1155CC,stroke-width:2px;
```
