How pvnkmrksk/matrexvr works
Project Type and Purpose
• The project is a Unity-based immersive simulation environment used in neuroscience research. It is designed to create naturalistic virtual reality settings to study collective behaviors (e.g., foraging in locusts) and integrates real-time control, sensor feedback, and data logging.
• It leverages VR hardware (LED panels, multi-camera viewports) plus external control via ZeroMQ (NetMQ library) to achieve closed-loop control and interactive simulations. It is also an open-source resource so that other researchers can extend or customize functionalities.
Main Components
• Unity Engine Core:
– This is the “heart” of the project; it manages scenes, game objects, physics, rendering, asset management (models, textures, prefabs, scenes).
– Represent all key asset folders (Scenes, Prefabs, Scripts, Models, Materials, Plugins) as part of this core.
• Simulation & Scene Components:
– Scenes and Prefabs: These include various environment setups (e.g., StressTest, Swarm, Choice, ControlScene) which simulate different experimental conditions.
– Scripts: Control logic that drives closed-loop control, object rotation (DrumRotator, SinusoidalGrating, Keyboard control, VR camera setup), and sensory input processing.
• Control & Communication Modules:
– Closed-Loop Control Module:
• Managed by scripts like ClosedLoop.cs, DrumRotator.cs, and Keyboard.cs which adjust object positions and rotations in real time.
• Detailed indications for directional control (pitch, yaw, roll) and step-wise rotational functionalities.
– ZMQ Listener and External Communication:
• ZmqListener.cs listens for external pose data via ZeroMQ, enabling integration between external sensor data and in-game object updates.
• NetMQ/ZeroMQ as the external messaging/dependency block interfacing with Unity.
• Data Logging & Analytics Subsystem:
– DataLogger.cs (plus various Scene and Master data logging scripts) captures sensor and object states.
– Shows buffering of log lines, generating CSV.GZ files, and connecting to various GameObjects.
• User Input/Interaction:
– Keyboard.cs: This script maps physical controls to in-game object movement and state toggling.
– Indicate the set of keyboard controls for both translation and closed-loop control toggles.
• External Configurations & Assets:
– StreamingAssets: Contains configuration JSON and Python scripts for permutation sequences or rotation configurations, which inform the simulation parameters.
– Mention additional support modules such as the 6-axis force-torque sensor simulation and 3D printable modules.
Relationships and Interactions
• The Scene & Prefab layer (which presents the visual and physical simulation)
• The Scripts layer (which provide behavior, control, and interactivity)
• Dependencies between the closed-loop control component and external communications (ZMQ listener receives data that drives scripts for object positioning and orientation).
• Keyboard Input module to both the closed-loop control components and the general simulation (indicating how user input can override or mix with external data).
Architectural Patterns and Design Principles
• Modularization: Each distinct subset (control, data logging, sensory input, scene management) is encapsulated—facilitating extensibility and easier troubleshooting.
• Use a layered approach:
• Presentation Layer: Visuals managed by Unity (Scenes, Prefabs, Cameras).
• Control Layer: All scripts that handle input (keyboard, ZMQ, closed-loop control).
• Data Management Layer: Data logging modules and configuration streaming (StreamingAssets).
• Integration of third-party libraries (NetMQ) that bridge external real-time communication with internal simulation mechanics.