How nburtonii/499-codronev2 works
1. What this repo is
- This is a drone/vehicle simulation platform built on Unreal Engine, with a Python SDK on top and a set of documented simulation contracts.
- The main architectural split is between the simulation runtime (sim/) and the client SDK / user programs (sdk/client/).
2. Simulation runtime boundary
- The Unreal project contains the executable world: maps, assets, vehicle blueprints, sensors, weather, and runtime plugins.
- The key runtime entry point is the Unreal project in sim/, with packaged builds already present under sim/Windows/.
3. Core sim plugin: ProjectAirSim
- sim/Plugins/ProjectAirSim is the main simulation engine layer.
- It owns the runtime logic for:
- robot and environment actors
- sensors such as camera, lidar, radar, and distance sensors
- world state, weather, and time-of-day
- renderer adapters for GIS / BlackShark / procedural mesh workflows
- This plugin is the bridge between Unreal scene content and the external SDK/API surface.
4. Vehicle-specific plugins
- sim/Plugins/Drone and sim/Plugins/Rover provide reusable vehicle assets and behavior-specific content.
- These are mostly model/content boundaries: drone airframes, propellers, landing pads, rover meshes, and their materials/physics assets.
5. Content and scenario layer
- Simulation scenarios are defined by Unreal maps and assets in sim/Content, plus project-specific content in sim/Plugins/*/Content.
- The repo includes arena/classroom/obstacle-course style maps, plus many imported environment assets.
- The simulation can therefore be configured by choosing a map, spawning a robot, and attaching sensors or environment actors.
6. SDK and client-facing API
- sdk/client/projectairsim is the primary Python package used by students and scripts.
- It exposes abstractions for:
- drones, rovers, env actors, world access
- mission planning
- image/lidar/radar utilities
- validation and configuration parsing
- data collection and autonomy helpers
- There is also a lower-level C++ client library under sdk/client/cpp, which mirrors the protocol-facing API.
7. How commands and telemetry flow
- User scripts and tests issue commands through the SDK.
- Those commands travel into the Unreal runtime through the AirSim/ProjectAirSim communication layer.
- The simulator returns telemetry and sensor outputs back to the client.
- That data is then written into the repo’s run artifacts as CSV/JSON files, especially under runs/.
8. Mission/configuration contract
- Simulation behavior is driven heavily by JSONC config files and formal schemas.
- Mission and robot/scene configuration specs are documented in docs/BUILD INFO, docs/ARCHITECTURE, and top-level protocol/spec docs.
- This makes the architecture contract-first: the simulator, SDK, tests, and example scripts all depend on the same config/schema definitions.
9. Logging, metrics, and regression outputs
- The runs/ directory is a major output boundary containing telemetry, command logs, event logs, metrics, and per-run JSON metadata.
- The docs define the expected log schema, metrics schema, sensor ranges, and command behavior.
- This suggests the system is designed for reproducible experiments and regression analysis, not just interactive flying.
10. Testing and golden baselines
- tests/ contains pytest-based regression checks for mission parsing, batch commands, collision/out-of-bounds events, sensor/range columns, and golden mission results.
- tests/golden/ stores baseline outputs used to detect simulator or SDK drift.
- In other words, the test boundary validates both protocol compatibility and simulation output stability.
11. Example and user-script layer
- sdk/client/python/example_user_scripts and examples/ are the main “application” layer for end users.
- These scripts cover:
- basic flight and motion control
- sensors and computer vision
- PX4 SITL/HITL integration
- autonomy and mission execution
- data collection and validation
- Simulink/Matlab and ONNX demos
- They show the intended usage pattern: scripts orchestrate missions against the simulator via the SDK.
12. Optional advanced integrations
- The repo includes integration paths for PX4, Matlab/Simulink, ONNX inference, Azure blob storage for data collection examples, and GIS/urban scenes.
- These are not the core runtime, but they widen the architecture from “simulator + SDK” to an experimentation platform for autonomy and ML workflows.
13. Remote visualization / streaming sidecar
- sim/Samples/PixelStreaming includes Unreal’s pixel-streaming web stack: signalling, frontend, matchmaker, and SFU components.
- This is an optional remote-display/control path for the simulator, separate from the Python SDK control plane.
14. Overall shape to diagram
- A good architecture diagram would show:
- Unreal simulator core at the center
- ProjectAirSim plugin as the runtime bridge
- vehicle/content plugins feeding the simulator
- Python SDK and C++ client libraries as external control clients
- configs/missions flowing into the simulator
- telemetry/logs/runs flowing back out
- tests consuming those logs for regression checks
- optional integrations for PX4, ML inference, and pixel streaming on the periphery
Main entry points: projectairsim Python SDK · ProjectAirSim ProjectAirSim · sim Unreal Sim · runs Run Artifacts
Components 20
Simulator
Unreal runtime, content, and simulation plugins.
- Unreal Sim · Top-level Unreal simulation runtime and packaged build target.
sim - ProjectAirSim · Main simulation engine layer bridging Unreal actors, sensors, world state, and external clients.
sim/Plugins/ProjectAirSim - Drone Assets · Reusable drone content such as airframes, propellers, landing pads, and materials.
sim/Plugins/Drone - Rover Assets · Reusable rover and sports-car content boundary for ground vehicles.
sim/Plugins/Rover - Maps & Content · Simulation maps, imported assets, UI, and scenario content used to assemble worlds.
sim/Content
SDK
Client libraries and user-facing control layer.
- Python SDK · Primary Python client package for driving drones, rovers, world access, and utilities.
sdk/client/projectairsim - C++ Client · Lower-level C++ API mirroring the simulator protocol surface.
sdk/client/cpp - Examples · Example scripts demonstrating flight, sensors, PX4, autonomy, validation, and data collection.
sdk/client/python/example_user_scripts - Mission Configs · Example missions and simulator configuration files consumed by the SDK and simulator.
missions
Contracts
Docs, schemas, and mission/config specifications.
- Specs & Docs · Architecture, protocol, log, metrics, and mission specification documents.
docs - SDK Schemas · Python package schemas for robot and scene configuration validation.
sdk/client/projectairsim/src/projectairsim/schema
Outputs
Run artifacts and regression baselines.
- Run Artifacts · Per-run telemetry, commands, events, metrics, and metadata captured from simulator runs.
runs
Integrations
Optional external systems and sidecars.
- Pixel Streaming · Optional remote visualization and control stack for Unreal streaming.
sim/Samples/PixelStreaming - PX4 Path · PX4 SITL/HITL integration examples and supporting configuration.
sdk/client/projectairsim/tests/px4_test_sitl.py - Matlab/Simulink · Matlab and Simulink demo workflows for control and physics co-simulation.
sdk/client/python/example_user_scripts/simulink - ONNX Serving · ONNX model serving and perception/autonomy helpers within the SDK.
sdk/client/projectairsim/src/projectairsim/autonomy/serving - Azure Data · Optional blob-storage workflow for data collection examples.
sdk/client/python/example_user_scripts/datacollection/azure
Validation
Regression tests and tooling that exercise the stack.
- Regression Tests · Tests covering mission parsing, commands, sensor columns, metrics, and golden drift.
tests - Golden Baselines · Reference outputs used to detect simulator or SDK regressions.
tests/golden - Regression Tools · Scripts for plotting telemetry and running batch or regression workflows.
tools
Connections 26
- Unreal Sim → ProjectAirSim: hosts · The Unreal project loads the core ProjectAirSim plugin.
- Unreal Sim → Maps & Content: packages · Maps and assets define the simulation world.
- Unreal Sim → Drone Assets: includes · Drone content is mounted into the sim project.
- Unreal Sim → Rover Assets: includes · Rover content is mounted into the sim project.
- ProjectAirSim → Maps & Content: drives · Runtime logic instantiates actors, sensors, weather, and scene behavior over content.
- Python SDK → ProjectAirSim: commands · Python clients send control and mission commands into the simulator runtime.
- C++ Client → ProjectAirSim: commands · C++ clients communicate over the same protocol boundary.
- ProjectAirSim → Python SDK: telemetry · Simulator returns sensor data and vehicle state to the Python SDK.
- ProjectAirSim → C++ Client: telemetry · Simulator returns protocol responses and sensor outputs to the C++ client.
- Specs & Docs → Python SDK: defines · Shared protocol, schema, and log contracts shape the Python API.
- Specs & Docs → ProjectAirSim: defines · Runtime behavior follows documented protocol and configuration contracts.
- SDK Schemas → Python SDK: validates · JSONC schemas support SDK-side configuration parsing and validation.
- Mission Configs → Unreal Sim: configures · Mission files select maps, robots, sensors, and scenario settings.
- Examples → Python SDK: uses · Example scripts exercise the Python SDK.
- Examples → Mission Configs: loads · Examples are driven by mission and sim config files.
- ProjectAirSim → Run Artifacts: writes · Simulation executions produce telemetry, command logs, events, and metadata.
- Python SDK → Run Artifacts: collects · Client workflows save artifacts and run metadata.
- Run Artifacts → Regression Tests: asserts · Regression tests validate recorded outputs and schema stability.
- Golden Baselines → Regression Tests: baselines · Golden outputs are compared by the test suite.
- Regression Tools → Run Artifacts: analyzes · Tooling consumes telemetry and run files for plotting and batch execution.
- Regression Tests → Specs & Docs: checks · Tests encode expectations from documented contracts.
- PX4 Path → Python SDK: extends · PX4 examples build on the Python client layer.
- Matlab/Simulink → Python SDK: extends · Matlab/Simulink demos interact through the SDK boundary.
- ONNX Serving → Python SDK: extends · ONNX serving supports perception and autonomy workflows built on the SDK.
- Azure Data → Run Artifacts: exports · Azure examples move collected data into external storage.
- Pixel Streaming → Unreal Sim: streams · Pixel streaming provides an alternate visualization/control path for the simulator.