# thomwolf/magic-sand architecture

> Architecture diagram of the GitHub repository thomwolf/magic-sand, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thomwolf/magic-sand
- Repository: https://github.com/thomwolf/magic-sand

## Overview

1. **What this repository is**
   - A desktop openFrameworks application for an augmented-reality sandbox.
   - It sits between a **Kinect depth sensor** and a **projector**, turning measured sand height into projected colors and, optionally, game logic.

2. **Top-level application shell**
   - `src/main.cpp` and `src/ofApp.*` are the entry point and app coordinator.
   - The app is structured around the openFrameworks update/draw loop, with a **normal UI window** and a separate **projector window**.

3. **Two-window rendering model**
   - The projector is treated as a distinct render target, not just another panel in the UI.
   - `main.cpp` wires a projector-window draw callback; the projector-side rendering is isolated so calibration and projection can be controlled independently from the GUI window.

4. **Sensor + calibration subsystem**
   - `src/KinectProjector/*` is the core infrastructure layer.
   - `KinectProjector` owns Kinect acquisition, calibration, ROI selection, base-plane estimation, coordinate transforms, and state tracking.
   - Supporting pieces include `KinectGrabber`, `TemporalFrameFilter`, and `KinectProjectorCalibration`.

5. **Coordinate-system boundary**
   - This subsystem is the main architectural boundary in the repo: it converts between
     - Kinect image coordinates,
     - 3D world coordinates,
     - projector coordinates.
   - The rest of the app consumes these conversions instead of talking to Kinect calibration directly.

6. **Surface visualization pipeline**
   - `src/SandSurfaceRenderer/*` turns depth/elevation data into projected imagery.
   - `ColorMap` and the GLSL shaders under `bin/data/shaders/*` implement the height-to-color mapping and projector-space rendering.
   - There are separate shader sets for GL2 and GL3, so rendering is adapted to the available graphics path.

7. **Game and interaction layer**
   - `src/Games/*` contains gameplay built on top of the sandbox surface.
   - `BoidGameController`, `MapGameController`, `SandboxScoreTracker`, `ReferenceMapHandler`, and `vehicle` implement different sandbox modes and scoring.
   - These controllers depend on the calibrated sand surface and projected output, but they do not own sensor calibration.

8. **Game-specific data**
   - The map game uses external reference assets in `bin/data/mapGame/ReferenceData` and stores results in `bin/data/mapGame/Scores`.
   - Color lookup tables live in `bin/data/colorMaps`, and calibration/runtime settings are persisted in `bin/data/settings/*.xml`.

9. **Runtime data flow**
   - Kinect frames enter the `KinectProjector` layer first.
   - They are filtered, calibrated, and transformed into sand height/elevation data.
   - `SandSurfaceRenderer` visualizes the surface in projector space.
   - Game controllers read the same calibrated terrain data to decide object movement, scoring, and state transitions.

10. **User-interface and setup boundary**
   - The app starts in a setup/calibration-oriented mode, then switches into live sandbox mode.
   - UI controls are provided by openFrameworks addons such as `ofxDatGui` and `ofxModal`, while the actual projection output remains separated in the projector window.

11. **Platform and build model**
   - This is a cross-platform native C++ project with build files for Visual Studio, Xcode, Qbs, and Make.
   - The repo is meant to be built against openFrameworks 0.9.3 and its addons, rather than as a standalone library or service.

12. **Embedded third-party code**
   - `src/KinectProjector/libs/dlib` vendors a substantial subset of dlib, indicating the project relies on local numeric/geometry utilities without an external package manager.
   - Most other dependencies are openFrameworks addons rather than separate services or networked components.

Main entry points: [`src/KinectProjector/KinectProjector.h`](https://github.com/thomwolf/magic-sand/blob/master/src/KinectProjector/KinectProjector.h) (KinectProjector), [`src/SandSurfaceRenderer/SandSurfaceRenderer.h`](https://github.com/thomwolf/magic-sand/blob/master/src/SandSurfaceRenderer/SandSurfaceRenderer.h) (SandSurfaceRenderer), [`src/ofApp.h`](https://github.com/thomwolf/magic-sand/blob/master/src/ofApp.h) (ofApp), [`src/Games/BoidGameController.h`](https://github.com/thomwolf/magic-sand/blob/master/src/Games/BoidGameController.h) (Boid Game)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["App Shell"]
  node_main["main.cpp<br/>entry point<br/>[main.cpp]"]
  node_ofapp["ofApp<br/>app coordinator<br/>[ofApp.h]"]
end

subgraph group_sensor["Sensor Boundary"]
  node_kinectprojector["KinectProjector<br/>coordination core<br/>[KinectProjector.h]"]
  node_kgrabber["Grabber<br/>sensor input<br/>[KinectGrabber.h]"]
  node_framefilter["Frame Filter<br/>temporal filter"]
  node_calibration["Calibration<br/>transform model"]
  node_dlib["dlib vendored<br/>embedded library"]
end

subgraph group_render["Surface Rendering"]
  node_sandrenderer["SandSurfaceRenderer<br/>render pipeline"]
  node_colormap["ColorMap<br/>lookup mapping<br/>[ColorMap.h]"]
  node_shadergl2["GL2 shaders<br/>shader set"]
  node_shadergl3["GL3 shaders<br/>shader set"]
end

subgraph group_games["Games"]
  node_boidgame["Boid Game<br/>game mode"]
  node_mapgame["Map Game<br/>game mode"]
  node_scoretracker["Score Tracker<br/>scoring"]
  node_refmap["Reference Maps<br/>reference data"]
  node_vehicle(("Vehicle<br/>game entity<br/>[vehicle.h]"))
end

subgraph group_assets["Runtime Assets"]
  node_settings["Settings<br/>runtime config"]
  node_maps["Map Assets<br/>reference assets"]
  node_shaders["Shader Assets<br/>runtime graphics assets"]
end

subgraph group_build["Build Targets"]
  node_buildfiles["Build Files<br/>project configs<br/>[Magic-Sand.sln]"]
end

node_main -->|"launches"| node_ofapp
node_main -->|"draw callback"| node_sandrenderer
node_ofapp -->|"owns"| node_kinectprojector
node_ofapp -->|"updates"| node_sandrenderer
node_ofapp -->|"switches mode"| node_boidgame
node_ofapp -->|"switches mode"| node_mapgame
node_kinectprojector -->|"acquires"| node_kgrabber
node_kinectprojector -->|"filters"| node_framefilter
node_kinectprojector -->|"transforms"| node_calibration
node_kinectprojector -.->|"uses"| node_dlib
node_kinectprojector -->|"provides terrain"| node_sandrenderer
node_kinectprojector -->|"provides terrain"| node_boidgame
node_kinectprojector -->|"provides terrain"| node_mapgame
node_sandrenderer -->|"maps colors"| node_colormap
node_sandrenderer -.->|"uses"| node_shadergl2
node_sandrenderer -.->|"uses"| node_shadergl3
node_sandrenderer -->|"loads config"| node_settings
node_calibration -->|"persists"| node_settings
node_boidgame -->|"updates entities"| node_vehicle
node_mapgame -->|"loads references"| node_refmap
node_scoretracker -->|"stores scores"| node_maps
node_refmap -->|"reads assets"| node_maps
node_sandrenderer -->|"loads assets"| node_shaders
node_ofapp -->|"loads ui state"| node_settings
node_ofapp -.->|"builds with"| node_buildfiles

click node_main "https://github.com/thomwolf/magic-sand/blob/master/src/main.cpp"
click node_ofapp "https://github.com/thomwolf/magic-sand/blob/master/src/ofApp.h"
click node_kinectprojector "https://github.com/thomwolf/magic-sand/blob/master/src/KinectProjector/KinectProjector.h"
click node_kgrabber "https://github.com/thomwolf/magic-sand/blob/master/src/KinectProjector/KinectGrabber.h"
click node_framefilter "https://github.com/thomwolf/magic-sand/blob/master/src/KinectProjector/TemporalFrameFilter.h"
click node_calibration "https://github.com/thomwolf/magic-sand/blob/master/src/KinectProjector/KinectProjectorCalibration.h"
click node_sandrenderer "https://github.com/thomwolf/magic-sand/blob/master/src/SandSurfaceRenderer/SandSurfaceRenderer.h"
click node_colormap "https://github.com/thomwolf/magic-sand/blob/master/src/SandSurfaceRenderer/ColorMap.h"
click node_shadergl2 "https://github.com/thomwolf/magic-sand/tree/master/bin/data/shaders/shadersGL2"
click node_shadergl3 "https://github.com/thomwolf/magic-sand/tree/master/bin/data/shaders/shadersGL3"
click node_boidgame "https://github.com/thomwolf/magic-sand/blob/master/src/Games/BoidGameController.h"
click node_mapgame "https://github.com/thomwolf/magic-sand/blob/master/src/Games/MapGameController.h"
click node_scoretracker "https://github.com/thomwolf/magic-sand/blob/master/src/Games/SandboxScoreTracker.h"
click node_refmap "https://github.com/thomwolf/magic-sand/blob/master/src/Games/ReferenceMapHandler.h"
click node_vehicle "https://github.com/thomwolf/magic-sand/blob/master/src/Games/vehicle.h"
click node_settings "https://github.com/thomwolf/magic-sand/tree/master/bin/data/settings"
click node_maps "https://github.com/thomwolf/magic-sand/tree/master/bin/data/mapGame"
click node_shaders "https://github.com/thomwolf/magic-sand/tree/master/bin/data/shaders"
click node_buildfiles "https://github.com/thomwolf/magic-sand/blob/master/Magic-Sand.sln"
click node_dlib "https://github.com/thomwolf/magic-sand/tree/master/src/KinectProjector/libs/dlib"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_ofapp toneBlue
class node_kinectprojector,node_kgrabber,node_framefilter,node_calibration,node_dlib toneAmber
class node_sandrenderer,node_colormap,node_shadergl2,node_shadergl3 toneMint
class node_boidgame,node_mapgame,node_scoretracker,node_refmap,node_vehicle toneRose
class node_settings,node_maps,node_shaders toneIndigo
class node_buildfiles toneTeal
```

## Components

### App Shell

openFrameworks entry point, window orchestration, and runtime mode switching.

- [main.cpp](https://github.com/thomwolf/magic-sand/blob/master/src/main.cpp): entry point · Starts the app and wires the projector-window callback. · `src/main.cpp`
- [ofApp](https://github.com/thomwolf/magic-sand/blob/master/src/ofApp.h): app coordinator · Coordinates update/draw flow, UI state, and mode switching. · `src/ofApp.h`

### Sensor Boundary

Kinect acquisition, calibration, filtering, and coordinate transforms.

- [KinectProjector](https://github.com/thomwolf/magic-sand/blob/master/src/KinectProjector/KinectProjector.h): coordination core · Owns acquisition, calibration, transforms, ROI, and state tracking. · `src/KinectProjector/KinectProjector.h`
- [Grabber](https://github.com/thomwolf/magic-sand/blob/master/src/KinectProjector/KinectGrabber.h): sensor input · Wraps Kinect frame acquisition. · `src/KinectProjector/KinectGrabber.h`
- [Frame Filter](https://github.com/thomwolf/magic-sand/blob/master/src/KinectProjector/TemporalFrameFilter.h): temporal filter · Smooths incoming depth frames before downstream processing. · `src/KinectProjector/TemporalFrameFilter.h`
- [Calibration](https://github.com/thomwolf/magic-sand/blob/master/src/KinectProjector/KinectProjectorCalibration.h): transform model · Stores calibration data and coordinate transforms between Kinect, world, and projector spaces. · `src/KinectProjector/KinectProjectorCalibration.h`
- [dlib vendored](https://github.com/thomwolf/magic-sand/tree/master/src/KinectProjector/libs/dlib): embedded library · Local numeric and geometry utilities bundled under KinectProjector. · `src/KinectProjector/libs/dlib`

### Surface Rendering

Height-to-color mapping and projector-space rendering pipeline.

- [SandSurfaceRenderer](https://github.com/thomwolf/magic-sand/blob/master/src/SandSurfaceRenderer/SandSurfaceRenderer.h): render pipeline · Builds the projected surface visualization from elevation data. · `src/SandSurfaceRenderer/SandSurfaceRenderer.h`
- [ColorMap](https://github.com/thomwolf/magic-sand/blob/master/src/SandSurfaceRenderer/ColorMap.h): lookup mapping · Maps height/elevation values to display colors. · `src/SandSurfaceRenderer/ColorMap.h`
- [GL2 shaders](https://github.com/thomwolf/magic-sand/tree/master/bin/data/shaders/shadersGL2): shader set · Projector-space rendering path for the GL2 pipeline. · `bin/data/shaders/shadersGL2`
- [GL3 shaders](https://github.com/thomwolf/magic-sand/tree/master/bin/data/shaders/shadersGL3): shader set · Projector-space rendering path for the GL3 pipeline. · `bin/data/shaders/shadersGL3`

### Games

Sandbox gameplay and scoring built on calibrated surface data.

- [Boid Game](https://github.com/thomwolf/magic-sand/blob/master/src/Games/BoidGameController.h): game mode · Sandbox gameplay controller for boid-style interaction. · `src/Games/BoidGameController.h`
- [Map Game](https://github.com/thomwolf/magic-sand/blob/master/src/Games/MapGameController.h): game mode · Map-based game controller using reference terrain data. · `src/Games/MapGameController.h`
- [Score Tracker](https://github.com/thomwolf/magic-sand/blob/master/src/Games/SandboxScoreTracker.h): scoring · Tracks and persists sandbox game scores. · `src/Games/SandboxScoreTracker.h`
- [Reference Maps](https://github.com/thomwolf/magic-sand/blob/master/src/Games/ReferenceMapHandler.h): reference data · Loads map references and matching settings for the map game. · `src/Games/ReferenceMapHandler.h`
- [Vehicle](https://github.com/thomwolf/magic-sand/blob/master/src/Games/vehicle.h): game entity · Movement/object logic used by sandbox game modes. · `src/Games/vehicle.h`

### Runtime Assets

Persistent settings, shaders, maps, and saved scores used at runtime.

- [Settings](https://github.com/thomwolf/magic-sand/tree/master/bin/data/settings): runtime config · Persisted calibration and renderer settings. · `bin/data/settings`
- [Map Assets](https://github.com/thomwolf/magic-sand/tree/master/bin/data/mapGame): reference assets · Reference data and stored scores for the map game. · `bin/data/mapGame`
- [Shader Assets](https://github.com/thomwolf/magic-sand/tree/master/bin/data/shaders): runtime graphics assets · GL2 and GL3 shader sources used by the surface renderer. · `bin/data/shaders`

### Build Targets

Platform-specific project and build files for native C++ builds.

- [Build Files](https://github.com/thomwolf/magic-sand/blob/master/Magic-Sand.sln): project configs · Native build definitions for Visual Studio, Xcode, Qbs, and Make. · `Magic-Sand.sln`

## Connections

- main.cpp → ofApp: launches · Entry point constructs the application coordinator.
- main.cpp → SandSurfaceRenderer: draw callback · Main wires the projector-side render path.
- ofApp → KinectProjector: owns · The app drives the sensor/calibration boundary.
- ofApp → SandSurfaceRenderer: updates · The app feeds calibrated surface state into rendering.
- ofApp → Boid Game: switches mode · App can activate sandbox game logic.
- ofApp → Map Game: switches mode · App can activate map-game logic.
- KinectProjector → Grabber: acquires · Coordinates Kinect frame capture.
- KinectProjector → Frame Filter: filters · Applies temporal smoothing to sensor data.
- KinectProjector → Calibration: transforms · Maps between Kinect, world, and projector coordinates.
- KinectProjector → dlib vendored: uses · Depends on bundled geometry and matrix utilities.
- KinectProjector → SandSurfaceRenderer: provides terrain · Supplies calibrated elevation/height data to rendering.
- KinectProjector → Boid Game: provides terrain · Game logic reads the same calibrated surface data.
- KinectProjector → Map Game: provides terrain · Map game consumes calibrated surface data.
- SandSurfaceRenderer → ColorMap: maps colors · Converts height values to projected colors.
- SandSurfaceRenderer → GL2 shaders: uses · GL2 rendering path.
- SandSurfaceRenderer → GL3 shaders: uses · GL3 rendering path.
- SandSurfaceRenderer → Settings: loads config · Renderer settings are persisted in XML.
- Calibration → Settings: persists · Calibration data is stored in runtime settings.
- Boid Game → Vehicle: updates entities · Boid gameplay uses vehicle/object motion logic.
- Map Game → Reference Maps: loads references · Map game consumes reference-map assets and settings.
- Score Tracker → Map Assets: stores scores · Score data is saved under map-game runtime assets.
- Reference Maps → Map Assets: reads assets · Reference map handler loads external map data.
- SandSurfaceRenderer → Shader Assets: loads assets · Renderer uses shader sources from data files.
- ofApp → Settings: loads ui state · App starts in setup/calibration mode using persisted settings.
- ofApp → Build Files: builds with · Repository is packaged as native desktop project files.