How ches00/360-sl-metasurface works
This repository implements a learned 360° structured-light system: synthetic scene data and camera calibration feed an end-to-end model that simulates metasurface illumination, renders fisheye images, and estimates depth. The README also describes synthetic and real-capture inference. The graph emphasizes these product stages and the camera, data, and neural-network layers supporting them. Training links are supported by the shown training excerpt; inference and some subsystem connections are documented at a high level, so unverified links are dashed.
Main entry points: train.py Training · renderer.py Active Stereo Renderer · StereoMatching.py Stereo Depth Estimator · test.py Inference
Components 16
Scene Data
- Scene Generator
dataset/Generator.py - Dataset Launcher
dataset/dataset_generator.py - Synthetic Dataset
dataset/dataset.py - EXR Reader
utils/render/openExr.py - Real Capture Dataset
dataset/dataset.py
Optical Simulation
- Camera Geometry
utils/Camera.py - Metasurface Propagation
model/Metasurface.py - Active Stereo Renderer
Image_formation/renderer.py - Image Noise
utils/render/noise.py
Depth Reconstruction
- Stereo Depth Estimator
model/StereoMatching.py - Neural Layers
utils/net/basic_layer.py
Model Workflow
- Training
train.py - Inference
test.py - Run Configuration
utils/ArgParser.py - End-to-End Model
model/e2e.py
Other components
- Researcher
Connections 20
- Researcher → Training: starts training
README.md - Researcher → Inference: starts inference
README.md - Training → Run Configuration: parses options
train.py - Training → Synthetic Dataset: loads batches
train.py - Training → End-to-End Model: trains model
train.py - Training → Metasurface Propagation: initializes phase
train.py - Training → Active Stereo Renderer: sets up rendering
train.py - Training → Stereo Depth Estimator: sets up estimator
train.py - Dataset Launcher → Scene Generator: launches generation
dataset/dataset_generator.py - Scene Generator → Camera Geometry: uses camera
dataset/Generator.py - Synthetic Dataset → EXR Reader: reads depth maps
dataset/dataset.py - Active Stereo Renderer → Metasurface Propagation: propagates pattern
Image_formation/renderer.py - Active Stereo Renderer → Camera Geometry: projects scene points
Image_formation/renderer.py - Active Stereo Renderer → Image Noise: samples noise
Image_formation/renderer.py - Stereo Depth Estimator → Camera Geometry: projects views
model/StereoMatching.py - Stereo Depth Estimator → Neural Layers: builds features
model/StereoMatching.py - End-to-End Model → Active Stereo Renderer: renders images
model/e2e.py - End-to-End Model → Stereo Depth Estimator: estimates depth
model/e2e.py - Inference → Synthetic Dataset: supports synthetic input
README.md - Inference → Real Capture Dataset: supports real captures
README.md