How jthan1218/mimogs works
1. What this repository is
- MIMO-GS is a research/ML codebase for rendering MIMO channels using a 3D Gaussian Splatting formulation.
- The core runtime is Python, with performance-critical rasterization and metric kernels implemented as CUDA/C++ extensions.
2. Top-level execution paths
- train.py is the main training entry point.
- fine_tuning.py is a separate pipeline for adapting an existing model to new data or conditions.
- debug_gaussian.py is a diagnostic tool for inspecting Gaussian state and rendering behavior.
- These scripts orchestrate the same underlying components: argument parsing, scene/data loading, Gaussian model updates, rendering, and loss computation.
3. Configuration and experiment setup
- arguments/ centralizes runtime configuration, likely exposing model, dataset, optimization, and rendering options.
- This package is the boundary between CLI/user-facing settings and the internal training/rendering pipeline.
- The repo is designed to be run in a conda environment defined in environment.yml.
4. Data and scene subsystem
- scene/ contains the data-side architecture:
- dataloader.py handles loading and batching of the MIMO/channel scene data.
- gaussian_model.py defines the scene representation in terms of learnable Gaussian primitives.
- This layer prepares domain data for the renderer and likely maintains camera/view or channel geometry metadata needed for projection and supervision.
5. Learnable scene representation
- The Gaussian model is the main stateful object in the system.
- It is responsible for storing and updating parameters of the Gaussian set used to represent the channel/scene.
- In the architecture diagram, this is the central learnable repository of geometry, appearance/channel attributes, and possibly per-Gaussian covariance/opacities/features.
6. Rendering subsystem
- gaussian_renderer/ is the rendering facade used by training and debugging scripts.
- It sits between the Python training loop and the native rasterization backend.
- Its role is to convert Gaussian model state into rendered outputs suitable for supervision and evaluation.
7. Native acceleration and external CUDA dependencies
- submodules/diff-gaussian-rasterization is the main differentiable renderer backend.
- submodules/simple-knn provides fast neighbor search, likely used for initialization, clustering, densification, or spatial queries.
- These are compiled extensions, not pure-Python dependencies, and form the performance boundary of the system.
- The repo also vendors submodules/fused-ssim, a CUDA implementation of SSIM used for image-quality loss or evaluation.
8. Training and optimization flow
- The training loop typically flows as:
- load arguments,
- construct scene/data loader,
- initialize Gaussian model,
- render predictions through the differentiable rasterizer,
- compare against ground truth via loss functions,
- backpropagate through the renderer into Gaussian parameters.
- This is an end-to-end differentiable pipeline, with gradients flowing from loss metrics back into the Gaussian representation.
9. Losses, metrics, and evaluation
- utils/loss.py contains objective functions used during optimization.
- The presence of fused-ssim suggests image similarity is an important training signal alongside pixel-level or structural losses.
- utils/logger.py likely collects scalar metrics, render previews, and training progress for experiments.
10. General-purpose utilities
- utils/general_utils.py and utils/system_utils.py provide shared helpers for math, tensor handling, filesystem/path management, and environment/device checks.
- These are support layers rather than architectural pillars, but they reduce coupling across training, rendering, and evaluation code.
11. Build and dependency boundaries
- The repo intentionally separates pure Python orchestration from compiled numerical kernels.
- The submodules/ directory contains third-party code that must be built or installed into the environment before training works.
- That makes the architecture split into:
- Python control plane,
- CUDA/C++ compute plane,
- dataset/scene inputs,
- experiment outputs/logs.
12. How to draw the architecture
- A useful diagram would place train.py/fine_tuning.py at the top as orchestrators.
- Below them, show arguments/ feeding configuration into scene/ and gaussian_renderer/.
- Put scene/dataloader.py on the input side, feeding data into scene/gaussian_model.py.
- Put the CUDA submodules under the renderer as acceleration backends.
- Show utils/ as shared support functions and logging/loss infrastructure around the main loop.
Main entry points: train.py train.py · __init__.py renderer · fine_tuning.py fine_tune · gaussian_model.py gaussian model
Components 16
Control plane
CLI entrypoints and runtime configuration
Scene core
Data loading and learnable Gaussian representation
- data loader · Loads and batches MIMO/channel scene data
scene/dataloader.py - gaussian model · Stateful Gaussian scene/channel representation
scene/gaussian_model.py - renderer · Python rendering interface used by scripts
gaussian_renderer/__init__.py
Native backends
CUDA/C++ acceleration and compiled extensions
- rasterizer · Differentiable Gaussian rasterization kernel
submodules/diff-gaussian-rasterization/ext.cpp - cuda raster · Forward and backward rasterization implementation
submodules/diff-gaussian-rasterization/cuda_rasterizer/rasterizer_impl.cu - simple knn · Fast neighbor search extension
submodules/simple-knn/ext.cpp - knn cuda · Spatial query and neighbor search kernels
submodules/simple-knn/simple_knn.cu - fused ssim · CUDA SSIM loss and metric implementation
submodules/fused-ssim/ext.cpp
Support
Losses, logging, and shared utilities
- losses · Training and evaluation loss computation
utils/loss.py - logger · Tracks metrics, previews, and progress
utils/logger.py - general utils · Common math, tensor, and filesystem helpers
utils/general_utils.py - system utils · Device, path, and environment utilities
utils/system_utils.py
Connections 26
- train.py → arguments: reads config
- fine_tune → arguments: reads config
- debug → arguments: reads config
- train.py → data loader: loads data
- train.py → gaussian model: updates state
- train.py → renderer: renders
- train.py → losses: optimizes with
- train.py → logger: logs
- fine_tune → data loader: loads data
- fine_tune → gaussian model: adapts state
- fine_tune → renderer: renders
- fine_tune → losses: optimizes with
- debug → gaussian model: inspects
- debug → renderer: renders
- renderer → rasterizer: calls
- rasterizer → cuda raster: accelerates
- renderer → simple knn: uses
- simple knn → knn cuda: accelerates
- losses → fused ssim: uses
- data loader → gaussian model: provides supervision to
- gaussian model → renderer: feeds parameters to
- general utils → train.py: supports
- general utils → renderer: supports
- system utils → train.py: supports
- system utils → fine_tune: supports
- logger → system utils: writes outputs via