# arcadelab/deepdrr architecture

> Architecture diagram of the GitHub repository arcadelab/deepdrr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arcadelab/deepdrr
- Repository: https://github.com/arcadelab/deepdrr

## Overview

##### Project Type and Purpose
- Type: Scientific simulation library / toolkit.
- Purpose: Generate realistic radiographs and fluoroscopy images from volumetric CT data at training-set scale using a mix of analytic models, machine-learning-based segmentation, and Monte Carlo GPU kernels.

##### File Structure
- Top-level:
– `deepdrr/` – core Python packages
– `example_projector.py`, `deepdrr_demo.ipynb` – usage examples
– `docs/` – documentation sources
– `tests/` – unit tests
– `setup.py`, `environment.yml`, `requirements.txt` – build/install
- Inside `deepdrr/`:
– `geo/` – geometry primitives (rays, segments, hyperplanes)
– `device/` – C-arm definitions (MobileCArm, SimpleDevice)
– `projector/` – projection engine (analytic & CUDA kernels, MC-GPU data)
– `annotations/` – helper classes for fiducials & line markup
– `instruments/` – tool/implant modeling abstractions
– `network_segmentation.py` – volumetric segmentation (V-Net)
– `load_dicom*.py`, `downsample_tool.py` – DICOM I/O & preprocessing
– `utils/` – I/O, image, mesh, heatmap utilities
– `vis.py` – visualization helpers

##### Extract Architectural Insights from README
- Pipeline steps (see “Method Overview” figure):
1. Load and orient CT volume
2. (Optional) Material decomposition via V-Net segmentation
3. Analytic attenuation projection + noise injection
4. Machine-learning–based scatter estimation or turn off scatter
- Key technologies: Python 3, PyCUDA/CUDA (C++/cu for high-performance kernels), PyTorch compatibility, NumPy, SciPy, Sphinx docs.

##### Main Components & Interactions

A. Entry Points (top layer)
- CLI / Scripts: `example_projector.py`, `deepdrr_demo.ipynb`
- API: Python calls into `deepdrr` modules

B. Core Processing Modules (middle layer)
1. Volume I/O & Preprocessing
– `load_dicom.py` / `load_dicom_tool.py`
– `downsample_tool.py`
2. Geometry & Device Model
– `deepdrr.geo.*` (Ray generation, transforms)
– `deepdrr.device.*` (C-arm pose, isocenter)
3. Material Segmentation
– `network_segmentation.py` (V-Net on GPU)
4. Projection Engine
– Analytic CPU/CUDA code (attenuation)
– GPU Monte Carlo kernels (`project_kernel.cu`, scatter kernels)
– Data tables (mcgpu_* directories) for spectra, scatter, mfp
5. Instrument Modeling
– `deepdrr.instruments.*` + binary tool volumes
6. Utilities & Visualization
– `deepdrr.utils.*`, `vis.py`

C. External Dependencies & Resources (side layer)
- GPU (CUDA driver, GPU memory)
- PyCUDA, PyTorch (optional), NumPy
- Input: CT in DICOM / NIfTI
- Output: 2D radiographs (NumPy arrays, images)
- Documentation / Sphinx, Tests

##### Relationships & Data Flow
– CT file → Volume I/O → Volume object
– Volume object + Device object → Projector init
– Projector call →
· (optional) Segmentation → material maps
· Analytic projection + CUDA kernels → raw projection
· Scatter estimation module → scatter map → add to projection
· Noise injection utilities → final image
– Final image → User script / Visualization

##### Architectural Patterns & Principles
- Modular layered architecture
- Separation of concerns: I/O, geometry, simulation, ML segmentation, utilities
- GPU-accelerated compute kernels isolated under `projector/`
- Extensibility points: add new device models, spectra, tools, segmentation nets

##### Technologies & Frameworks
- Python package structure
- PyCUDA / custom CUDA (.cu) kernels
- PyTorch (for segmentation and seamless integration)
- NumPy, SciPy, Nibabel (volume I/O)
- Sphinx (docs), pytest (tests)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Entry Points
    subgraph "Entry Points"
        direction TB
        ExampleProj["example_projector.py"]:::io
        DemoNotebook["deepdrr_demo.ipynb"]:::io
    end

    %% Core Pipeline
    subgraph "Core Pipeline"
        direction TB

        %% Input & Preprocessing
        subgraph "Volume I/O & Preprocessing" 
            direction TB
            CT["CT Input (DICOM/NIfTI)"]:::ext
            loadDicom["load_dicom.py"]:::io
            loadDicomTool["load_dicom_tool.py"]:::io
            downsample["downsample_tool.py"]:::io
        end

        %% Geometry & Device
        subgraph "Geometry & Device Model"
            direction TB
            GeoDir["deepdrr/geo/"]:::geom
            DeviceDir["deepdrr/device/"]:::geom
            VolumeObj["Volume Object"]:::io
            DeviceObj["Device Object"]:::geom
        end

        %% Material Segmentation
        subgraph "Material Segmentation"
            direction TB
            NetSeg["network_segmentation.py"]:::ml
            Segmentation["segmentation.py"]:::ml
        end

        %% Projection Engine
        subgraph "Projection Engine"
            direction TB
            ProjectorInit["projector.py"]:::gpu
            Analytic["analytic_generators.py"]:::gpu
            CudaKernel["project_kernel.cu"]:::gpu
            ScatterCore["scatter.py"]:::gpu
            Spectral["spectral_data.py"]:::gpu
            GPUData["mcgpu_* data"]:::gpu
            Noise["Noise Injection"]:::gpu
            RawProj["Raw Projection"]:::gpu
        end

        %% Instrument Modeling
        subgraph "Instrument Modeling"
            direction TB
            InstrumentsDir["deepdrr/instruments/"]:::geom
            KWire["kwire.py"]:::geom
            VolumeTool["volume.py"]:::geom
        end

        %% Utilities & Visualization
        subgraph "Utilities & Visualization"
            direction TB
            UtilsDir["deepdrr/utils/"]:::util
            Vis["vis.py"]:::util
            FinalImg["Final Image (NumPy array)"]:::util
        end
    end

    %% External Dependencies
    subgraph "External Dependencies"
        direction TB
        GPU["GPU (CUDA)"]:::ext
        PyCUDA["PyCUDA / PyTorch"]:::ext
        NumPySci["NumPy / SciPy"]:::ext
        DocsTests["Docs (Sphinx) / Tests (pytest)"]:::ext
    end

    %% Connections
    CT -->|"Volume files"| loadDicom
    CT --> loadDicomTool
    CT --> downsample
    loadDicom --> VolumeObj
    loadDicomTool --> VolumeObj
    downsample --> VolumeObj

    VolumeObj -->|"Init"| ProjectorInit
    DeviceDir --> DeviceObj
    DeviceObj -->|"Init"| ProjectorInit

    ProjectorInit --> RawProj
    ProjectorInit --> Analytic
    ProjectorInit --> CudaKernel
    RawProj --> ScatterCore
    ScatterCore --> Noise
    Analytic --> RawProj
    CudaKernel --> RawProj
    GPUData --> CudaKernel
    Spectral --> CudaKernel
    NetSeg --> ProjectorInit
    Segmentation --> ProjectorInit

    Noise --> FinalImg
    FinalImg --> Vis
    FinalImg --> ExampleProj
    FinalImg --> DemoNotebook

    GPU --> CudaKernel
    PyCUDA --> CudaKernel
    NumPySci --> loadDicom
    DocsTests --> ExampleProj
    DocsTests --> DemoNotebook

    InstrumentsDir --> KWire
    InstrumentsDir --> VolumeTool

    %% Click Events
    click ExampleProj "https://github.com/arcadelab/deepdrr/blob/main/example_projector.py"
    click DemoNotebook "https://github.com/arcadelab/deepdrr/blob/main/deepdrr_demo.ipynb"
    click loadDicom "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/load_dicom.py"
    click loadDicomTool "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/load_dicom_tool.py"
    click downsample "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/downsample_tool.py"
    click GeoDir "https://github.com/arcadelab/deepdrr/tree/main/deepdrr/geo/"
    click DeviceDir "https://github.com/arcadelab/deepdrr/tree/main/deepdrr/device/"
    click NetSeg "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/network_segmentation.py"
    click Segmentation "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/segmentation.py"
    click ProjectorInit "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/projector/projector.py"
    click Analytic "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/projector/analytic_generators.py"
    click CudaKernel "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/projector/project_kernel.cu"
    click ScatterCore "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/projector/scatter.py"
    click Spectral "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/projector/spectral_data.py"
    click GPUData "https://github.com/arcadelab/deepdrr/tree/main/deepdrr/projector/"
    click InstrumentsDir "https://github.com/arcadelab/deepdrr/tree/main/deepdrr/instruments/"
    click KWire "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/vol/kwire.py"
    click VolumeTool "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/vol/volume.py"
    click UtilsDir "https://github.com/arcadelab/deepdrr/tree/main/deepdrr/utils/"
    click Vis "https://github.com/arcadelab/deepdrr/blob/main/deepdrr/vis.py"

    %% Styles
    classDef io fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef geom fill:#98FB98,stroke:#333,stroke-width:1px
    classDef gpu fill:#FFA500,stroke:#333,stroke-width:1px
    classDef ml fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef util fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef ext fill:#F0E68C,stroke:#333,stroke-width:1px
```
