# aim-kannlab/itmt architecture

> Architecture diagram of the GitHub repository aim-kannlab/itmt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aim-kannlab/itmt
- Repository: https://github.com/aim-kannlab/itmt

## Overview

##### Project type and purpose
- This is an academic/research-oriented deep learning pipeline (“Automated Deep Learning TM-segmentation project”) for measuring temporalis muscle thickness (iTMT) from brain MRIs.
- It includes data curation, preprocessing (registration + normalization), two ML models (slice selection via DenseNet, segmentation via U-Net), post-processing (CSA/TMT computation), centile-curve generation (R scripts), and deployment via Docker or Conda.

##### High-level architectural overview
The system can be viewed as the following layers/components:
A. Data Ingestion & Curation
B. Preprocessing
C. Model Training & Evaluation
D. Inference Pipeline (Single-MRI)
E. Post-processing & Analytics
F. Centile Generation & Visualization
G. Deployment & Packaging

##### Component breakdown from the file tree

A. Data Ingestion & Curation
- Directory: data_curation_scripts/
- curate_*.py scripts for assembling MRI datasets by study/cohort
- dicoms_to_nii.py, extractor.py, extract_tgz.py for raw data conversion
- human_validator/ for manual QC pipelines

B. Preprocessing
- Directories: preprocessing/, docker/preprocessing/
- registration_mni.py (Elastix CLI integration for age-matched MNI registration)
- sitk_resample.py (SimpleITK resampling for DenseNet input)
- masks_to_npy.py (generates image+mask .npy for U-Net)
- scripts/preprocess_utils.py orchestrates bulk z-normalization and bias correction

C. Model Training & Evaluation
- Top-level scripts: train_slice_selection.py, test_slice_selection.py
- Top-level scripts: train_segmentation.py, test_segmentation.py
- model/densenet_models/ and model/unet_models/ hold saved weights & test harnesses
- scripts/ contains network definitions (densenet_regression.py, unet.py), generators, loss functions

D. Inference Pipeline (Single-MRI)
- docker/main.py + docker/predict.py orchestrate:
1. Input .nii through mounted volume
2. Preprocessing → slice selection → segmentation
3. TM metrics computation
4. Output .csv and registered images
- compute_population_pred.py for batch inference outside Docker

E. Post-processing & Analytics
- Python: scripts/measure_head_crm.py, generate_pseudolabels.py
- Centile computation in R: R_scripts/centile_curves_*.R

F. Centile Generation & Visualization
- R_scrips/ for generating growth charts (sex-specific percentiles)
- shared_data/percentiles_chart_*.csv used by web front-end

G. Deployment & Packaging
- Dockerfile + .dockerignore in docker/ for single-MRI inference container
- environment.yml and settings.py for Conda-based workflows
- example_notebooks/ and itmt_v2/ contain demo Jupyter pipelines and extended curation

##### Relationships & Data Flow
- Raw MRI sources → Data Curation → Clean NIfTI
↓
- Preprocessing (registration → normalization → format conversion)
↓
- Two parallel ML pipelines:
- DenseNet slice selection → selected slice index
- U-Net segmentation → binary mask
↓
- Post-processing: compute TMT/CSA from mask + slice index
↓
- Store intermediate results (.npy, .nii, .csv) in output/ or shared_data/
↓
- R scripts consume metrics CSV → generate centile curves & charts
↓
- Streamlit web app (external) reads charts + percentiles for visualization

##### Architectural patterns & principles
- Modular pipeline stages: clear separation of curation, preprocessing, modeling, post-processing
- Reproducible environments: Docker container or Conda + YAML
- “Script-oriented” orchestration rather than monolithic codebase
- Data artifacts checkpointed on disk between stages for speed and reusability

- Clear labels: e.g., “Registration Service (Elastix CLI)”

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Ingestion & Curation"
        DC1["dicoms_to_nii.py"]:::data
        DC2["extract_tgz.py"]:::data
        DC3["extractor.py"]:::data
        DC4["human_validator/"]:::data
    end

    %% Preprocessing Layer
    subgraph "Preprocessing"
        PP1["registration_mni.py"]:::compute
        PP2["sitk_resample.py"]:::compute
        PP3["masks_to_npy.py"]:::compute
    end

    Elastix["Elastix CLI"]:::external

    %% Modeling Layer
    subgraph "Model Training & Evaluation"
        MS1["train_slice_selection.py"]:::model
        MS2["test_slice_selection.py"]:::model
        MS3["train_segmentation.py"]:::model
        MS4["test_segmentation.py"]:::model
        subgraph "Networks & Utils"
            MU1["densenet_regression.py"]:::model
            MU2["unet.py"]:::model
            MU3["generators.py"]:::model
            MU4["loss_unet.py"]:::model
        end
        subgraph "Saved Models"
            MM1["densenet_models/"]:::model
            MM2["unet_models/"]:::model
        end
        GPU["GPU Cluster"]:::external
    end

    %% Inference Layer
    subgraph "Inference Pipeline"
        IP1["main.py"]:::compute
        IP2["predict.py"]:::compute
        IP3["compute_population_pred.py"]:::compute
    end
    DockerContainer["Docker Container"]:::deploy

    %% Post-processing Layer
    subgraph "Post-processing & Analytics"
        AN1["measure_head_crm.py"]:::analytics
        AN2["generate_pseudolabels.py"]:::analytics
    end

    %% Centile Layer
    subgraph "Centile Generation & Visualization"
        R1["centile_curves_*.R"]:::analytics
        SD1["percentiles_chart_boys.csv"]:::data
        SD2["percentiles_chart_girls.csv"]:::data
        SD3["percentiles_chart_boys_csa.csv"]:::data
        SD4["percentiles_chart_girls_csa.csv"]:::data
    end

    %% Deployment Layer
    subgraph "Deployment & Packaging"
        DP1["Dockerfile"]:::deploy
        DP2[".dockerignore"]:::deploy
        DP3["environment.yml"]:::deploy
        DP4["settings.py"]:::deploy
    end

    %% Demo & QC Layer
    subgraph "Demo & Manual QC"
        DM1["example_notebooks/"]:::demo
        DM2["itmt_v2/"]:::demo
    end

    %% Data Flow
    RawData(["Raw MRI Sources\n(DICOM, TAR)"]):::data
    RawData -->|ingest| DC1
    RawData --> DC2
    RawData --> DC3
    DC1 --> DC4
    DC2 --> DC4
    DC3 --> DC4
    DC4 -->|clean .nii| PP1

    PP1 -->|z-norm| PP2
    PP2 -->|→ .npy| PP3
    PP1 ---|uses| Elastix
    PP3 -->|features| MS1
    PP3 --> MS3

    MS1 -->|slice idx| IP1
    MS3 -->|mask| IP1
    IP1 --> IP2
    IP2 -->|results| AN1
    IP2 --> AN2
    GPU --- MS1
    GPU --- MS3

    AN1 -->|metrics.csv| R1
    AN2 --> R1
    R1 --> SD1
    R1 --> SD2
    R1 --> SD3
    R1 --> SD4

    SD1 -->|"charts"| DM1
    SD2 --> DM1
    DockerContainer --> IP1
    DP1 -.-> DockerContainer
    DP2 -.-> DockerContainer
    DP3 -.-> DockerContainer
    DP4 -.-> DockerContainer
    DP3 -->|env| PP1
    DM2 --> DC4
    DM2 --> PP1
    DM2 --> MS1
    DM2 --> MS3

    %% Classes
    classDef data fill:#ADD8E6,stroke:#000;
    classDef compute fill:#FFF59D,stroke:#000;
    classDef model fill:#A5D6A7,stroke:#000;
    classDef analytics fill:#FFCC80,stroke:#000;
    classDef deploy fill:#CE93D8,stroke:#000;
    classDef demo fill:#F8BBD0,stroke:#000;
    classDef external fill:#E0E0E0,stroke-dasharray: 2 2;

    %% Click Events
    click DC1 "https://github.com/aim-kannlab/itmt/blob/main/data_curation_scripts/dicoms_to_nii.py"
    click DC2 "https://github.com/aim-kannlab/itmt/blob/main/data_curation_scripts/extract_tgz.py"
    click DC3 "https://github.com/aim-kannlab/itmt/blob/main/data_curation_scripts/extractor.py"
    click DC4 "https://github.com/aim-kannlab/itmt/tree/main/data_curation_scripts/human_validator/"
    click PP1 "https://github.com/aim-kannlab/itmt/blob/main/preprocessing/registration_mni.py"
    click PP2 "https://github.com/aim-kannlab/itmt/blob/main/preprocessing/sitk_resample.py"
    click PP3 "https://github.com/aim-kannlab/itmt/blob/main/preprocessing/masks_to_npy.py"
    click PP1 "https://github.com/aim-kannlab/itmt/blob/main/docker/preprocessing/registration_mni.py"
    click PP2 "https://github.com/aim-kannlab/itmt/blob/main/docker/preprocessing/sitk_resample.py"
    click PP3 "https://github.com/aim-kannlab/itmt/blob/main/docker/preprocessing/masks_to_npy.py"
    click MS1 "https://github.com/aim-kannlab/itmt/blob/main/train_slice_selection.py"
    click MS2 "https://github.com/aim-kannlab/itmt/blob/main/test_slice_selection.py"
    click MS3 "https://github.com/aim-kannlab/itmt/blob/main/train_segmentation.py"
    click MS4 "https://github.com/aim-kannlab/itmt/blob/main/test_segmentation.py"
    click MU1 "https://github.com/aim-kannlab/itmt/blob/main/scripts/densenet_regression.py"
    click MU2 "https://github.com/aim-kannlab/itmt/blob/main/scripts/unet.py"
    click MU3 "https://github.com/aim-kannlab/itmt/blob/main/scripts/generators.py"
    click MU4 "https://github.com/aim-kannlab/itmt/blob/main/scripts/loss_unet.py"
    click MM1 "https://github.com/aim-kannlab/itmt/tree/main/model/densenet_models/"
    click MM2 "https://github.com/aim-kannlab/itmt/tree/main/model/unet_models/"
    click IP1 "https://github.com/aim-kannlab/itmt/blob/main/docker/main.py"
    click IP2 "https://github.com/aim-kannlab/itmt/blob/main/docker/predict.py"
    click IP3 "https://github.com/aim-kannlab/itmt/blob/main/compute_population_pred.py"
    click AN1 "https://github.com/aim-kannlab/itmt/blob/main/scripts/measure_head_crm.py"
    click AN2 "https://github.com/aim-kannlab/itmt/blob/main/scripts/generate_pseudolabels.py"
    click AN1 "https://github.com/aim-kannlab/itmt/blob/main/docker/scripts/measure_head_crm.py"
    click AN2 "https://github.com/aim-kannlab/itmt/blob/main/docker/scripts/generate_pseudolabels.py"
    click R1 "https://github.com/aim-kannlab/itmt/blob/main/R_scrips/centile_curves_*.R"
    click SD1 "https://github.com/aim-kannlab/itmt/blob/main/shared_data/percentiles_chart_boys.csv"
    click SD2 "https://github.com/aim-kannlab/itmt/blob/main/shared_data/percentiles_chart_girls.csv"
    click SD3 "https://github.com/aim-kannlab/itmt/blob/main/shared_data/percentiles_chart_boys_csa.csv"
    click SD4 "https://github.com/aim-kannlab/itmt/blob/main/shared_data/percentiles_chart_girls_csa.csv"
    click DP1 "https://github.com/aim-kannlab/itmt/tree/main/docker/Dockerfile"
    click DP2 "https://github.com/aim-kannlab/itmt/blob/main/docker/.dockerignore"
    click DP3 "https://github.com/aim-kannlab/itmt/blob/main/environment.yml"
    click DP4 "https://github.com/aim-kannlab/itmt/blob/main/settings.py"
    click DM1 "https://github.com/aim-kannlab/itmt/tree/main/example_notebooks/"
    click DM2 "https://github.com/aim-kannlab/itmt/tree/main/itmt_v2/"
```
