# lthinker/dimond architecture

> Architecture diagram of the GitHub repository lthinker/dimond, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lthinker/dimond
- Repository: https://github.com/lthinker/dimond

## Overview

##### Project Type and Purpose
- Type: Open-source scientific tool / deep-learning pipeline for diffusion MRI parameter estimation.
- Purpose: Ingest diffusion MRI volumes, map them through a neural network to model parameters, synthesize data via a forward tensor model, and optimize the network by minimizing reconstruction error.

##### File Structure
Top-level components:
- experiment_scripts/
– Entry-point CLI scripts (train_imgsubjFT.py) for training & inference.
- config_img/
– INI files defining hyperparameters (network depth M, fully-connected layers N, learning rates, dropout rates, dataset paths).
- dataio.py, utils.py, qtlib.py
– Data ingest, preprocessing, I/O utilities, quality-threshold routines.
- modules.py, modulesunit.py
– Neural-network building blocks: 3×3×3 conv layers, FC layers, ReLU, dropout wrappers.
- loss_functions.py
– MSE loss implementation, masking logic (only compute loss within tissue mask).
- pruning_functions.py
– Model-pruning routines applied post-training or during training.
- pyDKI/
– Domain-specific metrics (dkimetrics.py) and utilities for diffusion kurtosis imaging.
- pydti.py
– Tensor-model forward/backward implementations.
- training.py
– Orchestrator: reads config, instantiates DataLoader → Model → Loss → Optimizer → Checkpointing → Metrics → Pruning.

##### Extract Architectural Insights from README
- Three-stage pipeline:
a) Mapping (NN parameter estimation)
b) Modeling (synthesizing diffusion data via forward tensor model)
c) Optimization (gradient-based minimization of reconstruction loss)
- The core “engine” is training.py, orchestrated by experiment_scripts.
- External pre-processing (MRtrix’s dwibiascorrect, FSL, NODDI) sits upstream of this pipeline.

##### Main Components and Their Interactions
a. CLI Layer (experiment_scripts/train_imgsubjFT.py)
– Parses command-line args (config path, subject ID, train vs. eval mode)
– Hands off parameters to Training Orchestrator
b. Configuration Manager (configparser on .ini files)
– Supplies hyperparameters, dataset paths, model specs
c. Data Pipeline
– dataio.py loads NIfTI volumes & b-values/b-vectors
– utils.py & qtlib.py perform optional augmentations, masking
d. Model Builder
– modules.py & modulesunit.py instantiate convolutional and FC layers per config
e. Forward Model
– pydti.py synthesizes diffusion signals from estimated parameters
f. Loss & Metrics
– loss_functions.py computes masked MSE
– pyDKI/dkimetrics.py computes kurtosis metrics for logging
g. Training Engine (training.py)
– Optimizer loop (gradient descent on NN parameters)
– Checkpoint saving, logging, optional pruning (pruning_functions.py)
h. Output Storage
– Checkpoints, logs, synthesized volumes, evaluation metrics written under data/<dataset>/<subject>/<experiment_name>

##### Architectural Patterns & Design Principles
- Modular Separation of Concerns: I/O, model definition, loss, training, pruning each live in separate modules.
- Plugin-style extensibility: New diffusion models can be added by extending pydti.py or pyDKI utils.
- Configuration-driven execution: .ini files control nearly every aspect without code changes.
- Reproducibility: experiment_scripts + config + checkpointing enforce experiment traceability.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Tools
    subgraph "External Preprocessing" 
        direction TB
        MRtrix["MRtrix & FSL"]:::external
    end

    %% CLI Layer
    CLI["CLI Layer\n(train_imgsubjFT.py)"]:::code
    click CLI "https://github.com/lthinker/dimond/blob/main/experiment_scripts/train_imgsubjFT.py"

    %% Configuration
    Config["Configuration Manager\n(.ini files)"]:::code
    click Config "https://github.com/lthinker/dimond/blob/main/config_img/config_10subj-256h5cconv3e0encode32Part2.ini"

    %% Data Pipeline
    subgraph "Data Pipeline" 
        direction TB
        DataIO["Data I/O\n(dataio.py)"]:::code
        Utils["Utilities\n(utils.py)"]:::code
        click Utils "https://github.com/lthinker/dimond/blob/main/utils.py"
        QTlib["Masking & QA\n(qtlib.py)"]:::code
        click DataIO "https://github.com/lthinker/dimond/blob/main/dataio.py"
        click QTlib "https://github.com/lthinker/dimond/blob/main/qtlib.py"
    end

    %% Model Definition
    subgraph "Model Definition Layer"
        direction TB
        Modules["Conv & FC Blocks\n(modules.py)"]:::code
        ModulesUnit["Layer Units\n(modulesunit.py)"]:::code
        click Modules "https://github.com/lthinker/dimond/blob/main/modules.py"
        click ModulesUnit "https://github.com/lthinker/dimond/blob/main/modulesunit.py"
    end

    %% Domain-Specific
    subgraph "Domain Modules"
        direction TB
        ForwardModel["Forward Tensor Model\n(pydti.py)"]:::domain
        Loss["Loss Functions\n(loss_functions.py)"]:::code
        click ForwardModel "https://github.com/lthinker/dimond/blob/main/pydti.py"
        click Loss "https://github.com/lthinker/dimond/blob/main/loss_functions.py"
        DKI_metrics["DKI Metrics\n(pyDKI/dkimetrics.py)"]:::domain
        DKI_utils["pyDKI Utils\n(pyDKI/utils.py)"]:::domain
        click DKI_metrics "https://github.com/lthinker/dimond/blob/main/pyDKI/dkimetrics.py"
        click DKI_utils "https://github.com/lthinker/dimond/blob/main/pyDKI/utils.py"
    end

    %% Training Engine
    subgraph "Training & Orchestration"
        direction TB
        Training["Training Engine\n(training.py)"]:::code
        click Training "https://github.com/lthinker/dimond/blob/main/training.py"
        Pruning["Pruning Routines\n(pruning_functions.py)"]:::code
        click Pruning "https://github.com/lthinker/dimond/blob/main/pruning_functions.py"
    end

    %% Output
    Outputs["Checkpoints & Logs\n(data/<dataset>/...)"]:::external

    %% Core Pipeline Flow
    MRtrix -->|MRI volumes| CLI
    CLI -->|"config path, mode"| Config
    Config --> Training

    Training -->|"loads volumes"| DataIO
    DataIO --> Utils
    Utils --> QTlib
    QTlib --> Training

    Training -->|"builds model"| Modules
    Modules --> ModulesUnit
    ModulesUnit --> Training

    Training -->|"estimated parameters"| ForwardModel
    ForwardModel --> Training

    Training -->|"predicted signals"| Loss
    Loss --> Training
    Training -->|"MSE loss"| DKI_metrics
    DKI_metrics --> Training

    Training --> Pruning
    Pruning --> Training

    Training --> Outputs

    %% Legend
    subgraph "Legend"
        direction TB
        Code["Code Modules"]:::code
        Domain["Domain Modules"]:::domain
        External["External Tools & Data"]:::external
    end

    classDef code fill:#D0E1F9,stroke:#0366d6,color:#000
    classDef domain fill:#C8E6C9,stroke:#2e7d32,color:#000
    classDef external fill:#FFE0B2,stroke:#ef6c00,color:#000
```
