How lthinker/dimond works
Project Type and Purpose
• Type: Open-source scientific tool / deep-learning pipeline for diffusion MRI parameter estimation.
• Purpose: Ingest diffusion MRI volumes, map them through a neural network to model parameters, synthesize data via a forward tensor model, and optimize the network by minimizing reconstruction error.
File Structure
Top-level components:
• experiment_scripts/
– Entry-point CLI scripts (train_imgsubjFT.py) for training & inference.
• config_img/
– INI files defining hyperparameters (network depth M, fully-connected layers N, learning rates, dropout rates, dataset paths).
• dataio.py, utils.py, qtlib.py
– Data ingest, preprocessing, I/O utilities, quality-threshold routines.
• modules.py, modulesunit.py
– Neural-network building blocks: 3×3×3 conv layers, FC layers, ReLU, dropout wrappers.
• loss_functions.py
– MSE loss implementation, masking logic (only compute loss within tissue mask).
• pruning_functions.py
– Model-pruning routines applied post-training or during training.
• pyDKI/
– Domain-specific metrics (dkimetrics.py) and utilities for diffusion kurtosis imaging.
• pydti.py
– Tensor-model forward/backward implementations.
• training.py
– Orchestrator: reads config, instantiates DataLoader → Model → Loss → Optimizer → Checkpointing → Metrics → Pruning.
Extract Architectural Insights from README
• Three-stage pipeline:
a) Mapping (NN parameter estimation)
b) Modeling (synthesizing diffusion data via forward tensor model)
c) Optimization (gradient-based minimization of reconstruction loss)
• The core “engine” is training.py, orchestrated by experiment_scripts.
• External pre-processing (MRtrix’s dwibiascorrect, FSL, NODDI) sits upstream of this pipeline.
Main Components and Their Interactions
a. CLI Layer (experiment_scripts/train_imgsubjFT.py)
– Parses command-line args (config path, subject ID, train vs. eval mode)
– Hands off parameters to Training Orchestrator
b. Configuration Manager (configparser on .ini files)
– Supplies hyperparameters, dataset paths, model specs
c. Data Pipeline
– dataio.py loads NIfTI volumes & b-values/b-vectors
– utils.py & qtlib.py perform optional augmentations, masking
d. Model Builder
– modules.py & modulesunit.py instantiate convolutional and FC layers per config
e. Forward Model
– pydti.py synthesizes diffusion signals from estimated parameters
f. Loss & Metrics
– loss_functions.py computes masked MSE
– pyDKI/dkimetrics.py computes kurtosis metrics for logging
g. Training Engine (training.py)
– Optimizer loop (gradient descent on NN parameters)
– Checkpoint saving, logging, optional pruning (pruning_functions.py)
h. Output Storage
– Checkpoints, logs, synthesized volumes, evaluation metrics written under data/<dataset>/<subject>/<experiment_name>
Architectural Patterns & Design Principles
• Modular Separation of Concerns: I/O, model definition, loss, training, pruning each live in separate modules.
• Plugin-style extensibility: New diffusion models can be added by extending pydti.py or pyDKI utils.
• Configuration-driven execution: .ini files control nearly every aspect without code changes.
• Reproducibility: experiment_scripts + config + checkpointing enforce experiment traceability.