# sreejith-nair511/phd_work architecture

> Architecture diagram of the GitHub repository sreejith-nair511/phd_work, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sreejith-nair511/phd_work
- Repository: https://github.com/sreejith-nair511/phd_work

## Overview

1. **System purpose and shape**
   - This is a research training/evaluation framework for depression detection from clinical interview data.
   - The architecture is a pipeline: configuration → dataset loading/preprocessing → modality encoders → fusion → classifier → training/evaluation → visualization/output artifacts.

2. **Main orchestration entrypoint**
   - `run_experiments.py` is the top-level CLI for launching single experiments or the full ablation suite.
   - It delegates to `training/experiment_runner.py`, which coordinates config loading, dataset construction, model assembly, training, and evaluation.

3. **Configuration boundary**
   - `utils/config.py` loads YAML configs with inheritance from `configs/base_config.yaml`.
   - `configs/experiments/*.yaml` defines experiment-specific modality/fusion combinations, so modality selection is a config-level concern rather than a code change.
   - `utils/seed.py`, `utils/device.py`, and `utils/logger.py` provide reproducibility, hardware selection, and run logging.

4. **Dataset subsystem**
   - `dataset/base_dataset.py` defines the abstract dataset contract.
   - Concrete loaders are `dataset/daic_woz_dataset.py` and `dataset/modma_dataset.py`, corresponding to the two supported data sources described in the README.
   - `dataset/preprocessing.py` handles modality-specific preprocessing, while `dataset/collate.py` handles batches with missing modalities.
   - `dataset/dataset_factory.py` is the boundary that turns config into DataLoader-ready datasets.

5. **Modalities and missing-data handling**
   - The framework is modality-agnostic across Speech, Text, EEG, and Facial inputs.
   - Missing modalities are handled natively: loaders and collate logic can emit `None`/absent inputs, and downstream encoders/fusion are built to ignore missing streams rather than requiring a separate model variant.
   - This is a central architectural feature because it allows any subset of modalities to share the same training/evaluation path.

6. **Encoder subsystem**
   - `encoders/base_encoder.py` defines the common interface for modality encoders.
   - Implementations are split by modality: `speech_encoder.py`, `text_encoder.py`, `eeg_encoder.py`, and `facial_encoder.py`.
   - `encoders/encoder_factory.py` builds the active encoder set from config, so the model only instantiates encoders for enabled modalities.
   - The README indicates these encoders produce fixed-size embeddings, with `embedding_dim` as a shared contract into fusion.

7. **Fusion subsystem**
   - `fusion/base_fusion.py` defines the fusion interface.
   - Supported strategies are:
     - `early_fusion.py` for concatenation + MLP,
     - `late_fusion.py` for per-modality predictions and aggregation,
     - `attention_fusion.py` for attention over modality tokens,
     - `cross_modal_fusion.py` for pairwise cross-attention and aggregation.
   - `fusion/fusion_factory.py` is the switchpoint for choosing the fusion architecture from config.
   - Fusion is the key integration boundary between independent modality encoders and the downstream task head.

8. **Core model**
   - `models/multimodal_model.py` is the end-to-end model that connects encoders, fusion, and task head.
   - `models/classifier.py` provides the final prediction head, supporting classification or regression-style outputs depending on the experiment target.
   - This is the central graph of the system: per-modality features in, fused representation out, depression prediction out.

9. **Training subsystem**
   - `training/trainer.py` owns the training loop, early stopping, checkpointing, and mixed-precision execution.
   - `training/losses.py` contains task losses such as cross-entropy, label smoothing, and focal loss.
   - `training/optimizers.py` configures optimization behavior, including AdamW, layer-wise learning rates, and schedulers.
   - `training/experiment_runner.py` sits above the trainer and manages multi-experiment execution and ablation runs.

10. **Evaluation subsystem**
    - `evaluation/evaluator.py` handles inference-time evaluation and report generation.
    - `evaluation/metrics.py` computes the core metrics: accuracy, precision, recall, F1, and ROC AUC.
    - Evaluation is run after training and produces both machine-readable outputs and human-readable reports for research comparison.

11. **Visualization and research outputs**
    - `visualization/` is a post-processing layer for publication-oriented artifacts.
    - It includes training curves, confusion matrices, ROC curves, t-SNE projections of fused embeddings, and attention visualizations.
    - These modules consume training/evaluation outputs rather than participating in the core training loop, making them a separate analysis boundary.

12. **Data and experiment flow across the repo**
    - External datasets enter through `dataset/*`, are normalized into batched modality tensors, flow through selected modality encoders, are merged by the configured fusion module, and are classified by the model head.
    - Training artifacts and evaluation summaries then fan out into `evaluation/` and `visualization/outputs/<exp_name>/`.
    - The architecture is intentionally modular: dataset, encoder, fusion, and training/evaluation concerns are separated so experiments can be swapped by config rather than code edits.

Main entry points: [`training/experiment_runner.py`](https://github.com/sreejith-nair511/phd_work/blob/main/training/experiment_runner.py) (Experiment runner), [`evaluation/evaluator.py`](https://github.com/sreejith-nair511/phd_work/blob/main/evaluation/evaluator.py) (Evaluator), [`encoders/encoder_factory.py`](https://github.com/sreejith-nair511/phd_work/blob/main/encoders/encoder_factory.py) (Encoder factory), [`dataset/dataset_factory.py`](https://github.com/sreejith-nair511/phd_work/blob/main/dataset/dataset_factory.py) (Dataset factory)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_config["Configuration"]
  node_config_loader["Config loader<br/>yaml config<br/>[config.py]"]
  node_base_config["Base config<br/>defaults<br/>[base_config.yaml]"]
  node_exp_configs["Exp configs<br/>experiment yaml"]
  node_seed_device_logger["Runtime utils<br/>seed/device/logger<br/>[seed.py]"]
end

subgraph group_group_data["Data pipeline"]
  node_dataset_factory["Dataset factory<br/>dataset builder<br/>[dataset_factory.py]"]
  node_base_dataset["Dataset contract<br/>abstract dataset<br/>[base_dataset.py]"]
  node_daic_woz_dataset["DAIC-WOZ<br/>dataset loader"]
  node_modma_dataset["MODMA<br/>dataset loader<br/>[modma_dataset.py]"]
  node_preprocessing_collate["Preprocess & collate<br/>batch prep<br/>[preprocessing.py]"]
end

subgraph group_group_model["Model stack"]
  node_encoder_factory["Encoder factory<br/>encoder builder<br/>[encoder_factory.py]"]
  node_speech_encoder["Speech encoder<br/>modality encoder<br/>[speech_encoder.py]"]
  node_text_encoder["Text encoder<br/>modality encoder<br/>[text_encoder.py]"]
  node_eeg_encoder["EEG encoder<br/>modality encoder<br/>[eeg_encoder.py]"]
  node_facial_encoder["Facial encoder<br/>modality encoder<br/>[facial_encoder.py]"]
  node_fusion_factory["Fusion factory<br/>fusion builder<br/>[fusion_factory.py]"]
  node_fusion_strategies["Fusion strategies<br/>fusion methods<br/>[early_fusion.py]"]
  node_multimodal_model["Multimodal model<br/>core model"]
  node_classifier["Classifier<br/>task head<br/>[classifier.py]"]
end

subgraph group_group_train["Training and eval"]
  node_experiment_runner["Experiment runner<br/>orchestrator"]
  node_trainer["Trainer<br/>training loop<br/>[trainer.py]"]
  node_losses_optimizers["Losses & optimizers<br/>training ops<br/>[losses.py]"]
  node_evaluator["Evaluator<br/>evaluation<br/>[evaluator.py]"]
  node_metrics["Metrics<br/>metric suite<br/>[metrics.py]"]
end

subgraph group_group_viz["Visualization"]
  node_training_curves["Training curves<br/>viz<br/>[training_curves.py]"]
  node_confusion_matrix["Confusion matrix<br/>viz"]
  node_roc_curve["ROC curve<br/>viz<br/>[roc_curve.py]"]
  node_tsne_plot["t-SNE plot<br/>viz<br/>[tsne_plot.py]"]
  node_attention_viz["Attention viz<br/>[attention_viz.py]"]
end

node_run_experiments["Run experiments<br/>cli entrypoint<br/>[run_experiments.py]"]

node_run_experiments -->|"loads"| node_config_loader
node_run_experiments -->|"starts"| node_experiment_runner
node_config_loader -->|"inherits"| node_base_config
node_config_loader -->|"selects"| node_exp_configs
node_experiment_runner -->|"initializes"| node_seed_device_logger
node_experiment_runner -->|"builds data"| node_dataset_factory
node_dataset_factory -->|"contracts"| node_base_dataset
node_dataset_factory -->|"creates"| node_daic_woz_dataset
node_dataset_factory -->|"creates"| node_modma_dataset
node_daic_woz_dataset -->|"uses"| node_preprocessing_collate
node_modma_dataset -->|"uses"| node_preprocessing_collate
node_experiment_runner -->|"builds model"| node_encoder_factory
node_encoder_factory -->|"instantiates"| node_speech_encoder
node_encoder_factory -->|"instantiates"| node_text_encoder
node_encoder_factory -->|"instantiates"| node_eeg_encoder
node_encoder_factory -->|"instantiates"| node_facial_encoder
node_experiment_runner -->|"chooses"| node_fusion_factory
node_fusion_factory -->|"routes to"| node_fusion_strategies
node_experiment_runner -->|"assembles"| node_multimodal_model
node_multimodal_model -->|"predicts with"| node_classifier
node_experiment_runner -->|"trains"| node_trainer
node_trainer -->|"configures"| node_losses_optimizers
node_trainer -->|"evaluates"| node_evaluator
node_evaluator -->|"computes"| node_metrics
node_trainer -.->|"emits"| node_training_curves
node_evaluator -.->|"emits"| node_confusion_matrix
node_evaluator -.->|"emits"| node_roc_curve
node_evaluator -.->|"emits"| node_tsne_plot
node_evaluator -.->|"emits"| node_attention_viz

click node_run_experiments "https://github.com/sreejith-nair511/phd_work/blob/main/run_experiments.py"
click node_config_loader "https://github.com/sreejith-nair511/phd_work/blob/main/utils/config.py"
click node_base_config "https://github.com/sreejith-nair511/phd_work/blob/main/configs/base_config.yaml"
click node_exp_configs "https://github.com/sreejith-nair511/phd_work/tree/main/configs/experiments"
click node_seed_device_logger "https://github.com/sreejith-nair511/phd_work/blob/main/utils/seed.py"
click node_dataset_factory "https://github.com/sreejith-nair511/phd_work/blob/main/dataset/dataset_factory.py"
click node_base_dataset "https://github.com/sreejith-nair511/phd_work/blob/main/dataset/base_dataset.py"
click node_daic_woz_dataset "https://github.com/sreejith-nair511/phd_work/blob/main/dataset/daic_woz_dataset.py"
click node_modma_dataset "https://github.com/sreejith-nair511/phd_work/blob/main/dataset/modma_dataset.py"
click node_preprocessing_collate "https://github.com/sreejith-nair511/phd_work/blob/main/dataset/preprocessing.py"
click node_encoder_factory "https://github.com/sreejith-nair511/phd_work/blob/main/encoders/encoder_factory.py"
click node_speech_encoder "https://github.com/sreejith-nair511/phd_work/blob/main/encoders/speech_encoder.py"
click node_text_encoder "https://github.com/sreejith-nair511/phd_work/blob/main/encoders/text_encoder.py"
click node_eeg_encoder "https://github.com/sreejith-nair511/phd_work/blob/main/encoders/eeg_encoder.py"
click node_facial_encoder "https://github.com/sreejith-nair511/phd_work/blob/main/encoders/facial_encoder.py"
click node_fusion_factory "https://github.com/sreejith-nair511/phd_work/blob/main/fusion/fusion_factory.py"
click node_fusion_strategies "https://github.com/sreejith-nair511/phd_work/blob/main/fusion/early_fusion.py"
click node_multimodal_model "https://github.com/sreejith-nair511/phd_work/blob/main/models/multimodal_model.py"
click node_classifier "https://github.com/sreejith-nair511/phd_work/blob/main/models/classifier.py"
click node_experiment_runner "https://github.com/sreejith-nair511/phd_work/blob/main/training/experiment_runner.py"
click node_trainer "https://github.com/sreejith-nair511/phd_work/blob/main/training/trainer.py"
click node_losses_optimizers "https://github.com/sreejith-nair511/phd_work/blob/main/training/losses.py"
click node_evaluator "https://github.com/sreejith-nair511/phd_work/blob/main/evaluation/evaluator.py"
click node_metrics "https://github.com/sreejith-nair511/phd_work/blob/main/evaluation/metrics.py"
click node_training_curves "https://github.com/sreejith-nair511/phd_work/blob/main/visualization/training_curves.py"
click node_confusion_matrix "https://github.com/sreejith-nair511/phd_work/blob/main/visualization/confusion_matrix.py"
click node_roc_curve "https://github.com/sreejith-nair511/phd_work/blob/main/visualization/roc_curve.py"
click node_tsne_plot "https://github.com/sreejith-nair511/phd_work/blob/main/visualization/tsne_plot.py"
click node_attention_viz "https://github.com/sreejith-nair511/phd_work/blob/main/visualization/attention_viz.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config_loader,node_base_config,node_exp_configs,node_seed_device_logger toneBlue
class node_dataset_factory,node_base_dataset,node_daic_woz_dataset,node_modma_dataset,node_preprocessing_collate toneAmber
class node_encoder_factory,node_speech_encoder,node_text_encoder,node_eeg_encoder,node_facial_encoder,node_fusion_factory,node_fusion_strategies,node_multimodal_model,node_classifier toneMint
class node_experiment_runner,node_trainer,node_losses_optimizers,node_evaluator,node_metrics toneRose
class node_training_curves,node_confusion_matrix,node_roc_curve,node_tsne_plot,node_attention_viz toneIndigo
class node_run_experiments toneNeutral
```

## Components

### Configuration

Experiment definitions and runtime setup.

- [Config loader](https://github.com/sreejith-nair511/phd_work/blob/main/utils/config.py): yaml config · Loads base and experiment YAML settings. · `utils/config.py`
- [Base config](https://github.com/sreejith-nair511/phd_work/blob/main/configs/base_config.yaml): defaults · Shared experiment defaults and inherited settings. · `configs/base_config.yaml`
- [Exp configs](https://github.com/sreejith-nair511/phd_work/tree/main/configs/experiments): experiment yaml · Modality and fusion presets for runs. · `configs/experiments`
- [Runtime utils](https://github.com/sreejith-nair511/phd_work/blob/main/utils/seed.py): seed/device/logger · Reproducibility, device selection, and run logging. · `utils/seed.py`

### Data pipeline

Dataset loading, preprocessing, and batching.

- [Dataset factory](https://github.com/sreejith-nair511/phd_work/blob/main/dataset/dataset_factory.py): dataset builder · Builds dataset and dataloader objects from config. · `dataset/dataset_factory.py`
- [Dataset contract](https://github.com/sreejith-nair511/phd_work/blob/main/dataset/base_dataset.py): abstract dataset · Common interface for supported clinical datasets. · `dataset/base_dataset.py`
- [DAIC-WOZ](https://github.com/sreejith-nair511/phd_work/blob/main/dataset/daic_woz_dataset.py): dataset loader · Loader for interview-based depression data. · `dataset/daic_woz_dataset.py`
- [MODMA](https://github.com/sreejith-nair511/phd_work/blob/main/dataset/modma_dataset.py): dataset loader · Loader for the second supported data source. · `dataset/modma_dataset.py`
- [Preprocess & collate](https://github.com/sreejith-nair511/phd_work/blob/main/dataset/preprocessing.py): batch prep · Modality preprocessing and missing-modality batching. · `dataset/preprocessing.py`

### Model stack

Modal encoders, fusion, and prediction head.

- [Encoder factory](https://github.com/sreejith-nair511/phd_work/blob/main/encoders/encoder_factory.py): encoder builder · Instantiates enabled modality encoders from config. · `encoders/encoder_factory.py`
- [Speech encoder](https://github.com/sreejith-nair511/phd_work/blob/main/encoders/speech_encoder.py): modality encoder · Produces fixed-size speech embeddings. · `encoders/speech_encoder.py`
- [Text encoder](https://github.com/sreejith-nair511/phd_work/blob/main/encoders/text_encoder.py): modality encoder · Produces fixed-size text embeddings. · `encoders/text_encoder.py`
- [EEG encoder](https://github.com/sreejith-nair511/phd_work/blob/main/encoders/eeg_encoder.py): modality encoder · Produces fixed-size EEG embeddings. · `encoders/eeg_encoder.py`
- [Facial encoder](https://github.com/sreejith-nair511/phd_work/blob/main/encoders/facial_encoder.py): modality encoder · Produces fixed-size facial embeddings. · `encoders/facial_encoder.py`
- [Fusion factory](https://github.com/sreejith-nair511/phd_work/blob/main/fusion/fusion_factory.py): fusion builder · Selects fusion strategy from config. · `fusion/fusion_factory.py`
- [Fusion strategies](https://github.com/sreejith-nair511/phd_work/blob/main/fusion/early_fusion.py): fusion methods · Early, late, attention, and cross-modal fusion implementations. · `fusion/early_fusion.py`
- [Multimodal model](https://github.com/sreejith-nair511/phd_work/blob/main/models/multimodal_model.py): core model · Connects encoders, fusion, and prediction head. · `models/multimodal_model.py`
- [Classifier](https://github.com/sreejith-nair511/phd_work/blob/main/models/classifier.py): task head · Final prediction head for classification or regression. · `models/classifier.py`

### Training and eval

Optimization, execution, and reporting.

- [Experiment runner](https://github.com/sreejith-nair511/phd_work/blob/main/training/experiment_runner.py): orchestrator · Coordinates config, data, model assembly, and execution. · `training/experiment_runner.py`
- [Trainer](https://github.com/sreejith-nair511/phd_work/blob/main/training/trainer.py): training loop · Handles optimization, early stopping, checkpointing, and AMP. · `training/trainer.py`
- [Losses & optimizers](https://github.com/sreejith-nair511/phd_work/blob/main/training/losses.py): training ops · Loss functions, optimizer setup, and schedulers. · `training/losses.py`
- [Evaluator](https://github.com/sreejith-nair511/phd_work/blob/main/evaluation/evaluator.py): evaluation · Runs inference and generates reports. · `evaluation/evaluator.py`
- [Metrics](https://github.com/sreejith-nair511/phd_work/blob/main/evaluation/metrics.py): metric suite · Accuracy, precision, recall, F1, and ROC AUC. · `evaluation/metrics.py`

### Visualization

Post-run analysis and publication artifacts.

- [Training curves](https://github.com/sreejith-nair511/phd_work/blob/main/visualization/training_curves.py): viz · Plots loss and metric trajectories. · `visualization/training_curves.py`
- [Confusion matrix](https://github.com/sreejith-nair511/phd_work/blob/main/visualization/confusion_matrix.py): viz · Classification error breakdown. · `visualization/confusion_matrix.py`
- [ROC curve](https://github.com/sreejith-nair511/phd_work/blob/main/visualization/roc_curve.py): viz · ROC visualization for model comparison. · `visualization/roc_curve.py`
- [t-SNE plot](https://github.com/sreejith-nair511/phd_work/blob/main/visualization/tsne_plot.py): viz · Projection of fused embeddings. · `visualization/tsne_plot.py`
- [Attention viz](https://github.com/sreejith-nair511/phd_work/blob/main/visualization/attention_viz.py): viz · Attention and modality contribution visualizations. · `visualization/attention_viz.py`

### Other components

- [Run experiments](https://github.com/sreejith-nair511/phd_work/blob/main/run_experiments.py): cli entrypoint · Top-level command for single runs and ablation suites. · `run_experiments.py`

## Connections

- Run experiments → Config loader: loads · Reads base and experiment-specific settings.
- Run experiments → Experiment runner: starts · Launches a single run or ablation suite.
- Config loader → Base config: inherits · Uses shared defaults as the configuration base.
- Config loader → Exp configs: selects · Loads the chosen experiment preset.
- Experiment runner → Runtime utils: initializes · Sets reproducibility, device, and logging.
- Experiment runner → Dataset factory: builds data · Constructs datasets and dataloaders from config.
- Dataset factory → Dataset contract: contracts · Uses the common dataset interface.
- Dataset factory → DAIC-WOZ: creates · Builds DAIC-WOZ dataset objects when selected.
- Dataset factory → MODMA: creates · Builds MODMA dataset objects when selected.
- DAIC-WOZ → Preprocess & collate: uses · Applies preprocessing and missing-modality collation.
- MODMA → Preprocess & collate: uses · Applies preprocessing and missing-modality collation.
- Experiment runner → Encoder factory: builds model · Creates only the enabled modality encoders.
- Encoder factory → Speech encoder: instantiates · Creates speech encoder when configured.
- Encoder factory → Text encoder: instantiates · Creates text encoder when configured.
- Encoder factory → EEG encoder: instantiates · Creates EEG encoder when configured.
- Encoder factory → Facial encoder: instantiates · Creates facial encoder when configured.
- Experiment runner → Fusion factory: chooses · Selects the fusion strategy from config.
- Fusion factory → Fusion strategies: routes to · Resolves the active fusion implementation.
- Experiment runner → Multimodal model: assembles · Wires encoders, fusion, and head into one model.
- Multimodal model → Classifier: predicts with · Feeds fused representation to the task head.
- Experiment runner → Trainer: trains · Hands off the assembled model and data.
- Trainer → Losses & optimizers: configures · Sets objective, optimizer, and schedule.
- Trainer → Evaluator: evaluates · Runs validation and final testing.
- Evaluator → Metrics: computes · Produces core quantitative results.
- Trainer → Training curves: emits · Provides artifacts for training plots.
- Evaluator → Confusion matrix: emits · Provides artifacts for confusion analysis.
- Evaluator → ROC curve: emits · Provides artifacts for ROC plotting.
- Evaluator → t-SNE plot: emits · Provides fused embeddings for projection plots.
- Evaluator → Attention viz: emits · Provides attention data for visualization.