How sreejith-nair511/phd_work works
1. System purpose and shape
- This is a research training/evaluation framework for depression detection from clinical interview data.
- The architecture is a pipeline: configuration → dataset loading/preprocessing → modality encoders → fusion → classifier → training/evaluation → visualization/output artifacts.
2. Main orchestration entrypoint
- run_experiments.py is the top-level CLI for launching single experiments or the full ablation suite.
- It delegates to training/experiment_runner.py, which coordinates config loading, dataset construction, model assembly, training, and evaluation.
3. Configuration boundary
- utils/config.py loads YAML configs with inheritance from configs/base_config.yaml.
- configs/experiments/*.yaml defines experiment-specific modality/fusion combinations, so modality selection is a config-level concern rather than a code change.
- utils/seed.py, utils/device.py, and utils/logger.py provide reproducibility, hardware selection, and run logging.
4. Dataset subsystem
- dataset/base_dataset.py defines the abstract dataset contract.
- Concrete loaders are dataset/daic_woz_dataset.py and dataset/modma_dataset.py, corresponding to the two supported data sources described in the README.
- dataset/preprocessing.py handles modality-specific preprocessing, while dataset/collate.py handles batches with missing modalities.
- dataset/dataset_factory.py is the boundary that turns config into DataLoader-ready datasets.
5. Modalities and missing-data handling
- The framework is modality-agnostic across Speech, Text, EEG, and Facial inputs.
- Missing modalities are handled natively: loaders and collate logic can emit None/absent inputs, and downstream encoders/fusion are built to ignore missing streams rather than requiring a separate model variant.
- This is a central architectural feature because it allows any subset of modalities to share the same training/evaluation path.
6. Encoder subsystem
- encoders/base_encoder.py defines the common interface for modality encoders.
- Implementations are split by modality: speech_encoder.py, text_encoder.py, eeg_encoder.py, and facial_encoder.py.
- encoders/encoder_factory.py builds the active encoder set from config, so the model only instantiates encoders for enabled modalities.
- The README indicates these encoders produce fixed-size embeddings, with embedding_dim as a shared contract into fusion.
7. Fusion subsystem
- fusion/base_fusion.py defines the fusion interface.
- Supported strategies are:
- early_fusion.py for concatenation + MLP,
- late_fusion.py for per-modality predictions and aggregation,
- attention_fusion.py for attention over modality tokens,
- cross_modal_fusion.py for pairwise cross-attention and aggregation.
- fusion/fusion_factory.py is the switchpoint for choosing the fusion architecture from config.
- Fusion is the key integration boundary between independent modality encoders and the downstream task head.
8. Core model
- models/multimodal_model.py is the end-to-end model that connects encoders, fusion, and task head.
- models/classifier.py provides the final prediction head, supporting classification or regression-style outputs depending on the experiment target.
- This is the central graph of the system: per-modality features in, fused representation out, depression prediction out.
9. Training subsystem
- training/trainer.py owns the training loop, early stopping, checkpointing, and mixed-precision execution.
- training/losses.py contains task losses such as cross-entropy, label smoothing, and focal loss.
- training/optimizers.py configures optimization behavior, including AdamW, layer-wise learning rates, and schedulers.
- training/experiment_runner.py sits above the trainer and manages multi-experiment execution and ablation runs.
10. Evaluation subsystem
- evaluation/evaluator.py handles inference-time evaluation and report generation.
- evaluation/metrics.py computes the core metrics: accuracy, precision, recall, F1, and ROC AUC.
- Evaluation is run after training and produces both machine-readable outputs and human-readable reports for research comparison.
11. Visualization and research outputs
- visualization/ is a post-processing layer for publication-oriented artifacts.
- It includes training curves, confusion matrices, ROC curves, t-SNE projections of fused embeddings, and attention visualizations.
- These modules consume training/evaluation outputs rather than participating in the core training loop, making them a separate analysis boundary.
12. Data and experiment flow across the repo
- External datasets enter through dataset/*, are normalized into batched modality tensors, flow through selected modality encoders, are merged by the configured fusion module, and are classified by the model head.
- Training artifacts and evaluation summaries then fan out into evaluation/ and visualization/outputs/<exp_name>/.
- The architecture is intentionally modular: dataset, encoder, fusion, and training/evaluation concerns are separated so experiments can be swapped by config rather than code edits.
Main entry points: experiment_runner.py Experiment runner · evaluator.py Evaluator · encoder_factory.py Encoder factory · dataset_factory.py Dataset factory
Components 29
Configuration
Experiment definitions and runtime setup.
- Config loader · Loads base and experiment YAML settings.
utils/config.py - Base config · Shared experiment defaults and inherited settings.
configs/base_config.yaml - Exp configs · Modality and fusion presets for runs.
configs/experiments - Runtime utils · Reproducibility, device selection, and run logging.
utils/seed.py
Data pipeline
Dataset loading, preprocessing, and batching.
- Dataset factory · Builds dataset and dataloader objects from config.
dataset/dataset_factory.py - Dataset contract · Common interface for supported clinical datasets.
dataset/base_dataset.py - DAIC-WOZ · Loader for interview-based depression data.
dataset/daic_woz_dataset.py - MODMA · Loader for the second supported data source.
dataset/modma_dataset.py - Preprocess & collate · Modality preprocessing and missing-modality batching.
dataset/preprocessing.py
Model stack
Modal encoders, fusion, and prediction head.
- Encoder factory · Instantiates enabled modality encoders from config.
encoders/encoder_factory.py - Speech encoder · Produces fixed-size speech embeddings.
encoders/speech_encoder.py - Text encoder · Produces fixed-size text embeddings.
encoders/text_encoder.py - EEG encoder · Produces fixed-size EEG embeddings.
encoders/eeg_encoder.py - Facial encoder · Produces fixed-size facial embeddings.
encoders/facial_encoder.py - Fusion factory · Selects fusion strategy from config.
fusion/fusion_factory.py - Fusion strategies · Early, late, attention, and cross-modal fusion implementations.
fusion/early_fusion.py - Multimodal model · Connects encoders, fusion, and prediction head.
models/multimodal_model.py - Classifier · Final prediction head for classification or regression.
models/classifier.py
Training and eval
Optimization, execution, and reporting.
- Experiment runner · Coordinates config, data, model assembly, and execution.
training/experiment_runner.py - Trainer · Handles optimization, early stopping, checkpointing, and AMP.
training/trainer.py - Losses & optimizers · Loss functions, optimizer setup, and schedulers.
training/losses.py - Evaluator · Runs inference and generates reports.
evaluation/evaluator.py - Metrics · Accuracy, precision, recall, F1, and ROC AUC.
evaluation/metrics.py
Visualization
Post-run analysis and publication artifacts.
- Training curves · Plots loss and metric trajectories.
visualization/training_curves.py - Confusion matrix · Classification error breakdown.
visualization/confusion_matrix.py - ROC curve · ROC visualization for model comparison.
visualization/roc_curve.py - t-SNE plot · Projection of fused embeddings.
visualization/tsne_plot.py - Attention viz · Attention and modality contribution visualizations.
visualization/attention_viz.py
Other components
- Run experiments · Top-level command for single runs and ablation suites.
run_experiments.py
Connections 29
- Run experiments → Config loader: loads · Reads base and experiment-specific settings.
- Run experiments → Experiment runner: starts · Launches a single run or ablation suite.
- Config loader → Base config: inherits · Uses shared defaults as the configuration base.
- Config loader → Exp configs: selects · Loads the chosen experiment preset.
- Experiment runner → Runtime utils: initializes · Sets reproducibility, device, and logging.
- Experiment runner → Dataset factory: builds data · Constructs datasets and dataloaders from config.
- Dataset factory → Dataset contract: contracts · Uses the common dataset interface.
- Dataset factory → DAIC-WOZ: creates · Builds DAIC-WOZ dataset objects when selected.
- Dataset factory → MODMA: creates · Builds MODMA dataset objects when selected.
- DAIC-WOZ → Preprocess & collate: uses · Applies preprocessing and missing-modality collation.
- MODMA → Preprocess & collate: uses · Applies preprocessing and missing-modality collation.
- Experiment runner → Encoder factory: builds model · Creates only the enabled modality encoders.
- Encoder factory → Speech encoder: instantiates · Creates speech encoder when configured.
- Encoder factory → Text encoder: instantiates · Creates text encoder when configured.
- Encoder factory → EEG encoder: instantiates · Creates EEG encoder when configured.
- Encoder factory → Facial encoder: instantiates · Creates facial encoder when configured.
- Experiment runner → Fusion factory: chooses · Selects the fusion strategy from config.
- Fusion factory → Fusion strategies: routes to · Resolves the active fusion implementation.
- Experiment runner → Multimodal model: assembles · Wires encoders, fusion, and head into one model.
- Multimodal model → Classifier: predicts with · Feeds fused representation to the task head.
- Experiment runner → Trainer: trains · Hands off the assembled model and data.
- Trainer → Losses & optimizers: configures · Sets objective, optimizer, and schedule.
- Trainer → Evaluator: evaluates · Runs validation and final testing.
- Evaluator → Metrics: computes · Produces core quantitative results.
- Trainer → Training curves: emits · Provides artifacts for training plots.
- Evaluator → Confusion matrix: emits · Provides artifacts for confusion analysis.
- Evaluator → ROC curve: emits · Provides artifacts for ROC plotting.
- Evaluator → t-SNE plot: emits · Provides fused embeddings for projection plots.
- Evaluator → Attention viz: emits · Provides attention data for visualization.