# baovin/ml_merc architecture

> Architecture diagram of the GitHub repository baovin/ml_merc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/baovin/ml_merc
- Repository: https://github.com/baovin/ml_merc

## Overview

##### Project type and purpose
- This is a standalone ML training and evaluation project (“ML_MERC”) that performs multi-modal emotion recognition on the CREMA-D dataset.
- Key indicators: `data/CREMAD` (dataset files + preprocessing script), `dataset/` (dataset wrappers), `models/` (PyTorch model definitions), `main.py` (training/evaluation entrypoint), Jupyter notebooks for experimentation.

##### File structure
- Top-level:
- `data/CREMAD`: raw CSVs + `video_preprocessing.py`
- `dataset/`: classes for loading & batching the data
- `models/`: backbone networks, fusion modules, old models, SEPTR blocks
- `utils/`: evaluation metrics, helper functions
- `main.py`: orchestrates load → train → eval → checkpoint
- `ckpt/`: saved model checkpoints
- Notebooks (`.ipynb`): exploratory analysis & ensembling

##### README
- Minimal, but project name “ML_MERC” and dataset folder confirm purpose: A PyTorch-based emotion recognition pipeline.

##### Main components & their interactions
a. Data Ingestion & Preprocessing
- CREMA-D CSVs + raw videos → `video_preprocessing.py` → preprocessed frames/audio → stored back in `data/CREMAD`
b. Dataset Abstraction
- `CramedDataset.py` & `dataset.py` → PyTorch `Dataset` + `DataLoader`
c. Model Definition
- `backbone.py`: feature extraction networks (e.g., CNN, RNN)
- `fusion_modules.py`: combine audio & visual embeddings
- `septr.py` / `septr_block.py`: specialized transformer blocks
- `basic_model.py`: top-level model wrapper that ties backbone + fusion
d. Training & Evaluation
- `main.py`:
- loads config/hyperparams
- instantiates Dataset + DataLoader
- instantiates model(s)
- runs training loops (optimizer, loss, backprop)
- periodically saves to `ckpt/`
- calls `utils/evaluation.py` for metrics
e. Utilities & Experimentation
- `utils/utils.py`: logging, checkpoint helpers
- Notebooks: `ensemble_pytorch.ipynb`, `Another_copy_of_CREMA_D.ipynb` for model comparisons & ensembling

##### Architectural patterns & principles
- Pipeline pattern: clear separation between data preprocessing, dataset loading, model definition, and training loops.
- Modular code organization: each concern lives in its own folder.
- Checkpointing strategy: persistent storage of model state for later evaluation/ensemble.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Data Storage"
        RawData["Raw & Preprocessed Data"]:::data
    end

    Preprocess["Data Preprocessing Module"]:::compute
    Dataset1["CREMA-D Loader\n(dataset/CramedDataset.py)"]:::compute
    Dataset2["DataLoader Wrapper\n(dataset/dataset.py)"]:::compute

    subgraph "Model Components"
        Backbone["Backbone Definitions\n(models/backbone.py)"]:::compute
        Fusion["Fusion Modules\n(models/fusion_modules.py)"]:::compute
        SEPTR["SEPTR Transformer Blocks"]:::compute
        BasicModel["Top-Level Model Wrapper\n(models/basic_model.py)"]:::compute
        Legacy["Legacy Models\n(models/old_models/)"]:::compute
    end

    TrainEngine["Training & Evaluation Entrypoint\n(main.py)"]:::compute
    Storage["Checkpoint Store\n(ckpt/)"]:::storage
    Eval["Evaluation Metrics & Routines\n(utils/evaluation.py)"]:::compute
    Utils["Utility Helpers\n(utils/utils.py)"]:::compute

    subgraph "Experimentation & Ensembling Notebooks"
        Notebook1["Another_copy_of_CREMA_D.ipynb"]:::data
        Notebook2["ensemble_pytorch.ipynb"]:::data
    end

    RawData -->|"raw videos + CSVs"| Preprocess
    Preprocess -->|"preprocessed frames/audio"| RawData
    RawData -->|"batched tensors"| Dataset1
    Dataset1 --> Dataset2
    Dataset2 -->|"batches"| BasicModel

    Backbone --> BasicModel
    Fusion --> BasicModel
    SEPTR --> BasicModel
    Legacy --> BasicModel

    BasicModel --> TrainEngine
    TrainEngine --> Storage
    TrainEngine --> Utils
    TrainEngine --> Eval

    Storage --> Notebook1
    Storage --> Notebook2
    Eval --> Notebook2

    click Preprocess "https://github.com/baovin/ml_merc/blob/main/data/CREMAD/video_preprocessing.py"
    click RawData "https://github.com/baovin/ml_merc/tree/main/data/CREMAD/"
    click Dataset1 "https://github.com/baovin/ml_merc/blob/main/dataset/CramedDataset.py"
    click Dataset2 "https://github.com/baovin/ml_merc/blob/main/dataset/dataset.py"
    click Backbone "https://github.com/baovin/ml_merc/blob/main/models/backbone.py"
    click Fusion "https://github.com/baovin/ml_merc/blob/main/models/fusion_modules.py"
    click SEPTR "https://github.com/baovin/ml_merc/blob/main/models/septr.py"
    click SEPTR "https://github.com/baovin/ml_merc/blob/main/models/septr_block.py"
    click BasicModel "https://github.com/baovin/ml_merc/blob/main/models/basic_model.py"
    click Legacy "https://github.com/baovin/ml_merc/tree/main/models/old_models/"
    click TrainEngine "https://github.com/baovin/ml_merc/blob/main/main.py"
    click Eval "https://github.com/baovin/ml_merc/blob/main/utils/evaluation.py"
    click Utils "https://github.com/baovin/ml_merc/blob/main/utils/utils.py"
    click Storage "https://github.com/baovin/ml_merc/tree/main/ckpt/"
    click Notebook1 "https://github.com/baovin/ml_merc/blob/main/Another_copy_of_CREMA_D.ipynb"
    click Notebook2 "https://github.com/baovin/ml_merc/blob/main/ensemble_pytorch.ipynb"

    classDef data fill:#e5f5e0,stroke:#4caf50;
    classDef compute fill:#e0f2f1,stroke:#00695c;
    classDef storage fill:#fff3e0,stroke:#fb8c00;
```
