How baovin/ml_merc works
Project type and purpose
• This is a standalone ML training and evaluation project (“ML_MERC”) that performs multi-modal emotion recognition on the CREMA-D dataset.
• Key indicators: data/CREMAD (dataset files + preprocessing script), dataset/ (dataset wrappers), models/ (PyTorch model definitions), main.py (training/evaluation entrypoint), Jupyter notebooks for experimentation.
File structure
• Top-level:
• data/CREMAD: raw CSVs + video_preprocessing.py
• dataset/: classes for loading & batching the data
• models/: backbone networks, fusion modules, old models, SEPTR blocks
• utils/: evaluation metrics, helper functions
• main.py: orchestrates load → train → eval → checkpoint
• ckpt/: saved model checkpoints
• Notebooks (.ipynb): exploratory analysis & ensembling
README
• Minimal, but project name “ML_MERC” and dataset folder confirm purpose: A PyTorch-based emotion recognition pipeline.
Main components & their interactions
a. Data Ingestion & Preprocessing
• CREMA-D CSVs + raw videos → video_preprocessing.py → preprocessed frames/audio → stored back in data/CREMAD
b. Dataset Abstraction
• CramedDataset.py & dataset.py → PyTorch Dataset + DataLoader
c. Model Definition
• backbone.py: feature extraction networks (e.g., CNN, RNN)
• fusion_modules.py: combine audio & visual embeddings
• septr.py / septr_block.py: specialized transformer blocks
• basic_model.py: top-level model wrapper that ties backbone + fusion
d. Training & Evaluation
• main.py:
• loads config/hyperparams
• instantiates Dataset + DataLoader
• instantiates model(s)
• runs training loops (optimizer, loss, backprop)
• periodically saves to ckpt/
• calls utils/evaluation.py for metrics
e. Utilities & Experimentation
• utils/utils.py: logging, checkpoint helpers
• Notebooks: ensemble_pytorch.ipynb, Another_copy_of_CREMA_D.ipynb for model comparisons & ensembling
Architectural patterns & principles
• Pipeline pattern: clear separation between data preprocessing, dataset loading, model definition, and training loops.
• Modular code organization: each concern lives in its own folder.
• Checkpointing strategy: persistent storage of model state for later evaluation/ensemble.