How ici-bci/eeg2text-cet-mae works
Project type and purpose
• This is a research‐oriented Machine Learning pipeline for EEG‐to‐Text generation, centered on a Contrastive EEG‐Text Masked Autoencoder (CET-MAE).
• It provides end-to-end data ingestion (ZuCo EEG datasets), contrastive pretraining, fine-tuning for text decoding from EEG signals, and evaluation.
Main components
a. Data Layer
• Raw EEG “.mat” files (ZuCo v1 and v2) stored under zuco_dataset/.../Matlab_files
• Preprocessing scripts (dataset_preprocess.sh, data_factory/data2pickle_v*.py)
• Serialized pickled datasets via dataset.py
b. Model Layer
• model_mae.py – defines the masked autoencoder architecture
• Multi_Stream_TransformerEncoder.py – multi‐stream transformer encoder for EEG and text streams
• modules.zip – external modules (unzip per README)
c. Training/Pretraining Layer
• Pretraining scripts (pre_train_eval_cet_mae_later_project_*.py) driven by YAML configs in config/
• Optimizer definitions in optim_new.py
• Bash wrappers (cet_mae_eeg2text*.sh) to launch on specific GPU setups
d. Fine-Tuning/Evaluation Layer
• Fine-tuning scripts (train_decoding_eeg_2_text_cet_mae.py, train_decoding_eeg_2_text_vanilla.py)
• Evaluation scripts (eval_decoding_eeg_2_text_cet_mae.py, eval_decoding_eeg_2_text_vanilla.py, plus specialized eval for BART, vanilla)
• Metric wrappers (hf_metrics.py, hf_metrics_vanilla.py) leveraging Hugging Face metrics
e. Configuration & Environment
• requirements.txt – Python dependencies (PyTorch, Transformers, etc.)
• Conda environment instructions in README
• YAML config files under config/ for experiment parameters
Relationships & Interactions
• Raw EEG data → Preprocessing scripts → Pickled datasets
• Pickled datasets + YAML config → Pretraining script → CET-MAE model → Checkpoints
• Checkpoints + fine-tuning script → EEG-to-Text decoder → Fine-tuned model
• Fine-tuned model + eval script → Metrics module → Performance reports
• Bash scripts orchestrate end-to-end runs on targeted GPU resources
Key Architectural Patterns & Principles
• Modular separation: Data ingestion, preprocessing, model definition, training, evaluation
• Config-driven experiments: YAML files decouple hyperparameters from code
• Reusable components: shared dataset and metrics wrappers across multiple model variants (vanilla, BART, T5, XLM-RoBERTa)
• Shell orchestration for reproducibility and multi-GPU scaling
Technology & Framework Highlights
• Python 3.8, PyTorch, Hugging Face Transformers & metrics
• MATLAB .mat file handling via custom preprocessing pipelines
• YAML for experiment configuration
• Bash for environment setup and job submission