How jackiewang9811/cet-mae works
Project type and purpose
• This is a research‐oriented machine-learning pipeline for “Contrastive EEG-Text Masked Autoencoder” pretraining and EEG-to-text decoding.
• It combines data preprocessing of EEG recordings (ZuCo), a contrastive pretraining stage (CET-MAE), followed by fine-tuned EEG-to-text decoding using a BART-based architecture.
File structure
• Top level:
• Multi_Stream_TransformerEncoder.py, model_mae_bart.py → core model definitions (EEG+text transformer, masked autoencoder with BART).
• pre_train_eval_cet_mae_later_project_7575.py → contrastive pretraining driver.
• train_decoding_eeg_2_text_cet_mae.py → fine-tuning driver.
• eval_decoding_eeg_2_text_cet_mae.py → evaluation driver.
• cet_mae_eeg2text_gpu2_7575_bart.sh → wrapper script to launch the full pipeline.
• config/*.yaml → experiment configurations (pretraining, fine-tuning, decoding, evaluation).
• data_factory/ → preprocessing modules converting .mat EEG & text to pickled datasets.
• evaluate_moudle/ → metric calculators (ROUGE, WER).
Extract architecture insights from the README
• Two main phases:
a) Pretraining: contrastive masked autoencoder on EEG-text pairs.
b) EEG-Text Training: fine-tune a CET-MAE + BART model for decoding.
• Data flow: raw .mat files → data_factory → pickled tensors → training scripts.
• Evaluation metrics computed via evaluate_module.
Main components & their relationships
a. Data Ingestion & Preprocessing
• Component: data_factory
• Inputs: ZuCo .mat files (task1-SR, task2-NR, task3-TSR, task2-NR-2.0)
• Outputs: pickled EEG/Text pairs for training/eval
b. Configuration Manager
• Component: config/*.yaml
• Role: parameterize batch sizes, learning rates, dataset paths, model hyperparameters
c. Pretraining Module (Contrastive EEG-Text MAE)
• Script: pre_train_eval_cet_mae_later_project_7575.py
• Model: model_mae_bart.py + Multi_Stream_TransformerEncoder.py
• Process: masked reconstruction + contrastive loss between EEG and text embeddings
d. Fine-tuning Module (EEG-to-Text Decoding)
• Script: train_decoding_eeg_2_text_cet_mae.py
• Model: same encoder + BART decoder
e. Evaluation Module
• Script: eval_decoding_eeg_2_text_cet_mae.py
• Metrics: evaluate_module/rouge.py, evaluate_module/wer.py
f. Orchestration Script
• cet_mae_eeg2text_gpu2_7575_bart.sh ties preprocessing, pretraining, fine-tuning, and evaluation
Architectural patterns & design principles
• Modular ML pipeline: separate data, model, train, eval layers
• Configuration-driven experiments: YAML files control all hyperparameters
• Script wrappers for reproducibility and GPU orchestration
• Clear separation of concerns: preprocessing vs. model logic vs. metrics