# jackiewang9811/cet-mae architecture

> Architecture diagram of the GitHub repository jackiewang9811/cet-mae, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jackiewang9811/cet-mae
- Repository: https://github.com/jackiewang9811/cet-mae

## Overview

##### Project type and purpose
- This is a research‐oriented machine-learning pipeline for “Contrastive EEG-Text Masked Autoencoder” pretraining and EEG-to-text decoding.
- It combines data preprocessing of EEG recordings (ZuCo), a contrastive pretraining stage (CET-MAE), followed by fine-tuned EEG-to-text decoding using a BART-based architecture.

##### File structure
- Top level:
- Multi_Stream_TransformerEncoder.py, model_mae_bart.py → core model definitions (EEG+text transformer, masked autoencoder with BART).
- pre_train_eval_cet_mae_later_project_7575.py → contrastive pretraining driver.
- train_decoding_eeg_2_text_cet_mae.py → fine-tuning driver.
- eval_decoding_eeg_2_text_cet_mae.py → evaluation driver.
- cet_mae_eeg2text_gpu2_7575_bart.sh → wrapper script to launch the full pipeline.
- config/*.yaml → experiment configurations (pretraining, fine-tuning, decoding, evaluation).
- data_factory/ → preprocessing modules converting .mat EEG & text to pickled datasets.
- evaluate_moudle/ → metric calculators (ROUGE, WER).

##### Extract architecture insights from the README
- Two main phases:
a) Pretraining: contrastive masked autoencoder on EEG-text pairs.
b) EEG-Text Training: fine-tune a CET-MAE + BART model for decoding.
- Data flow: raw .mat files → data_factory → pickled tensors → training scripts.
- Evaluation metrics computed via evaluate_module.

##### Main components & their relationships
a. Data Ingestion & Preprocessing
- Component: data_factory
- Inputs: ZuCo .mat files (task1-SR, task2-NR, task3-TSR, task2-NR-2.0)
- Outputs: pickled EEG/Text pairs for training/eval
b. Configuration Manager
- Component: config/*.yaml
- Role: parameterize batch sizes, learning rates, dataset paths, model hyperparameters
c. Pretraining Module (Contrastive EEG-Text MAE)
- Script: pre_train_eval_cet_mae_later_project_7575.py
- Model: model_mae_bart.py + Multi_Stream_TransformerEncoder.py
- Process: masked reconstruction + contrastive loss between EEG and text embeddings
d. Fine-tuning Module (EEG-to-Text Decoding)
- Script: train_decoding_eeg_2_text_cet_mae.py
- Model: same encoder + BART decoder
e. Evaluation Module
- Script: eval_decoding_eeg_2_text_cet_mae.py
- Metrics: evaluate_module/rouge.py, evaluate_module/wer.py
f. Orchestration Script
- cet_mae_eeg2text_gpu2_7575_bart.sh ties preprocessing, pretraining, fine-tuning, and evaluation

##### Architectural patterns & design principles
- Modular ML pipeline: separate data, model, train, eval layers
- Configuration-driven experiments: YAML files control all hyperparameters
- Script wrappers for reproducibility and GPU orchestration
- Clear separation of concerns: preprocessing vs. model logic vs. metrics

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Raw Data
    RawData["Raw EEG & Text (.mat files)"]:::data

    %% Data Factory
    subgraph "Data Factory" 
        direction TB
        D1["data2pickle_v1.py"]:::script
        D2["data2pickle_v2.py"]:::script
        D3["data2pickle_TSRv2.py"]:::script
        D4["data_loading_helpers_modified.py"]:::script
    end
    PickledData["Pickled Dataset (.pkl)"]:::data

    %% Configuration Manager
    subgraph "Configuration Manager"
        direction TB
        C1["train_eval_cet_mae_gpu2_bart_3.yaml"]:::config
        C2["train_eval_cet_mae_gpu2_bart_4.yaml"]:::config
        C3["train_eval_cet_mae_gpu2_bart_5.yaml"]:::config
        C4["train_eval_decoding_eeg_text_gpu2_bart_3.yaml"]:::config
        C5["train_eval_decoding_eeg_text_gpu2_bart_4.yaml"]:::config
        C6["train_eval_decoding_eeg_text_gpu2_bart_5.yaml"]:::config
        C7["eval_decoding_eeg_text_3.yaml"]:::config
        C8["eval_decoding_eeg_text_4.yaml"]:::config
        C9["eval_decoding_eeg_text_5.yaml"]:::config
    end

    %% Pretraining Module
    subgraph "Pretraining (Contrastive CET-MAE)" 
        direction TB
        PreScript["pre_train_eval_cet_mae_later_project_7575.py"]:::script
        Encoder["Multi_Stream_TransformerEncoder.py"]:::model
        MAEModel["model_mae_bart.py"]:::model
    end
    PretrainedCheckpoint["Pretrained Model Checkpoint"]:::model

    %% Fine-tuning Module
    subgraph "Fine-tuning (EEG→Text Decoding)"
        direction TB
        FineScript["train_decoding_eeg_2_text_cet_mae.py"]:::script
        Encoder2["Multi_Stream_TransformerEncoder.py"]:::model
        MAEModel2["model_mae_bart.py"]:::model
    end
    FineTunedCheckpoint["Fine-tuned Model Checkpoint"]:::model

    %% Evaluation Module
    subgraph "Evaluation"
        direction TB
        EvalScript["eval_decoding_eeg_2_text_cet_mae.py"]:::script
        ROUGE["rouge.py"]:::metrics
        WER["wer.py"]:::metrics
    end
    Metrics["Evaluation Metrics (ROUGE, WER)"]:::metrics

    %% Orchestration
    Orchestrator["cet_mae_eeg2text_gpu2_7575_bart.sh"]:::script

    %% Connections
    RawData --> D1
    RawData --> D2
    RawData --> D3
    RawData --> D4
    D1 --> PickledData
    D2 --> PickledData
    D3 --> PickledData
    D4 --> PickledData

    PickledData --> PreScript
    C1 --> PreScript
    C2 --> PreScript
    C3 --> PreScript
    PreScript --> Encoder
    PreScript --> MAEModel
    Encoder --> PretrainedCheckpoint
    MAEModel --> PretrainedCheckpoint

    PickledData --> FineScript
    PretrainedCheckpoint --> FineScript
    C4 --> FineScript
    C5 --> FineScript
    C6 --> FineScript
    FineScript --> Encoder2
    FineScript --> MAEModel2
    Encoder2 --> FineTunedCheckpoint
    MAEModel2 --> FineTunedCheckpoint

    FineTunedCheckpoint --> EvalScript
    PickledData --> EvalScript
    C7 --> EvalScript
    C8 --> EvalScript
    C9 --> EvalScript
    EvalScript --> ROUGE
    EvalScript --> WER
    ROUGE --> Metrics
    WER --> Metrics

    Orchestrator --> RawData
    Orchestrator --> PreScript
    Orchestrator --> FineScript
    Orchestrator --> EvalScript

    %% Click Events
    click D1 "https://github.com/jackiewang9811/cet-mae/blob/main/data_factory/data2pickle_v1.py"
    click D2 "https://github.com/jackiewang9811/cet-mae/blob/main/data_factory/data2pickle_v2.py"
    click D3 "https://github.com/jackiewang9811/cet-mae/blob/main/data_factory/data2pickle_TSRv2.py"
    click D4 "https://github.com/jackiewang9811/cet-mae/blob/main/data_factory/data_loading_helpers_modified.py"
    click C1 "https://github.com/jackiewang9811/cet-mae/blob/main/config/train_eval_cet_mae_gpu2_bart_3.yaml"
    click C2 "https://github.com/jackiewang9811/cet-mae/blob/main/config/train_eval_cet_mae_gpu2_bart_4.yaml"
    click C3 "https://github.com/jackiewang9811/cet-mae/blob/main/config/train_eval_cet_mae_gpu2_bart_5.yaml"
    click C4 "https://github.com/jackiewang9811/cet-mae/blob/main/config/train_eval_decoding_eeg_text_gpu2_bart_3.yaml"
    click C5 "https://github.com/jackiewang9811/cet-mae/blob/main/config/train_eval_decoding_eeg_text_gpu2_bart_4.yaml"
    click C6 "https://github.com/jackiewang9811/cet-mae/blob/main/config/train_eval_decoding_eeg_text_gpu2_bart_5.yaml"
    click C7 "https://github.com/jackiewang9811/cet-mae/blob/main/config/eval_decoding_eeg_text_3.yaml"
    click C8 "https://github.com/jackiewang9811/cet-mae/blob/main/config/eval_decoding_eeg_text_4.yaml"
    click C9 "https://github.com/jackiewang9811/cet-mae/blob/main/config/eval_decoding_eeg_text_5.yaml"
    click PreScript "https://github.com/jackiewang9811/cet-mae/blob/main/pre_train_eval_cet_mae_later_project_7575.py"
    click Encoder "https://github.com/jackiewang9811/cet-mae/blob/main/Multi_Stream_TransformerEncoder.py"
    click MAEModel "https://github.com/jackiewang9811/cet-mae/blob/main/model_mae_bart.py"
    click Encoder2 "https://github.com/jackiewang9811/cet-mae/blob/main/Multi_Stream_TransformerEncoder.py"
    click MAEModel2 "https://github.com/jackiewang9811/cet-mae/blob/main/model_mae_bart.py"
    click ROUGE "https://github.com/jackiewang9811/cet-mae/blob/main/evaluate_moudle/rouge.py"
    click WER "https://github.com/jackiewang9811/cet-mae/blob/main/evaluate_moudle/wer.py"
    click Orchestrator "https://github.com/jackiewang9811/cet-mae/blob/main/cet_mae_eeg2text_gpu2_7575_bart.sh"

    %% Styles
    classDef data fill:#E3F2FD,stroke:#90CAF9,stroke-width:2px,shape:circle
    classDef script fill:#FFF3E0,stroke:#FFB74D,stroke-width:2px,shape:round
    classDef config fill:#FFE0B2,stroke:#FB8C00,stroke-width:2px
    classDef model fill:#E8F5E9,stroke:#66BB6A,stroke-width:2px
    classDef metrics fill:#F3E5F5,stroke:#AB47BC,stroke-width:2px
```
