# neuspeech/eeg-to-text architecture

> Architecture diagram of the GitHub repository neuspeech/eeg-to-text, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/neuspeech/eeg-to-text
- Repository: https://github.com/neuspeech/eeg-to-text

## Overview

##### Project type and purpose
- This is a research-oriented machine-learning experiment pipeline for evaluating and correcting “EEG-to-Text” and zero-shot sentiment classification models. It implements data preparation, model training (decoding and sentiment), evaluation (using corrected generation), and result storage for academic analysis.

##### File structure
- Top-level scripts:
- config.py – global settings (paths, hyperparameters)
- data.py – high-level dataset loading interfaces
- train_decoding.py / train_sentiment_*.py – Python entry points to kick off training
- model_decoding.py / model_sentiment.py – model definitions and wrappers (using Transformers)
- eval_decoding.py / eval_sentiment.py – evaluation code (with corrected model.generate calls)
- util/ – helper modules for constructing and loading datasets (from .mat to pickle), sentiment label extraction, data-loading utilities
- scripts/ – shell scripts orchestrating multi-step pipelines (dataset preparation, training splits, evaluation runs)
- results/ – plain-text outputs of each experiment
- environment.yml – Conda environment specification for reproducibility

##### README for additional insights
- Purpose: correct earlier inadvertent use of teacher forcing, switch to proper generation-based evaluation
- Architecture: single-stage seq2seq (T5-based) EEG-to-Text model and a sentiment classifier
- Workflow overview:
a. raw EEG + text data → preprocessing (util, scripts/prepare_dataset.sh)
b. train decoder (scripts/train_decoding*.sh → train_decoding.py/model_decoding.py)
c. train sentiment baseline/classifier (scripts/train_text_sentiment_classifier.sh, train_sentiment_*.py)
d. evaluate both (scripts/eval_decoding_*.sh → eval_decoding.py; scripts/eval_sentiment_zeroshot_pipeline.sh → eval_sentiment.py)
e. collect results in results/

1) Data Preparation
– util/construct_dataset_*.py
– scripts/prepare_dataset.sh
– Output: preprocessed EEG embeddings, text targets, sentiment labels
2) Model Training
– EEG-to-Text Decoder ([model_decoding.py](#), train_decoding.py, scripts/train_decoding*.sh)
– Sentiment Classifier ([model_sentiment.py](#), train_sentiment_*.py, scripts/train_text_sentiment_classifier.sh)
3) Model Evaluation
– Decoding evaluation (eval_decoding.py, scripts/eval_decoding_*.sh)
– Sentiment evaluation (eval_sentiment.py, scripts/eval_sentiment_zeroshot_pipeline.sh)
4) Results Management
– results/ directory with .txt outcome files
5) Environment & Dependencies
– environment.yml
– Python libraries: PyTorch, HuggingFace Transformers, NumPy, SciPy
- Raw data → Data Preparation → Preprocessed dataset → Training components
- Trained models → Model Evaluation → Prediction outputs → Results Management
- config.py & environment.yml → feed into all components for consistent parameters and dependencies
c. Architectural patterns / design principles:
– Modular scripts for separation of concerns (prepare/train/evaluate)
– Reusable utility functions in util/ for dataset operations
– Shell-script orchestration for reproducible experiment runs and variant sweeps
– Clear separation between decoding and sentiment tasks
d. Key technologies / frameworks:
– Python 3.x, Conda environment (environment.yml)
– PyTorch + HuggingFace Transformers (T5-based model.generate)
– MATLAB-to-Python dataset conversion (util/construct_dataset_mat_to_pickle_v?.py)

##### Detailed yet concise capture
- Component is clearly named and all critical interactions are visible
- External references: original Wang code (for historical context), arXiv/IJCAI papers

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Environment & Configuration
    ENV1["config.py"]:::config
    ENV2["environment.yml"]:::config

    %% Data Preparation
    subgraph "Data Preparation"
        direction TB
        V1["construct_dataset_mat_to_pickle_v1.py"]:::data
        V2["construct_dataset_mat_to_pickle_v2.py"]:::data
        V3["data_loading_helpers_modified.py"]:::data
        V4["get_SST_ternary_dataset.py"]:::data
        V5["get_sentiment_labels.py"]:::data
        PrepSh["scripts/prepare_dataset.sh"]:::data
        DataPy["data.py"]:::data
    end

    %% EEG-to-Text Decoder Training
    subgraph "Decoder Training"
        direction TB
        MD["model_decoding.py"]:::train
        TD["train_decoding.py"]:::train
        TDS1["scripts/train_decoding.sh"]:::train
        TDS2["scripts/train_decoding_1.sh"]:::train
    end

    %% Sentiment Classifier Training
    subgraph "Sentiment Training"
        direction TB
        MS1["model_sentiment.py"]:::train
        MS2["train_sentiment_baseline.py"]:::train
        MS3["train_sentiment_textbased.py"]:::train
        STSh1["scripts/train_text_sentiment_classifier.sh"]:::train
        STSh2["scripts/train_eeg_sentiment_baseline.sh"]:::train
    end

    %% Decoding Evaluation
    subgraph "Decoder Evaluation"
        direction TB
        EDpy["eval_decoding.py"]:::eval
        ED1["scripts/eval_decoding_1.sh"]:::eval
        ED2["scripts/eval_decoding_2.sh"]:::eval
        ED3["scripts/eval_decoding_3.sh"]:::eval
        ED4["scripts/eval_decoding_4.sh"]:::eval
    end

    %% Sentiment Evaluation
    subgraph "Sentiment Evaluation"
        direction TB
        ESpy["eval_sentiment.py"]:::eval
        ES1["scripts/eval_sentiment_zeroshot_pipeline.sh"]:::eval
        ES2["scripts/train_eval_zeroshot_pipeline.sh"]:::eval
    end

    %% Results Management
    Results["results/"]:::results

    %% Flows
    ENV1 --> V1
    ENV1 --> MD
    ENV1 --> MS1
    ENV1 --> EDpy
    ENV1 --> ESpy
    ENV2 --> V1
    ENV2 --> MD
    ENV2 --> MS1
    ENV2 --> EDpy
    ENV2 --> ESpy

    DataPy --> MD
    DataPy --> MS1

    MD --> EDpy
    MS1 --> ESpy

    ED4 --> Results
    ES2 --> Results

    %% Click Events
    click V1 "https://github.com/neuspeech/eeg-to-text/blob/main/util/construct_dataset_mat_to_pickle_v1.py"
    click V2 "https://github.com/neuspeech/eeg-to-text/blob/main/util/construct_dataset_mat_to_pickle_v2.py"
    click V3 "https://github.com/neuspeech/eeg-to-text/blob/main/util/data_loading_helpers_modified.py"
    click V4 "https://github.com/neuspeech/eeg-to-text/blob/main/util/get_SST_ternary_dataset.py"
    click V5 "https://github.com/neuspeech/eeg-to-text/blob/main/util/get_sentiment_labels.py"
    click PrepSh "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/prepare_dataset.sh"
    click DataPy "https://github.com/neuspeech/eeg-to-text/blob/main/data.py"

    click MD "https://github.com/neuspeech/eeg-to-text/blob/main/model_decoding.py"
    click TD "https://github.com/neuspeech/eeg-to-text/blob/main/train_decoding.py"
    click TDS1 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/train_decoding.sh"
    click TDS2 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/train_decoding_1.sh"

    click MS1 "https://github.com/neuspeech/eeg-to-text/blob/main/model_sentiment.py"
    click MS2 "https://github.com/neuspeech/eeg-to-text/blob/main/train_sentiment_baseline.py"
    click MS3 "https://github.com/neuspeech/eeg-to-text/blob/main/train_sentiment_textbased.py"
    click STSh1 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/train_text_sentiment_classifier.sh"
    click STSh2 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/train_eeg_sentiment_baseline.sh"

    click EDpy "https://github.com/neuspeech/eeg-to-text/blob/main/eval_decoding.py"
    click ED1 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/eval_decoding_1.sh"
    click ED2 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/eval_decoding_2.sh"
    click ED3 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/eval_decoding_3.sh"
    click ED4 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/eval_decoding_4.sh"

    click ESpy "https://github.com/neuspeech/eeg-to-text/blob/main/eval_sentiment.py"
    click ES1 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/eval_sentiment_zeroshot_pipeline.sh"
    click ES2 "https://github.com/neuspeech/eeg-to-text/blob/main/scripts/train_eval_zeroshot_pipeline.sh"

    click Results "https://github.com/neuspeech/eeg-to-text/tree/main/results/"

    %% Styles
    classDef data fill:#cce5ff,stroke:#2b6cb0,color:#2b6cb0;
    classDef train fill:#d4f4dd,stroke:#2f855a,color:#2f855a;
    classDef eval fill:#ffe5b4,stroke:#dd6b20,color:#dd6b20;
    classDef config fill:#e2e8f0,stroke:#4a5568,color:#4a5568;
    classDef results fill:#fde2e1,stroke:#c43b31,color:#c43b31;
```
