How stanford-stages/sleep-staging works
Project Type and Purpose
• This is a research-oriented, command-line Python tool for sleep staging classification using neural networks.
• It also includes MATLAB scripts for initial data extraction from raw EEG files.
• No web UI or database—just data-processing modules, model definitions, and training/testing scripts.
High-Level Components
a. Data Extraction Layer (MATLAB)
• Folder: matlab_training_data_scripts/
• Scripts: extractCC.m, get_cc_data.m, loadEDF.m, etc.
• Output: preprocessed .mat or CSV files containing EEG features or channel data.
b. Configuration Module
• File: sc_config.py
• Role: Defines file paths (training/test directories), hyperparameters, window lengths, model types (e.g., “ac_lh_ls_lstm”).
c. Data Reader & Preprocessor
• File: sc_reader.py (+ validation/test variants)
• Role: Loads extracted data files, applies any final preprocessing (normalization, window slicing).
d. Neural Network Definition
• Files: sc_network.py, sc_conv.py
• Role: Build Keras/TensorFlow or PyTorch model architectures (e.g., convolutional layers, LSTM memory layers).
e. Training Orchestration
• File: sc_train.py
• Role: Reads config, calls reader to fetch batches, feeds data into model, manages epochs, checkpoints, and logs.
f. Evaluation & Validation
• Files: sc_validate.py, sc_test.py, plus sc_reader_validate.py, sc_reader_test.py
• Role: Load trained model, run on held-out data, compute performance metrics.
g. External Dependencies
• MATLAB (for data extraction).
• Python 3.6+ and scientific stack (NumPy, SciPy, TensorFlow/Keras or PyTorch).
• File system for data storage and model checkpoints.
Relationships & Data Flow
• Raw EEG files (.edf, .hdr) → MATLAB Scripts → Extracted feature files
• Extracted files → Python sc_reader → Batch tensors
• Batches + sc_config → sc_train → sc_network/sc_conv → Model checkpoints
• Checkpoints + sc_reader_test → sc_test/sc_validate → Metrics & Reports
Architectural Patterns & Design Principles
• Modular layering: separate data-extraction, reading, modeling, training, and evaluation.
• Configuration-driven runs: sc_config.py enables reproducibility and easy switching of datasets or hyperparameters.
• Single-responsibility scripts: each script has a clear, isolated purpose (train vs. test vs. validate).
Must-Include Elements
• Component names (e.g., sc_reader.py)
• Data flows with artifacts (e.g., “Normalized EEG windows”)
• MATLAB scripts vs. Python modules
• Code vs. Data vs. Models vs. External