How sh7shir/speech-restoration-deep-learning works
This is a research pipeline for decoding sEEG features into log-Mel speech spectrograms, comparing diffusion and baseline neural models. The main workflow conditionally extracts EEG/audio features, trains and predicts per participant, evaluates reconstruction quality, and optionally plots results. Source excerpts show preprocessing, model selection, persistence, and evaluation; README documents the broader study and metrics. Some reported capabilities are documented but not independently verified in the sampled code.
Main entry points: audio_constructor.py Participant reconstruction · main.py Pipeline entry · vis.py Result visualizations · feature_extractor.py Feature extraction
Components 14
Feature preparation
- Run settings
src/config.py - Feature extraction
src/feature_extractor.py - Mel filter bank
src/MelFilterBank.py - Feature arrays
src/feature_extractor.py
Decoder pipeline
- Participant reconstruction
src/audio_constructor.py - Decoder architectures
src/models.py - Predicted spectrograms
src/audio_constructor.py
Evaluation and reporting
- Reconstruction metrics
src/audio_constructor.py - Evaluation results
src/audio_constructor.py - Result visualizations
src/vis.py - Generated figures
src/vis.py
Other components
- Researcher
- sEEG and audio dataset
- Pipeline entry
main.py
Connections 18
- Researcher → Pipeline entry: runs
README.md - Pipeline entry → Run settings: reads flags
main.py - Pipeline entry → Feature extraction: runs when enabled
main.py - sEEG and audio dataset → Feature extraction: provides recordings
src/feature_extractor.py - Feature extraction → Mel filter bank: converts spectra
src/feature_extractor.py - Feature extraction → Feature arrays: writes arrays
src/feature_extractor.py - Pipeline entry → Participant reconstruction: runs when enabled
main.py - Run settings → Participant reconstruction: configures run
src/audio_constructor.py - Feature arrays → Participant reconstruction: supplies inputs
src/audio_constructor.py - Participant reconstruction → Decoder architectures: selects and trains
src/audio_constructor.py - Participant reconstruction → Predicted spectrograms: saves predictions
src/audio_constructor.py - Participant reconstruction → Reconstruction metrics: evaluates predictions
src/audio_constructor.py - Reconstruction metrics → Evaluation results: saves scores
src/audio_constructor.py - Pipeline entry → Result visualizations: runs when enabled
main.py - Evaluation results → Result visualizations: plots scores
src/vis.py - Predicted spectrograms → Result visualizations: plots spectrograms
src/vis.py - Run settings → Result visualizations: sets plot locations
src/vis.py - Result visualizations → Generated figures: writes figures
src/vis.py