How czlwang/brain_treebank_code_release works
Project Type and Purpose
• This is a data-analysis/experiment orchestration tool for large-scale intracranial brain recordings.
• It provides multiple analysis “pipelines” (GLM fitting, word-onset t-tests, sentence-position F-tests, linear decoding) driven by Hydra config files.
• It is best characterized as an extensible CLI-based data-processing framework, with plugins for models, tasks, schedulers, criterions, and datasets.
File Structure
• Top-level scripts/entry points:
• run_glms.py, regression_runner.py, glm_runner.py, run_time_tests.py, runner.py, verify_frame_alignment.py
• Configuration:
• conf/ (Hydra config tree: data, exp, task, test, criterion)
• Data ingestion & representation:
• data/ (raw HDF5 readers, subject/trial/ timestamped data classes)
• datasets/ (wrappers that produce PyTorch-style datasets for GLM/regression)
• Preprocessing:
• preprocessors/ (e.g. wav_preprocessor)
• Modeling & training pipeline plugins:
• models/ (base_regression_model, classification_regression_model)
• criterions/ (base_criterion, empty_criterion)
• schedulers/ (base_scheduler, ramp_up, reduce_on_plateau)
• tasks/ (base_regression_task, classification_regression_task, batch_utils)
• Experiment orchestration:
• runner.py orchestrates instantiation of config, data, model, criterion, scheduler, task, trainer loops
• glm_runner.py and regression_runner.py specialize runner for GLM and regression pipelines
• Statistical analyses (post-hoc tests):
• stats/ (word_onset_tests.py, sentence_position_tests.py)
• Utilities:
• util/tensorboard_utils.py, data/utils.py, tasks/utils.py, verify_frame_alignment.py
README for Additional Insights
• Describes four main analysis types: GLM, word-responsiveness, sentence-position, linear decoding
• Shows that Hydra command-line arguments tie together config files (conf/data, conf/exp, conf/task, conf/test) with scripts
• External data source: braintreebank_data (raw HDF5, transcripts)
Main Components and Their Interactions
a. Config Layer (Hydra)
• conf/ tree provides parameter sets for data ingestion, experiment settings, model selection, tasks, tests, schedulers, criterions
• CLI scripts load Hydra configs, driving the rest of the pipeline
b. CLI Scripts / Entry Points
• run_glms.py, run_time_tests.py, verify_frame_alignment.py
• Call into runner modules after Hydra config is composed
c. Data Ingestion
• data/*.py, data/h5_data_reader.py read raw .h5 files, subject metadata, transcripts
• data utils perform splitting, timestamp alignment
d. Dataset Construction
• datasets/*.py wrap ingested data into training/evaluation datasets for PyTorch or custom loops
e. Preprocessing
• preprocessors/*.py for on-the-fly signal processing (e.g. wav → features)
f. Tasks
• tasks/*.py implement the experiment logic (e.g. sliding-window definitions, batching, label generation)
g. Models, Criterions, Schedulers
• models define forward and backward passes (GLM fit vs classification/regression)
• criterions define loss or fitting functions (e.g. empty_criterion for analytical fits)
• schedulers manage learning-rate schedules or ramp-up strategies
h. Runner / Trainer
• runner.py and specialized runners (glm_runner.py, regression_runner.py) tie together Task, Model, Criterion, Scheduler in a training or fitting loop
i. Stats Post-Processing
• stats/*.py perform t-tests or F-tests on saved outputs, generating figures and tables
j. Outputs
• Results written to specified out_dir directories (e.g. glm_outputs, word_onset_tstat_results)
• Logs and TensorBoard summaries via util/tensorboard_utils
Architectural Patterns and Design Principles
• Plugin-based modularity: data / datasets / preprocessors / models / criterions / schedulers / tasks register themselves via directory structure and Hydra
• Pipeline pattern: data ingestion → dataset → preprocessing → task logic → model fitting or statistical analysis → output
• Separation of concerns: configuration vs code, experiment logic vs I/O, model definition vs training orchestration
Elements
• An “External” boundary for the raw data store (braintreebank_data)
Detail Without Overthinking
• How Hydra glues together the pieces
• Plugin directories (models, tasks, schedulers, criterions) as extension points