How vandijklab/brainlm works
BrainLM is a PyTorch self-supervised model for learning brain dynamics from fMRI recordings. The main workflow prepares UK Biobank recordings, trains a masked-prediction encoder-decoder, evaluates predictions, and uses learned representations for inference and downstream analysis. The graph also preserves documented preprocessing and toolkit capabilities. Some subsystem paths and relationships are based on the README or tree rather than sampled source; those links are omitted where ownership is unclear.
Main entry points: train.py Training entrypoint · continue_train_same_wandb.py Resume training · modeling_brainlm.py Masked-prediction model · brainlm_trainer.py Trainer
Components 20
Data preparation
- UKB converter
convert_ukbiobank_to_arrow.py - Arrow datasets
- Data preprocessing
toolkit/BrainLM_Toolkit.py
Model and training
- BrainLM config
brainlm_mae/configuration_brainlm.py - Masked-prediction model
brainlm_mae/modeling_brainlm.py - Padded ViT-MAE
brainlm_mae/modeling_vit_mae_with_padding.py - Fine-tuning config
brainlm_mae/vit_image_finetune_config.py - Fine-tuning model
brainlm_mae/vit_image_finetuning_mlp_pred_head.py - Training entrypoint
train.py - Resume training
continue_train_same_wandb.py - Trainer
utils/brainlm_trainer.py - Evaluation metrics
utils/metrics.py - Training plots
utils/plots.py
Inference and analysis
- Inference workflows
brainlm_tutorial.ipynb - Representation analysis
inference_02_cls_token_knn_regressor.ipynb - Recording reconstruction
inference_05_reconstruct_whole_recording.ipynb
Brain imaging toolkit
- BrainLM toolkit
toolkit/BrainLM_Toolkit.py - Brain atlases
toolkit/atlases
Other components
- Researcher
- fMRI recordings
Connections 23
- Researcher → fMRI recordings: obtains data
README.md - fMRI recordings → UKB converter: normalizes recordings
convert_ukbiobank_to_arrow.py - UKB converter → Arrow datasets: writes datasets
convert_ukbiobank_to_arrow.py - Researcher → Data preprocessing: preprocesses data
README.md - Researcher → Training entrypoint: starts training
README.md - Training entrypoint → BrainLM config: configures
train.py - Training entrypoint → Masked-prediction model: instantiates
train.py - Training entrypoint → Trainer: trains with
train.py - Training entrypoint → Evaluation metrics: evaluates with
train.py - Resume training → Masked-prediction model: instantiates
continue_train_same_wandb.py - Resume training → BrainLM config: configures
continue_train_same_wandb.py - Resume training → Trainer: trains with
continue_train_same_wandb.py - Resume training → Evaluation metrics: evaluates with
continue_train_same_wandb.py - Masked-prediction model → Padded ViT-MAE: uses base layers
- Fine-tuning model → Fine-tuning config: uses config
brainlm_mae/vit_image_finetuning_mlp_pred_head.py - Trainer → Training plots: plots batches
utils/brainlm_trainer.py - Evaluation metrics → Training plots: creates evaluation plots
utils/metrics.py - Researcher → Inference workflows: runs predictions
README.md - Inference workflows → Masked-prediction model: uses pretrained model
- Researcher → Representation analysis: analyzes representations
README.md - Researcher → Recording reconstruction: reconstructs recordings
README.md - Researcher → BrainLM toolkit: uses preprocessing toolkit
README.md - BrainLM toolkit → Brain atlases: uses atlas assets