# vandijklab/brainlm architecture

> Architecture diagram of the GitHub repository vandijklab/brainlm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vandijklab/brainlm
- Repository: https://github.com/vandijklab/brainlm

## Overview

BrainLM is a PyTorch self-supervised model for learning brain dynamics from fMRI recordings. The main workflow prepares UK Biobank recordings, trains a masked-prediction encoder-decoder, evaluates predictions, and uses learned representations for inference and downstream analysis. The graph also preserves documented preprocessing and toolkit capabilities. Some subsystem paths and relationships are based on the README or tree rather than sampled source; those links are omitted where ownership is unclear.

Main entry points: [`train.py`](https://github.com/vandijklab/brainlm/blob/main/train.py) (Training entrypoint), [`continue_train_same_wandb.py`](https://github.com/vandijklab/brainlm/blob/main/continue_train_same_wandb.py) (Resume training), [`brainlm_mae/modeling_brainlm.py`](https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/modeling_brainlm.py) (Masked-prediction model), [`utils/brainlm_trainer.py`](https://github.com/vandijklab/brainlm/blob/main/utils/brainlm_trainer.py) (Trainer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_converter["UKB converter"]
  node_arrow[("Arrow datasets")]
  node_prep["Data preprocessing<br/>[BrainLM_Toolkit.py]"]
end

subgraph group_learning["Model and training"]
  node_config["BrainLM config"]
  node_model["Masked-prediction model"]
  node_padded_vit["Padded ViT-MAE"]
  node_finetune_config["Fine-tuning config"]
  node_finetune_head["Fine-tuning model"]
  node_train["Training entrypoint<br/>[train.py]"]
  node_resume["Resume training"]
  node_trainer["Trainer<br/>[brainlm_trainer.py]"]
  node_metrics["Evaluation metrics<br/>[metrics.py]"]
  node_plots["Training plots<br/>[plots.py]"]
end

subgraph group_analysis["Inference and analysis"]
  node_inference["Inference workflows"]
  node_downstream["Representation analysis"]
  node_reconstruction["Recording reconstruction"]
end

subgraph group_toolkit["Brain imaging toolkit"]
  node_toolkit_app["BrainLM toolkit<br/>[BrainLM_Toolkit.py]"]
  node_atlases[("Brain atlases")]
end

node_researcher(("Researcher"))
node_fmri[("fMRI recordings")]

node_researcher -->|"obtains data"| node_fmri
node_fmri -->|"normalizes recordings"| node_converter
node_converter -->|"writes datasets"| node_arrow
node_researcher -->|"preprocesses data"| node_prep
node_researcher -->|"starts training"| node_train
node_train -->|"configures"| node_config
node_train -->|"instantiates"| node_model
node_train -->|"trains with"| node_trainer
node_train -->|"evaluates with"| node_metrics
node_resume -->|"instantiates"| node_model
node_resume -->|"configures"| node_config
node_resume -->|"trains with"| node_trainer
node_resume -->|"evaluates with"| node_metrics
node_model -.->|"uses base layers"| node_padded_vit
node_finetune_head -->|"uses config"| node_finetune_config
node_trainer -->|"plots batches"| node_plots
node_metrics -->|"creates evaluation plots"| node_plots
node_researcher -->|"runs predictions"| node_inference
node_inference -.->|"uses pretrained model"| node_model
node_researcher -->|"analyzes representations"| node_downstream
node_researcher -->|"reconstructs recordings"| node_reconstruction
node_researcher -->|"uses preprocessing toolkit"| node_toolkit_app
node_toolkit_app -.->|"uses atlas assets"| node_atlases

click node_converter "https://github.com/vandijklab/brainlm/blob/main/convert_ukbiobank_to_arrow.py"
click node_prep "https://github.com/vandijklab/brainlm/blob/main/toolkit/BrainLM_Toolkit.py"
click node_config "https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/configuration_brainlm.py"
click node_model "https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/modeling_brainlm.py"
click node_padded_vit "https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/modeling_vit_mae_with_padding.py"
click node_finetune_config "https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/vit_image_finetune_config.py"
click node_finetune_head "https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/vit_image_finetuning_mlp_pred_head.py"
click node_train "https://github.com/vandijklab/brainlm/blob/main/train.py"
click node_resume "https://github.com/vandijklab/brainlm/blob/main/continue_train_same_wandb.py"
click node_trainer "https://github.com/vandijklab/brainlm/blob/main/utils/brainlm_trainer.py"
click node_metrics "https://github.com/vandijklab/brainlm/blob/main/utils/metrics.py"
click node_plots "https://github.com/vandijklab/brainlm/blob/main/utils/plots.py"
click node_inference "https://github.com/vandijklab/brainlm/blob/main/brainlm_tutorial.ipynb"
click node_downstream "https://github.com/vandijklab/brainlm/blob/main/inference_02_cls_token_knn_regressor.ipynb"
click node_reconstruction "https://github.com/vandijklab/brainlm/blob/main/inference_05_reconstruct_whole_recording.ipynb"
click node_toolkit_app "https://github.com/vandijklab/brainlm/blob/main/toolkit/BrainLM_Toolkit.py"
click node_atlases "https://github.com/vandijklab/brainlm/tree/main/toolkit/atlases"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_converter,node_arrow,node_prep toneBlue
class node_config,node_model,node_padded_vit,node_finetune_config,node_finetune_head,node_train,node_resume,node_trainer,node_metrics,node_plots,node_fmri toneAmber
class node_inference,node_downstream,node_reconstruction toneMint
class node_toolkit_app,node_atlases toneRose
class node_researcher toneIndigo
```

## Components

### Data preparation

- [UKB converter](https://github.com/vandijklab/brainlm/blob/main/convert_ukbiobank_to_arrow.py): `convert_ukbiobank_to_arrow.py`
- **Arrow datasets**
- [Data preprocessing](https://github.com/vandijklab/brainlm/blob/main/toolkit/BrainLM_Toolkit.py): `toolkit/BrainLM_Toolkit.py`

### Model and training

- [BrainLM config](https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/configuration_brainlm.py): `brainlm_mae/configuration_brainlm.py`
- [Masked-prediction model](https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/modeling_brainlm.py): `brainlm_mae/modeling_brainlm.py`
- [Padded ViT-MAE](https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/modeling_vit_mae_with_padding.py): `brainlm_mae/modeling_vit_mae_with_padding.py`
- [Fine-tuning config](https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/vit_image_finetune_config.py): `brainlm_mae/vit_image_finetune_config.py`
- [Fine-tuning model](https://github.com/vandijklab/brainlm/blob/main/brainlm_mae/vit_image_finetuning_mlp_pred_head.py): `brainlm_mae/vit_image_finetuning_mlp_pred_head.py`
- [Training entrypoint](https://github.com/vandijklab/brainlm/blob/main/train.py): `train.py`
- [Resume training](https://github.com/vandijklab/brainlm/blob/main/continue_train_same_wandb.py): `continue_train_same_wandb.py`
- [Trainer](https://github.com/vandijklab/brainlm/blob/main/utils/brainlm_trainer.py): `utils/brainlm_trainer.py`
- [Evaluation metrics](https://github.com/vandijklab/brainlm/blob/main/utils/metrics.py): `utils/metrics.py`
- [Training plots](https://github.com/vandijklab/brainlm/blob/main/utils/plots.py): `utils/plots.py`

### Inference and analysis

- [Inference workflows](https://github.com/vandijklab/brainlm/blob/main/brainlm_tutorial.ipynb): `brainlm_tutorial.ipynb`
- [Representation analysis](https://github.com/vandijklab/brainlm/blob/main/inference_02_cls_token_knn_regressor.ipynb): `inference_02_cls_token_knn_regressor.ipynb`
- [Recording reconstruction](https://github.com/vandijklab/brainlm/blob/main/inference_05_reconstruct_whole_recording.ipynb): `inference_05_reconstruct_whole_recording.ipynb`

### Brain imaging toolkit

- [BrainLM toolkit](https://github.com/vandijklab/brainlm/blob/main/toolkit/BrainLM_Toolkit.py): `toolkit/BrainLM_Toolkit.py`
- [Brain atlases](https://github.com/vandijklab/brainlm/tree/main/toolkit/atlases): `toolkit/atlases`

### Other components

- **Researcher**
- **fMRI recordings**

## Connections

- Researcher → fMRI recordings: obtains data (evidence: `README.md`)
- fMRI recordings → UKB converter: normalizes recordings (evidence: `convert_ukbiobank_to_arrow.py`)
- UKB converter → Arrow datasets: writes datasets (evidence: `convert_ukbiobank_to_arrow.py`)
- Researcher → Data preprocessing: preprocesses data (evidence: `README.md`)
- Researcher → Training entrypoint: starts training (evidence: `README.md`)
- Training entrypoint → BrainLM config: configures (evidence: `train.py`)
- Training entrypoint → Masked-prediction model: instantiates (evidence: `train.py`)
- Training entrypoint → Trainer: trains with (evidence: `train.py`)
- Training entrypoint → Evaluation metrics: evaluates with (evidence: `train.py`)
- Resume training → Masked-prediction model: instantiates (evidence: `continue_train_same_wandb.py`)
- Resume training → BrainLM config: configures (evidence: `continue_train_same_wandb.py`)
- Resume training → Trainer: trains with (evidence: `continue_train_same_wandb.py`)
- Resume training → Evaluation metrics: evaluates with (evidence: `continue_train_same_wandb.py`)
- Masked-prediction model → Padded ViT-MAE: uses base layers
- Fine-tuning model → Fine-tuning config: uses config (evidence: `brainlm_mae/vit_image_finetuning_mlp_pred_head.py`)
- Trainer → Training plots: plots batches (evidence: `utils/brainlm_trainer.py`)
- Evaluation metrics → Training plots: creates evaluation plots (evidence: `utils/metrics.py`)
- Researcher → Inference workflows: runs predictions (evidence: `README.md`)
- Inference workflows → Masked-prediction model: uses pretrained model
- Researcher → Representation analysis: analyzes representations (evidence: `README.md`)
- Researcher → Recording reconstruction: reconstructs recordings (evidence: `README.md`)
- Researcher → BrainLM toolkit: uses preprocessing toolkit (evidence: `README.md`)
- BrainLM toolkit → Brain atlases: uses atlas assets
