# epics-lab/brainfmbench architecture

> Architecture diagram of the GitHub repository epics-lab/brainfmbench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/epics-lab/brainfmbench
- Repository: https://github.com/epics-lab/brainfmbench
- Video tour (about a minute): https://gitdiagram.com/epics-lab/brainfmbench/video

## Overview

BrainFMBench is a structural-MRI foundation-model benchmark: submissions provide metadata and feature extractors (or precomputed features), extraction runs on a compute cluster, and downstream probes score features against shared labels for the leaderboard. The graph preserves the documented preprocessing, extraction, scoring, and publishing stages, plus cluster orchestration and representative model implementations. README-documented workflow edges are cited there; source excerpts confirm the runner-to-SSH dependency and the scoring script’s use of the scorer. Other model implementations are grouped where their source was not sampled.

Main entry points: [`scripts/score_all.py`](https://github.com/epics-lab/brainfmbench/blob/main/scripts/score_all.py) (Scoring orchestration), [`models`](https://github.com/epics-lab/brainfmbench/tree/main/models) (Feature vectors), [`models/brain2vec/preprocess.py`](https://github.com/epics-lab/brainfmbench/blob/main/models/brain2vec/preprocess.py) (Brain2Vec alignment), [`eval/scorer.py`](https://github.com/epics-lab/brainfmbench/blob/main/eval/scorer.py) (Downstream probes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_submission["Submissions"]
  node_submission_spec["Model submission"]
  node_example["Example submission<br/>[extract.py]"]
end

subgraph group_compute["Feature computation"]
  node_preprocessing["MRI preprocessing"]
  node_brain2vec_prep["Brain2Vec alignment<br/>[preprocess.py]"]
  node_brain2vec["Brain2Vec extractor<br/>[extract.py]"]
  node_dinov3d["Dinov3D extractor<br/>[extract.py]"]
  node_uncnn["un-CNN extractor<br/>[extract.py]"]
  node_other_extractors["Other model extractors"]
end

subgraph group_evaluation["Evaluation"]
  node_features["Feature vectors"]
  node_labels["Task labels"]
  node_score_all["Scoring orchestration<br/>[score_all.py]"]
  node_scorer["Downstream probes<br/>[scorer.py]"]
  node_results["Model results"]
end

subgraph group_operations["Benchmark operations"]
  node_cluster_runner["Cluster runner<br/>[cluster_runner.py]"]
  node_ssh_manager["Cluster SSH<br/>[ssh_manager.py]"]
  node_leaderboard["Leaderboard"]
end

node_contributor(("Contributor"))
node_cluster["Compute cluster"]
node_github["GitHub Actions"]

node_contributor -->|"submits model"| node_submission_spec
node_submission_spec -->|"queues extraction"| node_cluster_runner
node_cluster_runner -->|"uses SSH"| node_ssh_manager
node_ssh_manager -->|"stages and commands"| node_cluster
node_preprocessing -->|"prepares MRI data"| node_cluster
node_cluster -.->|"runs model preprocessing"| node_brain2vec_prep
node_brain2vec_prep -->|"aligns extractor input"| node_brain2vec
node_cluster -->|"runs extraction"| node_brain2vec
node_cluster -->|"runs extraction"| node_dinov3d
node_cluster -->|"runs extraction"| node_uncnn
node_cluster -->|"runs extraction"| node_other_extractors
node_example -->|"writes embeddings"| node_features
node_cluster -->|"returns extracted vectors"| node_features
node_features -->|"supplies features"| node_score_all
node_labels -->|"supplies labels"| node_score_all
node_score_all -->|"calls scorer"| node_scorer
node_scorer -->|"produces scores"| node_results
node_results -->|"feeds leaderboard"| node_leaderboard
node_github -->|"scores features"| node_score_all
node_github -->|"updates ranking"| node_leaderboard

click node_submission_spec "https://github.com/epics-lab/brainfmbench/tree/main/models"
click node_example "https://github.com/epics-lab/brainfmbench/blob/main/example-submission/extract.py"
click node_cluster_runner "https://github.com/epics-lab/brainfmbench/blob/main/cluster/cluster_runner.py"
click node_ssh_manager "https://github.com/epics-lab/brainfmbench/blob/main/cluster/ssh_manager.py"
click node_brain2vec_prep "https://github.com/epics-lab/brainfmbench/blob/main/models/brain2vec/preprocess.py"
click node_brain2vec "https://github.com/epics-lab/brainfmbench/blob/main/models/brain2vec/extract.py"
click node_dinov3d "https://github.com/epics-lab/brainfmbench/blob/main/models/dinov3d-neuro/extract.py"
click node_uncnn "https://github.com/epics-lab/brainfmbench/blob/main/models/uncnn/extract.py"
click node_other_extractors "https://github.com/epics-lab/brainfmbench/tree/main/models"
click node_features "https://github.com/epics-lab/brainfmbench/tree/main/models"
click node_labels "https://github.com/epics-lab/brainfmbench/tree/main/labels"
click node_score_all "https://github.com/epics-lab/brainfmbench/blob/main/scripts/score_all.py"
click node_scorer "https://github.com/epics-lab/brainfmbench/blob/main/eval/scorer.py"
click node_results "https://github.com/epics-lab/brainfmbench/tree/main/models"
click node_leaderboard "https://github.com/epics-lab/brainfmbench/blob/main/scripts/make_leaderboard.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_submission_spec,node_example toneBlue
class node_preprocessing,node_brain2vec_prep,node_brain2vec,node_dinov3d,node_uncnn,node_other_extractors toneAmber
class node_features,node_labels,node_score_all,node_scorer,node_results toneMint
class node_cluster_runner,node_ssh_manager,node_leaderboard toneRose
class node_contributor,node_cluster,node_github toneIndigo
```

## Components

### Submissions

- [Model submission](https://github.com/epics-lab/brainfmbench/tree/main/models): `models`
- [Example submission](https://github.com/epics-lab/brainfmbench/blob/main/example-submission/extract.py): `example-submission/extract.py`

### Feature computation

- **MRI preprocessing**
- [Brain2Vec alignment](https://github.com/epics-lab/brainfmbench/blob/main/models/brain2vec/preprocess.py): `models/brain2vec/preprocess.py`
- [Brain2Vec extractor](https://github.com/epics-lab/brainfmbench/blob/main/models/brain2vec/extract.py): `models/brain2vec/extract.py`
- [Dinov3D extractor](https://github.com/epics-lab/brainfmbench/blob/main/models/dinov3d-neuro/extract.py): `models/dinov3d-neuro/extract.py`
- [un-CNN extractor](https://github.com/epics-lab/brainfmbench/blob/main/models/uncnn/extract.py): `models/uncnn/extract.py`
- [Other model extractors](https://github.com/epics-lab/brainfmbench/tree/main/models): `models`

### Evaluation

- [Feature vectors](https://github.com/epics-lab/brainfmbench/tree/main/models): `models`
- [Task labels](https://github.com/epics-lab/brainfmbench/tree/main/labels): `labels`
- [Scoring orchestration](https://github.com/epics-lab/brainfmbench/blob/main/scripts/score_all.py): `scripts/score_all.py`
- [Downstream probes](https://github.com/epics-lab/brainfmbench/blob/main/eval/scorer.py): `eval/scorer.py`
- [Model results](https://github.com/epics-lab/brainfmbench/tree/main/models): `models`

### Benchmark operations

- [Cluster runner](https://github.com/epics-lab/brainfmbench/blob/main/cluster/cluster_runner.py): `cluster/cluster_runner.py`
- [Cluster SSH](https://github.com/epics-lab/brainfmbench/blob/main/cluster/ssh_manager.py): `cluster/ssh_manager.py`
- [Leaderboard](https://github.com/epics-lab/brainfmbench/blob/main/scripts/make_leaderboard.py): `scripts/make_leaderboard.py`

### Other components

- **Contributor**
- **Compute cluster**
- **GitHub Actions**

## Connections

- Contributor → Model submission: submits model (evidence: `README.md`)
- Model submission → Cluster runner: queues extraction (evidence: `README.md`)
- Cluster runner → Cluster SSH: uses SSH (evidence: `cluster/cluster_runner.py`)
- Cluster SSH → Compute cluster: stages and commands (evidence: `cluster/ssh_manager.py`)
- MRI preprocessing → Compute cluster: prepares MRI data (evidence: `README.md`)
- Compute cluster → Brain2Vec alignment: runs model preprocessing
- Brain2Vec alignment → Brain2Vec extractor: aligns extractor input (evidence: `models/brain2vec/preprocess.py`)
- Compute cluster → Brain2Vec extractor: runs extraction (evidence: `README.md`)
- Compute cluster → Dinov3D extractor: runs extraction (evidence: `README.md`)
- Compute cluster → un-CNN extractor: runs extraction (evidence: `README.md`)
- Compute cluster → Other model extractors: runs extraction (evidence: `README.md`)
- Example submission → Feature vectors: writes embeddings (evidence: `example-submission/extract.py`)
- Compute cluster → Feature vectors: returns extracted vectors (evidence: `README.md`)
- Feature vectors → Scoring orchestration: supplies features (evidence: `scripts/score_all.py`)
- Task labels → Scoring orchestration: supplies labels (evidence: `scripts/score_all.py`)
- Scoring orchestration → Downstream probes: calls scorer (evidence: `scripts/score_all.py`)
- Downstream probes → Model results: produces scores (evidence: `eval/scorer.py`)
- Model results → Leaderboard: feeds leaderboard (evidence: `README.md`)
- GitHub Actions → Scoring orchestration: scores features (evidence: `README.md`)
- GitHub Actions → Leaderboard: updates ranking (evidence: `README.md`)