# rohanshad/cmr_transformer architecture

> Architecture diagram of the GitHub repository rohanshad/cmr_transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rohanshad/cmr_transformer
- Repository: https://github.com/rohanshad/cmr_transformer

## Overview

This repository provides pretrained cardiac-MRI video encoders plus a Lightning workflow for fine-tuning and evaluation. The main path is configured training or evaluation through the trainer, data module, dataset and video model, producing task predictions; a separate quickstart loads pretrained weights and returns embeddings. README-described report-supervised contrastive pretraining and downstream tasks are represented, but their unsampled internals are not expanded. Some links rely on README statements or the source index rather than inspected call sites.

Main entry points: [`model_factory.py`](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py) (Model factory), [`mri_trainer.py`](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/mri_trainer.py) (Trainer CLI), [`minimal_run.py`](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/minimal_run.py) (Encoder quickstart), [`data/data_modules.py`](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/data/data_modules.py) (Data modules)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Training workflow"]
  node_trainer["Trainer CLI<br/>[mri_trainer.py]"]
  node_configs["Task configs"]
  node_quickstart["Encoder quickstart<br/>[minimal_run.py]"]
end

subgraph group_data["MRI data"]
  node_datamodule["Data modules<br/>[data_modules.py]"]
  node_dataset["MRI datasets<br/>[torch_datasets.py]"]
  node_mri_store[("MRI files")]
  node_transforms["Video transforms<br/>[data_modules.py]"]
end

subgraph group_models["Encoders and tasks"]
  node_model_factory["Model factory<br/>[model_factory.py]"]
  node_mvit["MViT encoder<br/>[mvit.py]"]
  node_resnet["ResNet encoder<br/>[resnet.py]"]
  node_text_encoder["Report text encoder<br/>[model_factory.py]"]
  node_task_heads["Task heads<br/>[model_factory.py]"]
  node_contrastive["Contrastive pretraining<br/>[model_factory.py]"]
  node_weights["Pretrained weights"]
end

subgraph group_analysis["Embeddings and attention"]
  node_embeddings["MRI embeddings<br/>[model_factory.py]"]
  node_attention_files[("Attention outputs<br/>[mri_trainer.py]")]
  node_attention_viz["Attention visualizer"]
end

node_researcher(("Researcher"))
node_huggingface(("Hugging Face"))

node_researcher -->|"runs task"| node_trainer
node_researcher -->|"runs encoder"| node_quickstart
node_trainer -->|"loads config"| node_configs
node_trainer -->|"uses data"| node_datamodule
node_datamodule -->|"creates datasets"| node_dataset
node_dataset -->|"reads scans"| node_mri_store
node_datamodule -->|"builds transforms"| node_transforms
node_trainer -->|"uses model"| node_model_factory
node_model_factory -->|"constructs encoder"| node_mvit
node_model_factory -->|"constructs encoder"| node_resnet
node_model_factory -->|"builds task head"| node_task_heads
node_model_factory -.->|"supports text model"| node_text_encoder
node_text_encoder -.->|"guides vision"| node_contrastive
node_contrastive -.->|"trains representations"| node_mvit
node_huggingface -->|"hosts checkpoint"| node_weights
node_quickstart -->|"downloads weights"| node_weights
node_quickstart -->|"loads encoder"| node_model_factory
node_model_factory -->|"returns embeddings"| node_embeddings
node_trainer -->|"writes maps"| node_attention_files
node_attention_files -->|"reads maps"| node_attention_viz

click node_trainer "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/mri_trainer.py"
click node_configs "https://github.com/rohanshad/cmr_transformer/tree/publicrelease/configs"
click node_datamodule "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/data/data_modules.py"
click node_dataset "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/data/torch_datasets.py"
click node_transforms "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/data/data_modules.py"
click node_model_factory "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py"
click node_mvit "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/arch/mvit.py"
click node_resnet "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/arch/resnet.py"
click node_text_encoder "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py"
click node_task_heads "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py"
click node_contrastive "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py"
click node_embeddings "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py"
click node_attention_files "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/mri_trainer.py"
click node_attention_viz "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/visualize_attentions.py"
click node_quickstart "https://github.com/rohanshad/cmr_transformer/blob/publicrelease/minimal_run.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_trainer,node_configs,node_quickstart toneBlue
class node_datamodule,node_dataset,node_mri_store,node_transforms toneAmber
class node_model_factory,node_mvit,node_resnet,node_text_encoder,node_task_heads,node_contrastive,node_weights toneMint
class node_embeddings,node_attention_files,node_attention_viz toneRose
class node_researcher,node_huggingface toneIndigo
```

## Components

### Training workflow

- [Trainer CLI](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/mri_trainer.py): `mri_trainer.py`
- [Task configs](https://github.com/rohanshad/cmr_transformer/tree/publicrelease/configs): `configs`
- [Encoder quickstart](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/minimal_run.py): `minimal_run.py`

### MRI data

- [Data modules](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/data/data_modules.py): `data/data_modules.py`
- [MRI datasets](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/data/torch_datasets.py): `data/torch_datasets.py`
- **MRI files**
- [Video transforms](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/data/data_modules.py): `data/data_modules.py`

### Encoders and tasks

- [Model factory](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py): `model_factory.py`
- [MViT encoder](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/arch/mvit.py): `arch/mvit.py`
- [ResNet encoder](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/arch/resnet.py): `arch/resnet.py`
- [Report text encoder](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py): `model_factory.py`
- [Task heads](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py): `model_factory.py`
- [Contrastive pretraining](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py): `model_factory.py`
- **Pretrained weights**

### Embeddings and attention

- [MRI embeddings](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/model_factory.py): `model_factory.py`
- [Attention outputs](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/mri_trainer.py): `mri_trainer.py`
- [Attention visualizer](https://github.com/rohanshad/cmr_transformer/blob/publicrelease/visualize_attentions.py): `visualize_attentions.py`

### Other components

- **Researcher**
- **Hugging Face**

## Connections

- Researcher → Trainer CLI: runs task (evidence: `README.md`)
- Researcher → Encoder quickstart: runs encoder (evidence: `README.md`)
- Trainer CLI → Task configs: loads config (evidence: `README.md`)
- Trainer CLI → Data modules: uses data (evidence: `mri_trainer.py`)
- Data modules → MRI datasets: creates datasets (evidence: `data/data_modules.py`)
- MRI datasets → MRI files: reads scans (evidence: `data/torch_datasets.py`)
- Data modules → Video transforms: builds transforms (evidence: `data/data_modules.py`)
- Trainer CLI → Model factory: uses model (evidence: `mri_trainer.py`)
- Model factory → MViT encoder: constructs encoder (evidence: `model_factory.py`)
- Model factory → ResNet encoder: constructs encoder (evidence: `model_factory.py`)
- Model factory → Task heads: builds task head (evidence: `model_factory.py`)
- Model factory → Report text encoder: supports text model (evidence: `README.md`)
- Report text encoder → Contrastive pretraining: guides vision (evidence: `README.md`)
- Contrastive pretraining → MViT encoder: trains representations (evidence: `README.md`)
- Hugging Face → Pretrained weights: hosts checkpoint (evidence: `README.md`)
- Encoder quickstart → Pretrained weights: downloads weights (evidence: `minimal_run.py`)
- Encoder quickstart → Model factory: loads encoder (evidence: `minimal_run.py`)
- Model factory → MRI embeddings: returns embeddings (evidence: `minimal_run.py`)
- Trainer CLI → Attention outputs: writes maps (evidence: `mri_trainer.py`)
- Attention outputs → Attention visualizer: reads maps (evidence: `visualize_attentions.py`)
