# aeryn666/regionrag architecture

> Architecture diagram of the GitHub repository aeryn666/regionrag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aeryn666/regionrag
- Repository: https://github.com/aeryn666/regionrag

## Overview

This tree is a ColPali-derived visual-document retrieval and training library, while the README describes RegionRAG’s region-level RAG workflow. The principal documented path is query and document-region retrieval, followed by focused generation; the package also provides contrastive training, retrieval/generation evaluation, and interpretability. The README supports the high-level RegionRAG stages, but the tree contains no dedicated RegionRAG application entrypoint or generator service, so those links are documented workflow rather than verified runtime calls.

Main entry points: [`colpali_engine/evaluation/retrieval_evaluator.py`](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/evaluation/retrieval_evaluator.py) (Retrieval evaluation), [`colpali_engine/trainer/colmodel_training.py`](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/trainer/colmodel_training.py) (Training orchestration), [`colpali_engine/utils/processing_utils.py`](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/utils/processing_utils.py) (Visual processor), [`colpali_engine/collators/visual_retriever_collator.py`](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/collators/visual_retriever_collator.py) (Training collators)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data and Processing"]
  node_dataset_prep["Dataset preparation"]
  node_processor["Visual processor"]
  node_collator["Training collators"]
  node_corpus_collator["Corpus query batches"]
end

subgraph group_models["Retrieval Models"]
  node_colpali["ColPali retrieval model"]
  node_model_family["Model variants"]
  node_pooling["Token pooling"]
  node_scoring["Multivector scoring"]
end

subgraph group_train["Training"]
  node_training["Training orchestration"]
  node_trainer["Contrastive trainer"]
  node_loss["Region contrastive loss"]
end

subgraph group_eval["Retrieval and Generation"]
  node_retrieval_eval["Retrieval evaluation"]
  node_region_group["Neighbor region grouping"]
  node_region_eval["Region similarity evaluation"]
  node_generation["Focused generation"]
  node_eval_utils["Evaluation utilities<br/>[eval_utils.py]"]
  node_maps["Similarity maps<br/>[similarity_maps.py]"]
end

node_user(("Research user"))
node_train_data[("Training datasets")]
node_eval_data[("Evaluation datasets")]
node_query["Text query"]

node_user -->|"asks about"| node_query
node_train_data -->|"loads and transforms"| node_dataset_prep
node_dataset_prep -.->|"supplies examples"| node_collator
node_processor -->|"processes images"| node_collator
node_processor -->|"processes inputs"| node_corpus_collator
node_collator -->|"specializes"| node_corpus_collator
node_training -->|"selects for corpus"| node_corpus_collator
node_training -->|"runs training"| node_trainer
node_training -->|"configures loss"| node_loss
node_training -->|"trains on"| node_train_data
node_colpali -.->|"pairs with processor"| node_processor
node_model_family -->|"exports model"| node_colpali
node_pooling -.->|"prepares vectors for"| node_scoring
node_query -->|"retrieves relevant regions"| node_retrieval_eval
node_eval_data -->|"evaluates against"| node_retrieval_eval
node_retrieval_eval -.->|"scores vectors"| node_scoring
node_retrieval_eval -.->|"extracts regions"| node_region_group
node_region_group -->|"provides focused context"| node_generation
node_retrieval_eval -->|"passes retrieval results"| node_generation
node_generation -->|"uses evaluation helpers"| node_eval_utils
node_retrieval_eval -.->|"supports region evaluation"| node_region_eval
node_retrieval_eval -.->|"interprets similarity"| node_maps
node_user -->|"trains retriever"| node_training

click node_dataset_prep "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/utils/dataset_transformation.py"
click node_processor "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/utils/processing_utils.py"
click node_collator "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/collators/visual_retriever_collator.py"
click node_corpus_collator "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/collators/corpus_query_collator.py"
click node_colpali "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/models/paligemma/colpali/modeling_colpali.py"
click node_model_family "https://github.com/aeryn666/regionrag/tree/main/colpali_engine/models"
click node_pooling "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/compression/token_pooling/base_token_pooling.py"
click node_scoring "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/utils/processing_utils.py"
click node_training "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/trainer/colmodel_training.py"
click node_trainer "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/trainer/contrastive_trainer.py"
click node_loss "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/loss/late_interaction_losses.py"
click node_retrieval_eval "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/evaluation/retrieval_evaluator.py"
click node_region_group "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/utils/processing_utils.py"
click node_region_eval "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/evaluation/region_sim_evaluator.py"
click node_generation "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/evaluation/generation_evaluator.py"
click node_eval_utils "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/evaluation/eval_utils.py"
click node_maps "https://github.com/aeryn666/regionrag/blob/main/colpali_engine/interpretability/similarity_maps.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset_prep,node_processor,node_collator,node_corpus_collator,node_user toneBlue
class node_colpali,node_model_family,node_pooling,node_scoring,node_train_data,node_eval_data toneAmber
class node_training,node_trainer,node_loss toneMint
class node_retrieval_eval,node_region_group,node_region_eval,node_generation,node_eval_utils,node_maps toneRose
class node_query toneIndigo
```

## Components

### Data and Processing

- [Dataset preparation](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/utils/dataset_transformation.py): `colpali_engine/utils/dataset_transformation.py`
- [Visual processor](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/utils/processing_utils.py): `colpali_engine/utils/processing_utils.py`
- [Training collators](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/collators/visual_retriever_collator.py): `colpali_engine/collators/visual_retriever_collator.py`
- [Corpus query batches](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/collators/corpus_query_collator.py): `colpali_engine/collators/corpus_query_collator.py`

### Retrieval Models

- [ColPali retrieval model](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/models/paligemma/colpali/modeling_colpali.py): `colpali_engine/models/paligemma/colpali/modeling_colpali.py`
- [Model variants](https://github.com/aeryn666/regionrag/tree/main/colpali_engine/models): `colpali_engine/models`
- [Token pooling](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/compression/token_pooling/base_token_pooling.py): `colpali_engine/compression/token_pooling/base_token_pooling.py`
- [Multivector scoring](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/utils/processing_utils.py): `colpali_engine/utils/processing_utils.py`

### Training

- [Training orchestration](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/trainer/colmodel_training.py): `colpali_engine/trainer/colmodel_training.py`
- [Contrastive trainer](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/trainer/contrastive_trainer.py): `colpali_engine/trainer/contrastive_trainer.py`
- [Region contrastive loss](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/loss/late_interaction_losses.py): `colpali_engine/loss/late_interaction_losses.py`

### Retrieval and Generation

- [Retrieval evaluation](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/evaluation/retrieval_evaluator.py): `colpali_engine/evaluation/retrieval_evaluator.py`
- [Neighbor region grouping](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/utils/processing_utils.py): `colpali_engine/utils/processing_utils.py`
- [Region similarity evaluation](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/evaluation/region_sim_evaluator.py): `colpali_engine/evaluation/region_sim_evaluator.py`
- [Focused generation](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/evaluation/generation_evaluator.py): `colpali_engine/evaluation/generation_evaluator.py`
- [Evaluation utilities](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/evaluation/eval_utils.py): `colpali_engine/evaluation/eval_utils.py`
- [Similarity maps](https://github.com/aeryn666/regionrag/blob/main/colpali_engine/interpretability/similarity_maps.py): `colpali_engine/interpretability/similarity_maps.py`

### Other components

- **Research user**
- **Training datasets**
- **Evaluation datasets**
- **Text query**

## Connections

- Research user → Text query: asks about (evidence: `README.md`)
- Training datasets → Dataset preparation: loads and transforms (evidence: `colpali_engine/utils/dataset_transformation.py`)
- Dataset preparation → Training collators: supplies examples
- Visual processor → Training collators: processes images (evidence: `colpali_engine/collators/visual_retriever_collator.py`)
- Visual processor → Corpus query batches: processes inputs (evidence: `colpali_engine/collators/corpus_query_collator.py`)
- Training collators → Corpus query batches: specializes (evidence: `colpali_engine/collators/corpus_query_collator.py`)
- Training orchestration → Corpus query batches: selects for corpus (evidence: `colpali_engine/trainer/colmodel_training.py`)
- Training orchestration → Contrastive trainer: runs training (evidence: `colpali_engine/trainer/colmodel_training.py`)
- Training orchestration → Region contrastive loss: configures loss (evidence: `colpali_engine/trainer/colmodel_training.py`)
- Training orchestration → Training datasets: trains on (evidence: `README.md`)
- ColPali retrieval model → Visual processor: pairs with processor
- Model variants → ColPali retrieval model: exports model (evidence: `colpali_engine/models/__init__.py`)
- Token pooling → Multivector scoring: prepares vectors for
- Text query → Retrieval evaluation: retrieves relevant regions (evidence: `README.md`)
- Evaluation datasets → Retrieval evaluation: evaluates against (evidence: `colpali_engine/evaluation/retrieval_evaluator.py`)
- Retrieval evaluation → Multivector scoring: scores vectors
- Retrieval evaluation → Neighbor region grouping: extracts regions
- Neighbor region grouping → Focused generation: provides focused context (evidence: `README.md`)
- Retrieval evaluation → Focused generation: passes retrieval results (evidence: `README.md`)
- Focused generation → Evaluation utilities: uses evaluation helpers (evidence: `colpali_engine/evaluation/generation_evaluator.py`)
- Retrieval evaluation → Region similarity evaluation: supports region evaluation
- Retrieval evaluation → Similarity maps: interprets similarity
- Research user → Training orchestration: trains retriever (evidence: `README.md`)
