How aeryn666/regionrag works
This tree is a ColPali-derived visual-document retrieval and training library, while the README describes RegionRAG’s region-level RAG workflow. The principal documented path is query and document-region retrieval, followed by focused generation; the package also provides contrastive training, retrieval/generation evaluation, and interpretability. The README supports the high-level RegionRAG stages, but the tree contains no dedicated RegionRAG application entrypoint or generator service, so those links are documented workflow rather than verified runtime calls.
Main entry points: retrieval_evaluator.py Retrieval evaluation · colmodel_training.py Training orchestration · processing_utils.py Visual processor · visual_retriever_collator.py Training collators
Components 21
Data and Processing
- Dataset preparation
colpali_engine/utils/dataset_transformation.py - Visual processor
colpali_engine/utils/processing_utils.py - Training collators
colpali_engine/collators/visual_retriever_collator.py - Corpus query batches
colpali_engine/collators/corpus_query_collator.py
Retrieval Models
- ColPali retrieval model
colpali_engine/models/paligemma/colpali/modeling_colpali.py - Model variants
colpali_engine/models - Token pooling
colpali_engine/compression/token_pooling/base_token_pooling.py - Multivector scoring
colpali_engine/utils/processing_utils.py
Training
- Training orchestration
colpali_engine/trainer/colmodel_training.py - Contrastive trainer
colpali_engine/trainer/contrastive_trainer.py - Region contrastive loss
colpali_engine/loss/late_interaction_losses.py
Retrieval and Generation
- Retrieval evaluation
colpali_engine/evaluation/retrieval_evaluator.py - Neighbor region grouping
colpali_engine/utils/processing_utils.py - Region similarity evaluation
colpali_engine/evaluation/region_sim_evaluator.py - Focused generation
colpali_engine/evaluation/generation_evaluator.py - Evaluation utilities
colpali_engine/evaluation/eval_utils.py - Similarity maps
colpali_engine/interpretability/similarity_maps.py
Other components
- Research user
- Training datasets
- Evaluation datasets
- Text query
Connections 23
- Research user → Text query: asks about
README.md - Training datasets → Dataset preparation: loads and transforms
colpali_engine/utils/dataset_transformation.py - Dataset preparation → Training collators: supplies examples
- Visual processor → Training collators: processes images
colpali_engine/collators/visual_retriever_collator.py - Visual processor → Corpus query batches: processes inputs
colpali_engine/collators/corpus_query_collator.py - Training collators → Corpus query batches: specializes
colpali_engine/collators/corpus_query_collator.py - Training orchestration → Corpus query batches: selects for corpus
colpali_engine/trainer/colmodel_training.py - Training orchestration → Contrastive trainer: runs training
colpali_engine/trainer/colmodel_training.py - Training orchestration → Region contrastive loss: configures loss
colpali_engine/trainer/colmodel_training.py - Training orchestration → Training datasets: trains on
README.md - ColPali retrieval model → Visual processor: pairs with processor
- Model variants → ColPali retrieval model: exports model
colpali_engine/models/__init__.py - Token pooling → Multivector scoring: prepares vectors for
- Text query → Retrieval evaluation: retrieves relevant regions
README.md - Evaluation datasets → Retrieval evaluation: evaluates against
colpali_engine/evaluation/retrieval_evaluator.py - Retrieval evaluation → Multivector scoring: scores vectors
- Retrieval evaluation → Neighbor region grouping: extracts regions
- Neighbor region grouping → Focused generation: provides focused context
README.md - Retrieval evaluation → Focused generation: passes retrieval results
README.md - Focused generation → Evaluation utilities: uses evaluation helpers
colpali_engine/evaluation/generation_evaluator.py - Retrieval evaluation → Region similarity evaluation: supports region evaluation
- Retrieval evaluation → Similarity maps: interprets similarity
- Research user → Training orchestration: trains retriever
README.md