How avarshn/mdrchallenge_2025_task1 works
Main Components
a. Data Ingestion & Storage
b. Preprocessing & Page Extraction
c. Embedding Generation (Image + Text)
d. Fusion & Embedding Store
e. Query Processing & Retrieval Engine
f. Results Output
External Dependencies
– Hugging Face Dataset (MMDocIR evaluation set)
– ColQwen-based Vision‑Language Model
– PyTorch & GPU Hosts (A100 / L40S)
Relationships & Data Flow
– Data Ingestion → Preprocessing
– Preprocessing → Embedding Generation
– Embedding Generation → Fusion → Embedding Store
– Query Text → Query Embedding → Retrieval Engine
– Retrieval Engine → Top‑5 Pages → Output
Architectural Patterns
– Pipeline pattern: linear stages from data → embeddings → retrieval
– Modular separation: each stage encapsulated in its own module (e.g., “task1_ColQwen2.py”)
– Late interaction retrieval: shows per‑page embedding vs. query interaction
Technologies & Frameworks
– Python 3.11 (conda environment)
– PyTorch 2.5.1 + CUDA 12.1
– Transformers / ColQwen VLM
– JSONL for ground‑truth & Hugging Face datasets
– Shell script (“run.sh”) to orchestrate end‑to‑end pipeline