# avarshn/mdrchallenge_2025_task1 architecture

> Architecture diagram of the GitHub repository avarshn/mdrchallenge_2025_task1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/avarshn/mdrchallenge_2025_task1
- Repository: https://github.com/avarshn/mdrchallenge_2025_task1

## Overview

##### Main Components
a. Data Ingestion & Storage
b. Preprocessing & Page Extraction
c. Embedding Generation (Image + Text)
d. Fusion & Embedding Store
e. Query Processing & Retrieval Engine
f. Results Output

##### External Dependencies
– Hugging Face Dataset (MMDocIR evaluation set)
– ColQwen-based Vision‑Language Model
– PyTorch & GPU Hosts (A100 / L40S)

##### Relationships & Data Flow
– Data Ingestion → Preprocessing
– Preprocessing → Embedding Generation
– Embedding Generation → Fusion → Embedding Store
– Query Text → Query Embedding → Retrieval Engine
– Retrieval Engine → Top‑5 Pages → Output

##### Architectural Patterns
– Pipeline pattern: linear stages from data → embeddings → retrieval
– Modular separation: each stage encapsulated in its own module (e.g., “task1_ColQwen2.py”)
– Late interaction retrieval: shows per‑page embedding vs. query interaction

##### Technologies & Frameworks
– Python 3.11 (conda environment)
– PyTorch 2.5.1 + CUDA 12.1
– Transformers / ColQwen VLM
– JSONL for ground‑truth & Hugging Face datasets
– Shell script (“run.sh”) to orchestrate end‑to‑end pipeline

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Systems
    HF["Hugging Face Dataset"]:::external
    GPU["GPU Cluster (A100/L40S)"]:::external
    Ref["Workflow Reference Diagram"]:::external

    %% Data Store
    JSONL["data_task1/MMDocIR_gt_remove.jsonl"]:::datastore

    %% Code Modules Group
    subgraph "src/ (code modules)" 
        style src/ stroke-dasharray: 5 5
        Preprocessor["Preprocessing Module<br/>(pdf → pages as images/text)"]:::compute
        ImageEncoder["Image Encoder<br/>(ColQwen visual)"]:::compute
        TextEncoder["Text Encoder<br/>(ColQwen text)"]:::compute
        Fusion["Embedding Fusion Module<br/>(concatenate & project)"]:::compute
        QueryProc["Query Processing Module<br/>(Text query → text embedding)"]:::compute
        RetrievalEngine["Retrieval Engine<br/>(late interaction similarity)"]:::compute
        Results["Results: Top-5 Pages"]:::datastore
    end

    Runner["Pipeline Orchestration<br/>(run.sh)"]:::compute
    Req["requirements.txt"]:::datastore

    %% Flow
    HF -->|raw dataset| JSONL
    JSONL -->|JSONL data| Preprocessor
    Preprocessor -->|"pages as images/text"| ImageEncoder
    Preprocessor -->|"pages as text"| TextEncoder
    ImageEncoder -->|"image embeddings"| Fusion
    TextEncoder -->|"text embeddings"| Fusion
    Fusion -->|"fused embeddings"| EmbedStore[(model_cache)]:::datastore
    EmbedStore[(model_cache)]:::datastore -->|stored embeddings| RetrievalEngine
    QueryProc -->|"query embedding"| RetrievalEngine
    RetrievalEngine -->|"top-5 pages"| Results

    Runner -.->|orchestrates| Preprocessor
    GPU -->|compute host| Runner
    Ref -.->|reference| Runner
    Req -.->|env spec| Runner

    %% Click Events
    click HF "https://github.com/avarshn/mdrchallenge_2025_task1/tree/main/N/A"
    click JSONL "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/data_task1/MMDocIR_gt_remove.jsonl"
    click Preprocessor "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/src/task1_ColQwen2.py"
    click ImageEncoder "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/src/task1_ColQwen2.py"
    click TextEncoder "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/src/task1_ColQwen2.py"
    click Fusion "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/src/task1_ColQwen2.py"
    click QueryProc "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/src/task1_ColQwen2.py"
    click RetrievalEngine "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/src/task1_ColQwen2.py"
    click Results "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/src/task1_ColQwen2.py"
    click Runner "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/src/run.sh"
    click Ref "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/misc/retrieval_workflow.png"
    click Req "https://github.com/avarshn/mdrchallenge_2025_task1/blob/main/requirements.txt"

    %% Styles
    classDef external fill:#FFA500,stroke:#333,stroke-width:1px
    classDef compute fill:#9FE2BF,stroke:#333,stroke-width:1px
    classDef datastore fill:#87CEFA,stroke:#333,stroke-width:1px
```
