# jjovalle99/colpali-rag-app architecture

> Architecture diagram of the GitHub repository jjovalle99/colpali-rag-app, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jjovalle99/colpali-rag-app
- Repository: https://github.com/jjovalle99/colpali-rag-app

## Overview

##### Project Type and Purpose
- This is a backend-only Retrieval-Augmented Generation (RAG) service built with FastAPI.
- Purpose: Ingest PDF documents as images, index them in a vector DB (Qdrant) using the ColPali method, and at query time retrieve relevant pages and feed them into a multimodal LLM (Claude Sonnet) for a natural-language response.

##### High-Level Architecture Components
a. Client / UI
- External web or script that uploads PDFs or issues query requests to the FastAPI endpoints.
b. FastAPI Backend (server.py + src/app/api)
- Endpoints:
- POST /pdf_ingest → kicks off ingestion.
- POST /query → performs similarity search + LLM call.
- Lifespan & state modules hold application startup/shutdown hooks and global singletons (Qdrant client, Supabase client, model loader).
- Dependency layer wires up per-request clients.
c. Ingestion Subsystem
- PDF→JPEG conversion: pdf2image in src/app/colpali/loaders.py
- Image upload: img_uploader service → Supabase object storage
- Embedding: ColQwen 2.5 via colpali-engine → vector embeddings
- Vector write: Qdrant via qdrant_utils in src/app/utils
d. Inference Subsystem
- Query embedding: same ColQwen 2.5 model
- Vector search: Qdrant returns top-K metadata (image keys)
- Image download: img_downloader service from Supabase
- LLM invocation: Claude Sonnet 3.7 via Anthropics
e. Persistence & External Services
- Qdrant (vector database)
- Supabase (object storage for images)
- Anthropics API (LLM calls)

##### Relationships & Data Flows
[Client] → HTTP POST /pdf_ingest → [FastAPI Ingestion Endpoint] →
→ pdf2image → [Local JPEGs] → img_uploader → [Supabase Storage] →
→ colpali loader → embeddings → qdrant_utils → [Qdrant Collection]
[Client] → HTTP POST /query → [FastAPI Query Endpoint] →
→ embed query → [Qdrant Search] → list of image IDs →
→ img_downloader → [Supabase] → images → Claude Sonnet → [Response JSON]

##### Architectural Patterns & Principles
- Layered Design: Presentation (API) → Application services → Integration utilities → External systems
- Single Responsibility:
- api/endpoints only parse requests & orchestrate calls
- services handle I/O (upload/download)
- utils handle DB/client wrappers
- colpali/loaders handle conversion & embedding
- Dependency Injection via FastAPI’s Depends for clients (Qdrant, Supabase, Anthropics).
- Containerized Deployment: Dockerfile + Makefile for build/run.

## Diagram (Mermaid)

```mermaid
flowchart TB
    classDef api fill:#e0f7fa,stroke:#0288d1,stroke-width:2px
    classDef colpali fill:#a7f3d0,stroke:#2e7d32,stroke-width:2px
    classDef pers fill:#ffe0b2,stroke:#f57c00,stroke-width:2px
    classDef llm fill:#e1bee7,stroke:#8e24aa,stroke-width:2px
    classDef ext fill:#cfd8dc,stroke:#455a64,stroke-width:2px
    classDef utils fill:#fff9c4,stroke:#fbc02d,stroke-width:2px
    classDef infra fill:#f0f4c3,stroke:#827717,stroke-width:2px

    Client["Client / UI"]:::ext

    subgraph "Docker Container"
        subgraph "API Layer"
            Server["FastAPI Server\nserver.py"]:::api
            PDFIngest["POST /pdf_ingest\npdf_ingest.py"]:::api
            QueryEndpoint["POST /query\nquery.py"]:::api
            Dependencies["Dependencies\ndependencies.py"]:::api
            Lifespan["Lifespan Hooks\nlifespan.py"]:::api
            State["Global State\nstate.py"]:::api
        end

        subgraph "Ingestion Subsystem"
            Loader["PDF→JPEG + Embedding\nloaders.py"]:::colpali
            Uploader["Image Upload\nimg_uploader.py"]:::api
            QWrite["Vector Write\nqdrant_utils.py"]:::utils
        end

        subgraph "Inference Subsystem"
            QueryEmbed["Query Embedding\nloaders.py"]:::colpali
            QSearch["Vector Search\nqdrant_utils.py"]:::utils
            Downloader["Image Download\nimg_downloader.py"]:::api
            LLMInvoke["LLM Invocation\nvia Anthropics"]:::llm
        end

        subgraph "Utility & Support"
            PromptUtils["Prompt Utils\nprompt_utils.py"]:::utils
            Settings["Configuration\nsettings.py"]:::utils
        end

        subgraph "Infra & DevOps"
            CreateColl["Qdrant Collection Setup\ncreate_collection.py"]:::infra
            DockerfileNode["Dockerfile"]:::infra
            MakefileNode["Makefile"]:::infra
            EnvFiles[".env / .gitignore"]:::infra
        end
    end

    QdrantDB["Qdrant\n(Vector DB)"]:::pers
    SupabaseStorage["Supabase\n(Object Storage)"]:::pers
    AnthropicAPI["Anthropic API\n(Claude Sonnet)"]:::llm

    %% Ingestion Flow
    Client -->|"POST /pdf_ingest"| PDFIngest
    PDFIngest -->|"pdf2image conversion"| Loader
    Loader -->|"upload via supabase-py"| Uploader
    Uploader --> SupabaseStorage
    Loader -->|"embed via colpali-engine"| QWrite
    QWrite --> QdrantDB

    %% Inference Flow
    Client -->|"POST /query"| QueryEndpoint
    QueryEndpoint -->|"embed query"| QueryEmbed
    QueryEmbed -->|"vector search"| QSearch
    QSearch --> QdrantDB
    QSearch -->|"download via supabase-py"| Downloader
    Downloader --> SupabaseStorage
    Downloader -->|"invoke LLM"| LLMInvoke
    LLMInvoke --> AnthropicAPI
    AnthropicAPI -->|"response JSON"| QueryEndpoint

    %% Click Events
    click Server "https://github.com/jjovalle99/colpali-rag-app/blob/main/server.py"
    click PDFIngest "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/api/endpoints/pdf_ingest.py"
    click QueryEndpoint "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/api/endpoints/query.py"
    click Dependencies "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/api/dependencies.py"
    click Lifespan "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/api/lifespan.py"
    click State "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/api/state.py"
    click Loader "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/colpali/loaders.py"
    click Uploader "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/services/img_uploader.py"
    click QWrite "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/utils/qdrant_utils.py"
    click QueryEmbed "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/colpali/loaders.py"
    click QSearch "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/utils/qdrant_utils.py"
    click Downloader "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/services/img_downloader.py"
    click PromptUtils "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/utils/prompt_utils.py"
    click Settings "https://github.com/jjovalle99/colpali-rag-app/blob/main/src/app/settings.py"
    click CreateColl "https://github.com/jjovalle99/colpali-rag-app/blob/main/scripts/create_collection.py"
    click DockerfileNode "https://github.com/jjovalle99/colpali-rag-app/tree/main/Dockerfile"
    click MakefileNode "https://github.com/jjovalle99/colpali-rag-app/tree/main/Makefile"
    click EnvFiles "https://github.com/jjovalle99/colpali-rag-app/blob/main/.env"
    click EnvFiles "https://github.com/jjovalle99/colpali-rag-app/blob/main/.gitignore"
```
