# softlandia-ltd/vision-is-all-you-need architecture

> Architecture diagram of the GitHub repository softlandia-ltd/vision-is-all-you-need, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/softlandia-ltd/vision-is-all-you-need
- Repository: https://github.com/softlandia-ltd/vision-is-all-you-need

## Overview

##### Data Flow Representation

a) Document Processing Flow:
**PDF Upload → PDF to Image Conversion → VLM Processing → Vector Storage**

b) Query Flow:
**User Query → Query Embedding → Vector Search → Image+Query Processing → Response Generation**

##### Specific Component Details

- File Upload Interface
- Question Interface
- Attention Map Visualization
- API Client Module

- PDF Processing Service (pdf_to_image.py)
- ColPali VLM Service (colpali.py)
- Vector Database Client (qdrant_client.py)
- Main RAG Logic (vrag.py)

- OpenAI GPT-4V
- Hugging Face Transformers
- QDrant Vector DB

##### Important Technical Details

Annotations showing:
- TypeScript/React for frontend
- Python 3.11+ for backend
- Modal deployment platform
- Vector embedding process
- PDF to image conversion using pypdfium
- ColPali VLM implementation
- GPT-4V/GPT-4V-mini integration

##### Architecture Patterns to Highlight

- RESTful API communication between frontend and backend
- Asynchronous processing for PDF handling
- Vector similarity search patterns
- Vision-language model pipeline

##### Additional Notes

- Version information for key components
- Scaling considerations
- Security implementations (API keys, authentication)
- Development vs. production configurations

Remember to maintain a balance between high-level architecture and important implementation details, as this system has unique characteristics in its vision-based RAG approach that should be clearly represented.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client
        App["Frontend App"]:::frontend
        Upload["File Upload Interface"]:::frontend
        Question["Question Interface"]:::frontend
        AttMap["Attention Map"]:::frontend
        ColorToggle["Color Scheme Toggle"]:::frontend
        APIClient["API Client"]:::frontend
        Theme["Theme"]:::frontend
        Styles["Global Styles"]:::frontend
        Header["Header"]:::frontend
    end

    subgraph Backend
        MainEntry["System Entry"]:::backend
        BackendApp["Backend App"]:::backend
        PDFProcessor["PDF Processing Service"]:::backend
        VLMService["ColPali VLM Service"]:::backend
        VectorDB["Vector Database Client"]:::backend
        RAGLogic["Main RAG Logic"]:::backend
    end

    subgraph External
        OpenAI["OpenAI GPT-4V"]:::external
        HuggingFace["Hugging Face"]:::external
        QDrant["QDrant Vector DB"]:::external
    end

    App --> Upload
    App --> Question
    App --> AttMap
    App --> ColorToggle
    App --> Header
    App --> APIClient
    App --> Theme
    App --> Styles

    APIClient --> BackendApp
    BackendApp --> MainEntry
    MainEntry --> PDFProcessor
    MainEntry --> VLMService
    MainEntry --> VectorDB
    MainEntry --> RAGLogic

    VLMService --> OpenAI
    VLMService --> HuggingFace
    VectorDB --> QDrant

    %% Click Events
    click App "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/frontend/src/App.tsx"
    click Upload "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/frontend/src/Dropzone/Dropzone.tsx"
    click AttMap "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/frontend/src/AttentionMap/AttentionMap.tsx"
    click Question "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/frontend/src/Question/Question.tsx"
    click ColorToggle "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/frontend/src/ColorSchemeToggle/ColorSchemeToggle.tsx"
    click APIClient "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/frontend/src/api/index.ts"
    click PDFProcessor "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/vrag/pdf_to_image.py"
    click VLMService "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/vrag/colpali.py"
    click VectorDB "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/vrag/qdrant_client.py"
    click RAGLogic "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/vrag/vrag.py"
    click BackendApp "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/vrag/app.py"
    click MainEntry "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/main.py"
    click Theme "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/frontend/src/theme.ts"
    click Styles "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/frontend/src/styles/global.css.ts"
    click Header "https://github.com/Softlandia-Ltd/vision-is-all-you-need/blob/main/frontend/src/Header/Header.tsx"

    %% Styling
    classDef frontend fill:#3498db,stroke:#2980b9,color:white
    classDef backend fill:#2ecc71,stroke:#27ae60,color:white
    classDef external fill:#9b59b6,stroke:#8e44ad,color:white

    %% Legend
    subgraph Legend
        Frontend["Frontend Component"]:::frontend
        Backend["Backend Service"]:::backend
        External["External Service"]:::external
    end
```
