# mlvanguards/evaluate-ocr-rag-systems architecture

> Architecture diagram of the GitHub repository mlvanguards/evaluate-ocr-rag-systems, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mlvanguards/evaluate-ocr-rag-systems
- Repository: https://github.com/mlvanguards/evaluate-ocr-rag-systems

## Overview

##### Project Overview and Type
- This project is designed to evaluate OCR and retrieval-augmented generation (RAG) systems.
- Its purpose is to process PDF documents (from the “data” folder), apply OCR and layout analysis (via modules in src/ocr_benchmark and src/pdf_embedding_decider), and subsequently index and retrieve data using a Vespa-powered component (found under src/vespa). There’s also integration with specific schemas/configs (for example, the “colpal” directory and similar test variants).
- The overall structure suggests a pipeline that ingests PDFs, processes them, and then uses search/indexing and possibly inference modules to support RAG.

##### Main Components and Their Roles
a. Input & Data Layer:
- “data” directory: Contains PDF files to be processed.
- “logs”: For tracking processing events and errors (e.g., logs/failed_documents).
- “colpal” and similar test directories (test, test2, etc.): Provide configuration files like schemas, search-related XML files, and Vespa service definitions.

b. OCR and Preprocessing Layer:
- src/ocr_benchmark: Contains the logic for benchmarking OCR engines. Within this folder:
- “engines” (easy_ocr, config, etc.): Implements the available OCR approaches.
- “utils” (image_processing.py, pdf_processing.py): Handles PDF and image preprocessing.

c. PDF Embedding and Analysis Layer:
- src/pdf_embedding_decider: Focuses on analyzing PDFs, using components such as:
- layout_analyzer.py
- table_detector.py
- text_density_analyzer.py
- visual_element_analyzer.py
- This layer likely decides how to create embeddings or determine key structures for further processing.

d. Vespa Integration Layer:
- src/vespa: Contains “indexing” (with modules like pdf_processor.py, prepare_feed.py, run.py) and “retrieval” (retrieval/run.py) subfolders.
- This module configures and manages Vespa indexing, retrieval, and overall search services.
- Additional modules (vespa_config.yaml, model.py, etc.) suggest that Vespa is a core component in managing the RAG aspect.

e. Additional Modules:
- src/interfaces: Possibly containing abstract definitions or API endpoints (interfaces.py).
- src/prompts: Contains modules like llama_vision.py, suggesting there might be prompt generation for interfacing with LLMs or vision-based inference.
- Main scripts (main.py, vespa_inference.py, vespa_vision_rag.py) act as entry points into various workflows/pipelines.
- Test files (test.py, test_vespa_indexing.py, etc.) ensure that each component is verified independently.

##### Relationships and Data Flow
- The system starts by ingesting PDF files from the “data” directory.
- The OCR benchmark module extracts text/images using one or more OCR engines.
- Preprocessed PDF data is then passed to the PDF embedding decider, where layout, table, and visual analyses are conducted to determine embedding strategies.
- The processed and annotated content then gets fed into the Vespa indexing module, configuring and storing the data in a search/retrieval-friendly format.
- Retrieval processes (and possibly inference through vespa_inference.py or vespa_vision_rag.py) act on the indexed documents to support RAG operations.
- The “interfaces” and “prompts” modules could be used at integration points to interact with external systems or LLMs.

##### Architectural Patterns & Design Principles
- Modularization: The project is clearly split into distinct modules (OCR, PDF analysis, Vespa integration, etc.) which interact through well-defined interfaces.
- Pipeline Flow: There’s a sequential processing pipeline (data ingestion → OCR processing → embedding/analysis → indexing / retrieval) that should be clearly delineated.
- Configuration-Driven: The presence of multiple configuration directories and service definitions (e.g., in colpal and test* directories) indicates a reliance on external configuration to drive behavior.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion Layer
    subgraph "Data Ingestion Layer"
        A1["Data (PDF Files)"]:::dataLayer
        A2["Log Files"]:::dataLayer
        A3["Colpal Configurations"]:::dataLayer
        A4["Test Configurations"]:::dataLayer
        A5["Test2 Configurations"]:::dataLayer
        A6["Test3 Configurations"]:::dataLayer
        A7["Test4 Configurations"]:::dataLayer
    end

    %% OCR & Preprocessing Layer
    subgraph "OCR & Preprocessing Layer"
        B1["OCR Benchmark Module"]:::ocrLayer
        B2["OCR Engines & Configurations"]:::ocrLayer
        B3["Preprocessing Utilities"]:::ocrLayer
    end

    %% PDF Embedding & Analysis Layer
    subgraph "PDF Embedding & Analysis Layer"
        C1["PDF Embedding Decider Module"]:::pdfAnalysis
        C2["Core Analysis Script"]:::pdfAnalysis
        C3["Layout Analyzer"]:::pdfAnalysis
        C4["Table Detector"]:::pdfAnalysis
        C5["Text Density Analyzer"]:::pdfAnalysis
        C6["Visual Element Analyzer"]:::pdfAnalysis
    end

    %% Vespa Integration Layer
    subgraph "Vespa Integration Layer"
        D1["Vespa Module"]:::vespaLayer
        D2["Vespa Indexing Submodule"]:::vespaLayer
        D3["Vespa Retrieval Submodule"]:::vespaLayer
        D4["Vespa Config"]:::vespaLayer
        D5["Vespa Model"]:::vespaLayer
        D6["Vespa Utils"]:::vespaLayer
    end

    %% Additional Modules
    subgraph "Additional Modules"
        E1["Interfaces"]:::additionalLayer
        E2["Prompts Module"]:::additionalLayer
        E3["Main Application Scripts"]:::additionalLayer
        E4["Testing Scripts"]:::additionalLayer
        %% Main Application Subnodes (grouped under E3)
        E31["main.py"]:::additionalLayer
        E32["vespa_inference.py"]:::additionalLayer
        E33["vespa_vision_rag.py"]:::additionalLayer
        %% Testing Subnodes (grouped under E4)
        E41["test.py"]:::additionalLayer
        E42["test_vespa_indexing.py"]:::additionalLayer
        E43["test_vespa_inference.py"]:::additionalLayer
    end

    %% Connections Between Layers
    A1 -->|"PDF_Ingestion"| B1
    B1 -->|"OCR_Output"| C1
    C1 -->|"PDF_Embeddings"| D1
    D1 -->|"Search/Retrieval"| E3

    %% Internal Connections within Layers
    %% OCR & Preprocessing internal connections
    B1 --> B2
    B1 --> B3

    %% PDF Embedding internal connections
    C1 --> C2
    C1 --> C3
    C1 --> C4
    C1 --> C5
    C1 --> C6

    %% Vespa Integration internal connections
    D1 --> D2
    D1 --> D3
    D1 --> D4
    D1 --> D5
    D1 --> D6

    %% Additional Modules internal connections
    E3 --> E31
    E3 --> E32
    E3 --> E33
    E4 --> E41
    E4 --> E42
    E4 --> E43

    %% Click Events for Data Ingestion Layer
    click A1 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/data/"
    click A2 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/logs/failed_documents/"
    click A3 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/colpal/"
    click A4 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/test/"
    click A5 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/test2/"
    click A6 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/test3/"
    click A7 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/test4/"

    %% Click Events for OCR & Preprocessing Layer
    click B2 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/src/ocr_benchmark/engines/"
    click B3 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/src/ocr_benchmark/utils/"

    %% Click Events for PDF Embedding & Analysis Layer
    click C2 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/pdf_embedding_decider/analyze.py"
    click C3 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/pdf_embedding_decider/components/layout_analyzer.py"
    click C4 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/pdf_embedding_decider/components/table_detector.py"
    click C5 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/pdf_embedding_decider/components/text_density_analyzer.py"
    click C6 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/pdf_embedding_decider/components/visual_element_analyzer.py"

    %% Click Events for Vespa Integration Layer
    click D2 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/src/vespa/indexing/"
    click D3 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/tree/main/src/vespa/retrieval/"
    click D4 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/vespa/vespa_config.yaml"
    click D5 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/vespa/model.py"
    click D6 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/vespa/utils.py"

    %% Click Events for Additional Modules
    click E1 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/interfaces/interfaces.py"
    click E2 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/src/prompts/llama_vision.py"
    click E31 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/main.py"
    click E32 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/vespa_inference.py"
    click E33 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/vespa_vision_rag.py"
    click E41 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/test.py"
    click E42 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/test_vespa_indexing.py"
    click E43 "https://github.com/mlvanguards/evaluate-ocr-rag-systems/blob/main/test_vespa_inference.py"

    %% Styles
    classDef dataLayer fill:#fef2c0,stroke:#333,stroke-width:2px;
    classDef ocrLayer fill:#c0fef2,stroke:#333,stroke-width:2px;
    classDef pdfAnalysis fill:#c0dff2,stroke:#333,stroke-width:2px;
    classDef vespaLayer fill:#f2c0f2,stroke:#333,stroke-width:2px;
    classDef additionalLayer fill:#f2e0c0,stroke:#333,stroke-width:2px;
```
