# dicklesworthstone/llm_aided_ocr architecture

> Architecture diagram of the GitHub repository dicklesworthstone/llm_aided_ocr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dicklesworthstone/llm_aided_ocr
- Repository: https://github.com/dicklesworthstone/llm_aided_ocr

## Overview

##### Project Type and Purpose
- This project is a specialized command-line application (with optional Docker deployment) focused on enhancing OCR outputs. It takes a PDF, converts it to images, extracts raw text using Tesseract OCR, and then uses LLMs (local and/or API-based) to correct errors and format the text, optionally into Markdown.
- The overall purpose is to transform raw OCR output into a polished, human-readable document with added quality assurance and optional formatting enhancements.

##### File Structure
- Top-level files include Python scripts (llm_aided_ocr.py and the CLI variant), configuration files (requirements.txt, .python-version, .gitignore), and Docker configuration under the "docker" folder. This confirms that the tool is designed to be used both directly via the CLI and containerized.
- The existence of various PDF and text examples in the file tree also indicates that input and output file handling is an important part of the design.
- There is no explicit separation into “frontend” and “backend,” which indicates a single-process command-line tool with clearly modular functions for different processing stages.

##### README for Architectural Details
- The README breaks down the process into multiple logical components:
- PDF Processing: Converting PDFs to images (using pdf2image).
- OCR Extraction: Utilizing Tesseract for text extraction.
- Text Chunking: Splitting the OCR text into manageable segments.
- Error Correction/Formatting: Employing LLMs to correct errors and optionally convert to Markdown.
- Output generation: Saving raw OCR outputs and final corrected/ formatted text.
- Additional Steps: Duplicate content removal, token management with adaptive adjustments, quality assessment, asynchronous processing for API-based LLM calls, and comprehensive logging.

a. Main Components to Include:
- Input Module: File loader that handles PDF files.
- PDF to Image Conversion Module: Illustrate the function that converts PDF pages into images.
- OCR Module: A component that uses Tesseract (with functions like ocr_image() and preprocess_image()).
- Text Processing Pipeline: Representing the chunk creation (process_document()), error correction (process_chunk()) and Markdown formatting steps.
- LLM Integration Module: This should branch into two options:
- Local LLM Processing (using libraries like llama_cpp).
- API-based LLM Processing (with functions like generate_completion_from_openai() and generate_completion_from_claude()), alongside asynchronous processing.
- Quality Assessment Module: To evaluate and compare the OCR output quality.
- Output Module: Saving files as {base_name}__raw_ocr_output.txt and {base_name}_llm_corrected.md.
- Logging and Error Handling: A centralized logging mechanism that monitors each step.

b. Relationships and Interactions:
- From OCR Module, output goes into the Text Processing Pipeline, which splits the text for LLM-based error correction.
- The LLM Integration Module has access to configuration (local vs. API-based) and interacts asynchronously if needed.
- After correction, the processed text is then sent to the Output module.
- A quality assessment block is also inserted right before final output to compute quality metrics.
- All modules should specify their dependencies (e.g., pdf2image, pytesseract, asyncio for async calls, and logging libraries).

c. Architectural Patterns and Design Principles:
- Separation of Concerns: Separate functional blocks for OCR, text processing, LLM integration, quality assessment, and output handling.
- Modularity: Each function (e.g., image preprocessing, chunk creation, token management) should represent a distinct module.
- Asynchronous and Concurrent Processing: Especially for API calls—this should be highlighted with specific elements (e.g., “Async Handler”).
- Configurability: Emphasize the usage of environment variables and Docker for containerized deployment.

d. Technologies, Frameworks, and Libraries:
- Python 3.12+ as the language runtime.
- Tesseract OCR, pdf2image for PDF conversion, and pytesseract for OCR.
- LLM libraries (e.g., llama_cpp for local LLM inference) and API tooling for OpenAI and Anthropic integration.
- Asynchronous processing via asyncio.
- Docker for containerization, as indicated by the Dockerfile and docker-compose.yml.

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["User PDF Input"]:::user
    B["Input Processing & PDF Conversion"]:::processing
    C["OCR Module (Tesseract)"]:::processing
    D["Text Processing & LLM Integration"]:::processing
    E["Local LLM Processing"]:::integration
    F["API-based LLM Processing"]:::integration
    G["Async Handler"]:::integration
    H["Quality Assessment"]:::quality
    I["Output Generation"]:::io
    J["CLI Interface"]:::interface
    K["Deployment & Containerization"]:::deployment

    A -->|"loadPDF"| B
    B -->|"convertPDFtoImages"| C
    C -->|"extractText"| D
    D -->|"useLocalLLM"| E
    D -->|"useAPILLM"| F
    F -->|"asyncProcess"| G
    E -->|"mergeResults"| H
    G -->|"mergeResults"| H
    H -->|"generateOutputs"| I
    J -->|"triggersProcessing"| B
    K -->|"providesEnv"| B

    click B "https://github.com/dicklesworthstone/llm_aided_ocr/blob/main/llm_aided_ocr.py"
    click C "https://github.com/dicklesworthstone/llm_aided_ocr/blob/main/llm_aided_ocr.py"
    click D "https://github.com/dicklesworthstone/llm_aided_ocr/blob/main/llm_aided_ocr.py"
    click J "https://github.com/dicklesworthstone/llm_aided_ocr/blob/main/llm-aided-ocr-cli.py"
    click K "https://github.com/dicklesworthstone/llm_aided_ocr/tree/main/docker/"
    click I "https://github.com/dicklesworthstone/llm_aided_ocr/tree/main/Root-level generated files"

    classDef user fill:#FFCDD2,stroke:#C62828,stroke-width:2px;
    classDef processing fill:#FFE0B2,stroke:#E65100,stroke-width:2px;
    classDef integration fill:#C5E1A5,stroke:#558B2F,stroke-width:2px;
    classDef quality fill:#BBDEFB,stroke:#1E88E5,stroke-width:2px;
    classDef io fill:#F0F4C3,stroke:#827717,stroke-width:2px;
    classDef interface fill:#D1C4E9,stroke:#512DA8,stroke-width:2px;
    classDef deployment fill:#B3E5FC,stroke:#039BE5,stroke-width:2px;
```
