How dicklesworthstone/llm_aided_ocr works
Project Type and Purpose
• This project is a specialized command-line application (with optional Docker deployment) focused on enhancing OCR outputs. It takes a PDF, converts it to images, extracts raw text using Tesseract OCR, and then uses LLMs (local and/or API-based) to correct errors and format the text, optionally into Markdown.
• The overall purpose is to transform raw OCR output into a polished, human-readable document with added quality assurance and optional formatting enhancements.
File Structure
• Top-level files include Python scripts (llm_aided_ocr.py and the CLI variant), configuration files (requirements.txt, .python-version, .gitignore), and Docker configuration under the "docker" folder. This confirms that the tool is designed to be used both directly via the CLI and containerized.
• The existence of various PDF and text examples in the file tree also indicates that input and output file handling is an important part of the design.
• There is no explicit separation into “frontend” and “backend,” which indicates a single-process command-line tool with clearly modular functions for different processing stages.
README for Architectural Details
• The README breaks down the process into multiple logical components:
• PDF Processing: Converting PDFs to images (using pdf2image).
• OCR Extraction: Utilizing Tesseract for text extraction.
• Text Chunking: Splitting the OCR text into manageable segments.
• Error Correction/Formatting: Employing LLMs to correct errors and optionally convert to Markdown.
• Output generation: Saving raw OCR outputs and final corrected/ formatted text.
• Additional Steps: Duplicate content removal, token management with adaptive adjustments, quality assessment, asynchronous processing for API-based LLM calls, and comprehensive logging.
a. Main Components to Include:
• Input Module: File loader that handles PDF files.
• PDF to Image Conversion Module: Illustrate the function that converts PDF pages into images.
• OCR Module: A component that uses Tesseract (with functions like ocr_image() and preprocess_image()).
• Text Processing Pipeline: Representing the chunk creation (process_document()), error correction (process_chunk()) and Markdown formatting steps.
• LLM Integration Module: This should branch into two options:
• Local LLM Processing (using libraries like llama_cpp).
• API-based LLM Processing (with functions like generate_completion_from_openai() and generate_completion_from_claude()), alongside asynchronous processing.
• Quality Assessment Module: To evaluate and compare the OCR output quality.
• Output Module: Saving files as {base_name}__raw_ocr_output.txt and {base_name}_llm_corrected.md.
• Logging and Error Handling: A centralized logging mechanism that monitors each step.
b. Relationships and Interactions:
• From OCR Module, output goes into the Text Processing Pipeline, which splits the text for LLM-based error correction.
• The LLM Integration Module has access to configuration (local vs. API-based) and interacts asynchronously if needed.
• After correction, the processed text is then sent to the Output module.
• A quality assessment block is also inserted right before final output to compute quality metrics.
• All modules should specify their dependencies (e.g., pdf2image, pytesseract, asyncio for async calls, and logging libraries).
c. Architectural Patterns and Design Principles:
• Separation of Concerns: Separate functional blocks for OCR, text processing, LLM integration, quality assessment, and output handling.
• Modularity: Each function (e.g., image preprocessing, chunk creation, token management) should represent a distinct module.
• Asynchronous and Concurrent Processing: Especially for API calls—this should be highlighted with specific elements (e.g., “Async Handler”).
• Configurability: Emphasize the usage of environment variables and Docker for containerized deployment.
d. Technologies, Frameworks, and Libraries:
• Python 3.12+ as the language runtime.
• Tesseract OCR, pdf2image for PDF conversion, and pytesseract for OCR.
• LLM libraries (e.g., llama_cpp for local LLM inference) and API tooling for OpenAI and Anthropic integration.
• Asynchronous processing via asyncio.
• Docker for containerization, as indicated by the Dockerfile and docker-compose.yml.