How vikparuchuri/surya works
Main Components
• Input & Preprocessing Layer: Represented by files in the “surya/input” directory as well as configuration (“surya/settings.py”). This module handles loading images, PDFs, and any preprocessing (e.g., deskewing, binarizing) before sending to OCR pipelines.
• Text / Line Detection Module: Found in “surya/detection” (and benchmark/detection.py). Here the system detects where text is located in the document image. Include components like “affinity,” “heatmap,” and post-processing utilities.
• Layout Analysis & Reading Order: Shown by the “surya/layout” module that handles detecting various document regions (tables, headers, images, text blocks) and organizing the detected text blocks into a coherent reading order.
• Text Recognition: Located in “surya/recognition,” this module performs OCR on the detected text regions and supports 90+ languages. Include the use of models, encoders/decoders, tokenizers, and postprocessing.
• Table Recognition: The “surya/table_rec” directory captures the table extraction functionalities – it detects rows, columns, cells, and handles spanning items.
• LaTeX OCR Extraction: (in “surya/texify”) dedicated to recognizing equations and converting them into LaTeX-compatible output.
• Common Utilities & Debug Components: The “surya/common” (with submodules for shared utilities like S3 integration, polygon functions, etc.) and “surya/debug” modules support logging, drawing, and visual verification of predictions.
• Auxiliary Tools: A variety of command-line utilities (e.g., ocr_app.py, detect_layout.py, texify_app.py) and scripts in “surya/scripts” provide entry points for running prediction pipelines and even interactive streamlit interfaces.
• Benchmarking & Testing: The “benchmark” directory and “tests” folder provide frameworks for performance evaluation and quality tests. They resemble separate subsystems in the overall design.
Relationships & Data Flow
• The Input Handler loads the document (via surya/input) and, optionally, applies preliminary processing.
• The system then channels the detected regions to multiple specialized OCR modules:
– Recognition for generic text.
– Table Recognition for structured data extraction.
– LaTeX OCR for mathematical expressions.
• Throughout, helper libraries (in surya/common, surya/util, etc.) supply model configuration, data formatting, and I/O support.
• Results are aggregated and output (as JSON files with text, positions, and confidence scores) per the command usage illustrated in the README.
• Additionally, benchmarking tools and testing scripts validate performance and accuracy independently of production flow.
Architectural Patterns & Design Principles
• Modular Design: Each “feature” (detection, recognition, layout, table recognition, LaTeX OCR) is separated into its own module, allowing for independent development, testing, and potential compilation (as seen with the COMPILE_* environment variables).
• Extensibility: The use of common utilities and a dedicated scripts folder in “surya” indicates that the system is designed to be extensible both from development and operational perspectives.
Relevant Technologies & Frameworks
• Python 3.10+ and PyTorch (as primary frameworks for model training and inference).
• Streamlit for the interactive UI.
• Utilization of external libraries for model architectures (referencing Donut, EfficientViT, and timm).
• CI/CD pipelines (GitHub workflows) indicate automated testing and deployment.
• Environment variable driven configuration (e.g., TORCH_DEVICE, batch sizes, compile flags).
• Use clear, descriptive labels for each component/module: “Input Handler”, “Text Detection”, “Layout Analysis & Reading Order”, “Text Recognition”, “Table Recognition”, “LaTeX OCR”, “Common Utilities”, “Debug & Visualization Tools”, “CLI/Interactive App”, and “Benchmark & Testing Modules.”
• User/CLI input → Input Handler → (branching into) Detection, Layout Analysis.
• Detection output → Recognition module.
• Layout output → (branching into) Reading Order, Table Extraction.
• Detection/Layout outputs → LaTeX OCR (for equations, from pre-cropped regions).
• Data flow with key elements like JSON outputs (with bboxes, texts, and confidences) and include notes on performance tuning areas (batch sizes, GPU/CPU differentiation).
• External integrations (like S3 for weight downloads, or the hosted API in the README) as separate components linking to the main pipeline.