How datalab-to/surya works
Project Type and Purpose
• Surya is an open-source, modular document OCR toolkit (Python library + CLI + optional interactive Streamlit apps) that provides pipelines for:
• Text detection
• Multilingual OCR
• Layout analysis & reading order
• Table recognition
• LaTeX equation OCR
• It is not a monolithic web app; rather a library with command-line scripts (surya_detect, surya_ocr, surya_layout, etc), Python API (Predictor classes), and optional Streamlit UIs.
High-Level File Structure & Architectural Layers
• Top-level:
• surya/ – main Python package, subdivided by task (detection, recognition, layout, table_rec, ocr_error, common utilities, debug aids)
• CLI wrappers & convenience scripts: detect_layout.py, ocr_app.py, texify_app.py, plus surya/scripts/ for Streamlit entrypoints
• benchmark/ – benchmarking harness for each task
• static/ – fonts/images for documentation & apps
• CI/CD: .github/workflows/*
• Tests: tests/
• Config: surya/settings.py, pyproject.toml, pytest.ini
A. Input Layer
• Supported inputs: image files, PDFs, Word docs, PowerPoints
• Preprocessing: loaders in surya/input/*
B. Core Task Modules (each as a separate “micro-pipeline”)
1. Detection Predictor
• surya/detection/
• Model: semantic segmentation (EfficientViT/Craft), heatmap → bboxes
2. Recognition Predictor
• surya/recognition/
• Models: Donut-derived encoder/decoder for line-level OCR
3. Layout Predictor & Reading Order
• surya/layout/
• Models: encoder/decoder for classifying regions, assigning read positions
4. TableRec Predictor
• surya/table_rec/
• Models: encoder/decoder for rows/cols/cells
5. Texify Predictor (LaTeX OCR)
• surya/common/donut + surya/texify.py
C. Shared Utilities
• surya/common/ – model loaders, config readers, S3 weight downloader, polygon & bbox utils
• surya/logging.py, surya/settings.py
D. CLI & API Layer
• Python entrypoints (surya/scripts/…, console_scripts in pyproject.toml)
• User calls pipeline:
Input → Task Predictor → JSON + optional annotated images
E. Optional Interactive GUI
• Streamlit apps (surya/scripts/streamlit_app.py, texify_gui)
• Uses same Predictor classes under the hood
F. Benchmarking & CI/CD
• benchmark/ modules run pipelines at scale, produce charts
• GitHub Actions for CI, benchmarks, CLA, publishing
Relationships and Data Flow
• branches to [Recognition] for text OCR
• branches to [Layout] for region classification & reading order
• branches to [TableRec] for table cell extraction
• branches to [Texify] for LaTeX if equations are cropped
• Each Predictor uses shared utilities (common) and model configs, downloads weights from S3 as needed
• CLI scripts orchestrate these calls; Streamlit UIs wrap the same pipelines
Architectural Patterns & Design Principles
• Modular monolith / plugin-style: each OCR task is a self-contained subpackage
• Pipeline pattern: sequential processing stages, with clear inputs/outputs (JSON schemas in schema.py)
• Shared core utilities for DRY (config loading, logging, bbox math)
• Config-driven: surya/settings.py + environment-variable overrides
• Optional compilation flags for JIT compiling PyTorch models
Technologies, Frameworks, External Services
• Python 3.10+, PyTorch, HuggingFace Transformers / Donut, timm, EfficientViT
• Streamlit for interactive UIs
• S3 (via surya/common/s3.py) for model weight hosting
• CI/CD: GitHub Actions
• Packaging: Poetry (pyproject.toml)
• Testing: pytest
Essential Elements to Capture
• All five task modules as separate components
• Shared common utilities & config layer
• CLI entrypoints vs Python API
• Streamlit GUI as an alternative front end
• Benchmarking harness & CI/CD pipeline