How kotaro-kinoshita/yomitoku works
Project Type and Purpose
• YomiToku is a specialized full-stack Document AI engine aimed at processing Japanese document images. Its major functionality includes OCR, layout parsing, table structure recognition, and subsequent export of results in multiple formats (e.g., HTML, Markdown, CSV, JSON).
• It is designed to run efficiently even on GPU-equipped systems by leveraging PyTorch and CUDA, while being accessible via a CLI interface.
Analyzing the File Structure
• Top-level directories such as “src”, “configs”, “demo”, “docs”, “tests”, and “static” indicate a modular project design.
• The “src/yomitoku” directory is core and is broken into:
• CLI components (src/yomitoku/cli) for command-line execution.
• Configuration modules (src/yomitoku/configs) for different model settings (layout parser, text detector, table structure recognizer, text recognizer).
• Data handling modules (src/yomitoku/data) managing datasets and preprocessed functions.
• OCR and various document analysis modules such as “document_analyzer.py”, “layout_analyzer.py”, “layout_parser.py”, “ocr.py”, “table_structure_recognizer.py”, and “text_recognizer.py”.
• A “models” directory with subdirectories for specialized models (e.g., dbnet, parseq, rtdetr) and layers that encapsulate the architecture of each ML model.
• “postprocessor” modules to refine the raw outputs from recognition models.
• “export” modules defining multiple output formats.
• The “configs” directory hosts YAML configuration files specific to each model, hinting that the system is configurable based on specific task nuances.
• Additional folders like “docs” and “tests” inform documentation and automated testing practices.
Key Architectural Components
a. User Interface / Input:
• Command Line Interface (CLI) as found in src/yomitoku/cli/main.py.
• The user-provided input source (files/images/PDFs) leads into the processing pipeline.
b. Core Processing Engine (Document Analyzer):
• The Document Analyzer module (src/yomitoku/document_analyzer.py) sits at the heart of the system. This module coordinates OCR, layout parsing, table recognition, and reading order estimation.
c. Modular ML Model Components:
• OCR and Layout Analysis Modules:
• Text Detector (e.g. src/yomitoku/text_detector.py and associated configuration in configs).
• Text Recognizer (src/yomitoku/text_recognizer.py).
• Layout Parser / Analyzer (src/yomitoku/layout_parser.py and layout_analyzer.py).
• Table Structure Recognizer (src/yomitoku/table_structure_recognizer.py).
• These modules rely on underlying pretrained models built with PyTorch, such as “dbnet”, “parseq”, and “rtdetr” found in src/yomitoku/models.
d. Postprocessing and Utilities:
• Postprocessor functions (src/yomitoku/postprocessor) to refine ML outputs.
• Utility libraries (src/yomitoku/utils) for logging, visualization, graphing, and miscellaneous functionalities.
e. Export Layer:
• Modules under src/yomitoku/export for formatting results into CSV, HTML, JSON, and Markdown.
f. Configurations and Model Settings:
• Configuration files present in both the “configs” folder and within src/yomitoku/configs/ define various settings for model behavior and optimization.
g. External and Supporting Components:
• Dockerfile: Indicates containerization support.
• CI/CD workflows (.github/workflows) for testing, linting, and documentation builds.
• Demo scripts (in “demo”) for live analysis showcases.
• Documentation (docs) providing installation and operational guidelines.
Relationships and Data Flow
• Start from the User Input (CLI) which triggers the Document Analyzer.
• The Document Analyzer directs input data into the specialized OCR and layout modules.
• Each OCR component (text detection, recognition) will call its dedicated ML model and then pass data to the postprocessing modules.
• After processing, the Export modules format the output into desired formats.
• Support components (like utilities and configuration files) ensure that each function works with correct parameters and logs or visualizes activity.
• A clear data flow from input → processing (OCR/layout analysis) → postprocessing → export is established.
Architectural Patterns and Technologies
• Layered Architecture: Separates concerns among interface, processing logic, ML models, and output generation.
• Modularity: Each component (e.g., text detection, recognition, parsing, exporting) is built in separate modules, making the design scalable and maintainable.
• Configurable Pipeline: Use of YAML configuration files to adjust model settings across different document types.
• GPU Acceleration: Emphasize that the core model inference leverages PyTorch and CUDA for optimized performance.
Elements
• CLI / Input Handler
• Document Analyzer Core
• OCR / Text Detection
• Text Recognition
• Layout Analysis (including table structure and reading order)
• Postprocessing Unit
• Export Formatter Modules
• Configuration Management
• ML Model Components (with specific sub-items for “dbnet”, “parseq”, “rtdetr”)
• Utility and Logging Services
• From user inputs to CLI.
• From CLI to Document Analyzer, then sequentially through OCR and layout modules, and finally through postprocessors and exporters.
• Processing modules versus configuration/utility modules.
• External inputs/outputs (user files, exported results) versus internal pipelines (model inference, postprocessing).
• Use annotations where necessary to point out technologies such as PyTorch, CUDA, and show that the system supports GPU acceleration.