How ds4v/nomnaocr works
Main Components
• Data Ingestion & Collection:
• Components: “Data collection” folder with workflow.json and automa2txt.py.
• Inputs: Web pages/images of historical texts (collected via Automa) and external download tools like Internet Download Manager.
• Output: Raw image URLs and extracted text files (nom.txt, modern.txt).
• Data Labeling & Annotation:
• Components: “Data labeling” directory containing Auto annotation folders, PPOCRLabel tool (annotators.zip and composer.zip), and label guidelines (for both poem and prose).
• Data Splitting & Preparation:
• Components: “Data splitting” folder with various Python scripts (IHRNomDB_Rs.py, split_pages.py, split_patches.py) and notebooks (statistics.ipynb).
• Functionality: Ensure the distribution of pages/patches is balanced for training and validation, aligning with the IHR-NomDB distribution technique.
• Text Detection Module:
• Components: “Text detection” folder containing configuration files (DBNet.yml, EAST.yml) and evaluation notebooks (CLEval.ipynb).
• Functionality: Train and evaluate detection models (using configurations compatible with PaddleOCR).
• Text Recognition Module:
• Components: “Text recognition” sub-folders (e.g., CNNxSeq2Seq, CRNNxCTC, TransformerOCR, ImageCaptioning).
• Functionality: Notebooks for both pretraining (using synthetic Nom String datasets) and fine-tuning; vary in architecture – some include Skip Connection variations (Addition and Concatenation) to enhance performance.
• External scripts: evaluator.py, loss functions, and metrics modules to guide training evaluation.
• Preprocessing and Utilities:
• Files such as preprocess_generator.py and common utilities in text recognition (loader.py, utils.py, visualizer.py) that help process images and text inputs.
• Evaluation and Model Deployment:
• Components: Evaluation pipeline (illustrated by evaluation_pipeline_vi.jpg) using CLEval for combined Text Detection and End-to-End evaluation.
• Additional outputs: Links to pretrained weights and references to performance metrics – Sequence Accuracy, Character Accuracy, and Character Error Rate.
Establish Relationships & Data Flow
• Labeled data then flows into the Data Splitting module, which partitions the dataset into Train and Validate sets.
• From there, the pipeline splits into two parallel streams:
• One stream for Text Detection, making use of detection-specific configurations and training modules.
• The other stream for Text Recognition, which itself branches into several model architectures (CNNxSeq2Seq, CRNNxCTC, TransformerOCR) with respective pretraining and fine-tuning processes.
• Finally, both streams feed into the Evaluation stage, where metrics are computed and results are visualized.
Architectural Patterns & Design Principles
• Modularity: Emphasize clear separation between collection, labeling, splitting, detection, recognition, and evaluation.
• Reusability: Note the use of external tools (Automa, PPOCRLabel, CLEval) and reusing proven design patterns from PaddleOCR.
• Pipeline-based Design: Data flows sequentially as raw data → preprocessing → separate detection and recognition streams → evaluation.
• Adaptability: Different text recognition model notebooks indicate experimentation with multiple architectures and skip connection techniques.
Technical Stack & Technologies
• Python scripts and Jupyter notebooks form the processing backbone.
• Tools like Automa for data collection and PPOCRLabel for annotation.
• Deep Learning frameworks (likely PaddlePaddle given the references to PaddleOCR, as well as potential usage of TensorFlow/PyTorch in notebooks) for training text detection and recognition models.
• Evaluation through CLEval, with support from custom metric calculators and loss functions.
• Supporting assets (e.g., thesis_en.pdf, slide_vi.pptx) could be noted as reference documentation to the design.