How belval/textrecognitiondatagenerator works
Project Type & Purpose
• This is a Python-based CLI tool *and* library whose sole purpose is to generate synthetic text images (Latin, non-Latin, handwritten) for OCR training. There is no “frontend” or “backend” in the web-app sense—rather, you have a CLI entrypoint plus a modular Python API.
File Structure Analysis
• Root level
• CI/CD: .circleci/, .github/workflows/, codecov.yml
• Packaging: setup.py, setup.cfg, MANIFEST.in, LICENSE, Dockerfile
• Tests: tests.py, tests/, tests/expected_results/
• Samples: samples/ (example images)
• trdg package
• Entry: run.py
• Core modules: background_generator.py, computer_text_generator.py, data_generator.py, distorsion_generator.py, string_generator.py, utils.py
• Generators: trdg/generators/{from_dict, from_random, from_strings, from_wikipedia}.py
• Resources:
– dicts/*.txt (word lists by language)
– fonts/{latin, cn, ja, ko, th}/
– images/ (background textures)
– handwritten_model/ (TensorFlow model files & translation.pkl)
Key README Insights
• Four generation strategies (dict, random, strings, wikipedia) all share the same transformation pipeline
• Optional handwritten pipeline powered by TensorFlow (if –hw flag is used)
• Uses Pillow for rendering, applies skew/blur/distortion, supports custom backgrounds and fonts
• Can be used as a CLI (trdg …) or as a Python module
a. Main Components
1. CLI Entrypoint
– trdg/run.py (parses args, dispatches to chosen Generator)
2. Python API
– GeneratorFromDict, GeneratorFromRandom, GeneratorFromStrings, GeneratorFromWikipedia
3. Resource Loaders
– Dictionary Loader (trdg/dicts/*.txt)
– Fonts Loader (trdg/fonts/*)
– Background Images Loader (trdg/images/)
4. Core Pipeline
– Text Generator (computer_text_generator.py or handwritten_text_generator.py)
– Background Generator (background_generator.py)
– Distortion Module (distorsion_generator.py)
– Blur/Skew Module (part of data_generator.py)
– Composite & Output Writer (saves image + label; supports masks)
5. Handwritten Pipeline (optional)
– TensorFlow Model (handwritten_model/)
6. Packaging & Tooling
– Docker Container (Dockerfile)
– CI/CD (CircleCI, GitHub Actions)
– Tests Suite (tests/)
b. Relationships & Data Flow
• CLI/API → Resource Loaders → Text Generator → Background Generator → Distortion → Blur/Skew → Composite → Output
• Handwritten branch: CLI/API + TensorFlow Model → same downstream pipeline
• Packaging/CI pipelines sit orthogonal to core data flow
c. Architectural Patterns & Principles
– Strategy Pattern: interchangeable “Generator” classes
– Pipeline Pattern: sequential transformation stages
– Resource-Driven: fonts, dicts, images are external pluggable assets
– Modular Test Harness: test files mirror expected outputs
d. Key Technologies & Libraries
– Python 3.x, Pillow (PIL), NumPy
– TensorFlow (only for handwritten)
– click/argparse (CLI argument parsing)
– Docker for containerization
– pytest/unittest for tests