How dmdaksh/docai works
Project Type & Purpose
• The project is a document processing system designed for fine-tuning a pretrained LayoutLMv3 transformer-based model using custom datasets.
Major Components & Their Roles
• CLI Interface & Configuration:
• The project is configured and executed via a command line interface (Typer-based), as seen in the training (docai/training/main.py) and inference (docai/inference/inference.py) modules.
• Data Generation & Annotation:
• The "data_gen" directory contains scripts (e.g., layoutlmv3_datagen.py) that handle transforming raw OCR outputs and JSON annotations into training-ready examples.
• Data Loading:
• The "datasets/loader.py" is responsible for converting JSON-based annotation files into a format digestible by the training process.
• Model Definition:
• The "models/lmv3model.py" file encapsulates the model setup and any modifications required for fine-tuning the pretrained LayoutLMv3 network.
• Training Pipeline:
• The "training" folder (with engine.py and main.py) runs the training loop, fine-tunes the model, handles evaluation, checkpointing, and saving of the best model.
• Inference Pipeline:
• Utility Functions & Configurations:
• Utility functions (in docai/utils) support shared tasks—such as GCS acquisition and data formatting (e.g., converting JSON training data).
• The configuration file (docai/config.py) likely centralizes settings and parameters used across modules.
Relationships & Interactions
• A user (or an automated process) starts by running CLI commands that trigger either training or inference.
• For training:
o The process begins with loading and preprocessing data (using the custom data loader and data generation scripts).
o The training engine (in docai/training/engine.py) leverages the processed dataset and the model (from docai/models/lmv3model.py) for fine-tuning.
o The pipeline continuously saves checkpoints and outputs the best-performing model based on training metrics.
• For inference:
o The pipeline loads the saved model and applies it to new document images.
o OCR (via pytesseract, referenced in project description) might be applied upstream to initially generate text and spatial data if needed.
Architectural Patterns & Design Principles
• Modular Design: The project is segmented into distinct modules—data generation, loading, training, inference, and utilities—facilitating extensibility and easier maintenance.
• Separation of Concerns: Each submodule handles a specific aspect of the document processing pipeline.
• Command Line Driven: Emphasis on reproducibility and configurability through a CLI (Typer) and configuration files.
Relevant Technologies/Frameworks
• LayoutLMv3 (Transformer-based model for document understanding)
• pytesseract for OCR operations.
• Typer for CLI management.
• Likely the use of PyTorch (or a similar deep learning framework) for model training and fine-tuning.
• Standard Python packaging tools (pyproject.toml, setup.cfg) indicate good packaging practices.