# dali92002/docentr architecture

> Architecture diagram of the GitHub repository dali92002/docentr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dali92002/docentr
- Repository: https://github.com/dali92002/docentr

## Overview

##### Project type and purpose
- DocEnTR is a Python-based, open-source machine-learning tool that implements an end-to-end document image enhancement transformer (DocEnTr) on top of the vit-pytorch library.
- Its primary purpose is to ingest degraded document images, preprocess them into patches, train/ﬁne-tune a Vision Transformer model for binarization, and produce enhanced (binarized) outputs.

##### File structure
- Top level:
- config.py, cog.yaml — configuration and inference spec
- load_data.py, process_dibco.py — prepare/split datasets (DIBCO, H-DIBCO, PALM)
- train.py — training pipeline, logging, saving weights, visualizations
- test.py / predict.py — model evaluation and inference scripts
- models/ — model definitions (binae.py defines the DocEnTr architecture)
- utils.py — helper functions (e.g., data loaders, transforms)
- demo/ + demo.ipynb — interactive notebook for single-image inference
- git_images/ — sample input/output illustrations
- There are no separate “frontend” or “backend” services—this is a monolithic ML codebase.

##### README for additional insights
- The README describes data preparation, training, testing, demo usage, and links to pretrained model weights.
- It highlights usage patterns (CLI args), expected directory layouts, and output artifacts (visualizations, weights folder).

##### Main components & interactions (
a. Data Storage
- Raw Datasets (DIBCOSETS on disk)
- Configuration (config.py, CLI args)
b. Data Processing
- process_dibco.py → splits images into patches, organizes train/val/test sets
- load_data.py + utils.py → PyTorch Dataset/DataLoader abstraction, transforms
c. Model Core
- models/binae.py → DocEnTr architecture (Vision Transformer backbone + decoder head)
- vit-pytorch library (external dependency)
d. Training Pipeline
- train.py → consumes DataLoader → forward/backward passes → logs metrics → saves best weights → produces “vis…” folder with sample outputs
e. Evaluation / Testing
- test.py / predict.py → loads saved weights → runs inference on test set → writes binarized images to “vis…”
f. Demo / Notebook
- demo.ipynb → sample single-image inference using pretrained weights
g. External Services
- Pretrained model zoo (Google Drive links)
- Cog.yaml defines API or containerized deployment (Replicate badge)

Interactions:
Data Storage → Data Processing → Model Core
↳ Training Pipeline → Saved Weights & Visualizations
↳ Evaluation → Binarized Outputs
Model Core + Saved Weights → Demo Notebook

##### Architectural patterns & design principles
- Clear separation of concerns: data prep vs. model definition vs. training vs. inference
- Modular code: models/ for architecture, utils/ for helpers, scripts for each major pipeline step
- CLI-driven workflows: adjustable hyperparameters without code changes
- Reproducibility: config.py + cog.yaml for environment and deployment

##### Be as detailed as possible
- Scripts as separate components if desired (process_dibco.py, train.py, test.py, demo notebook)
- Configuration injection (config.py, command-line args) feeding into each pipeline stage
- Feedback loops (e.g., hyperparameter tweaking → retraining)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Configuration
    subgraph "Configuration"
        Config["config.py"]:::config
        Cog["cog.yaml"]:::config
    end

    %% Data Storage
    subgraph "Data Storage"
        Raw["Raw Datasets (DIBCOSETS)"]:::storage
    end

    %% Data Processing
    subgraph "Data Processing"
        Proc["process_dibco.py"]:::compute
        Load["load_data.py"]:::compute
        Utils["utils.py"]:::compute
    end

    %% Model Core
    subgraph "Model Core"
        Init["models/__init__.py"]:::compute
        Binae["models/binae.py"]:::compute
        External["vit-pytorch (external)"]:::external
    end

    %% Pipelines
    subgraph "Pipelines"
        Train["train.py"]:::compute
        Test["test.py"]:::compute
        Predict["predict.py"]:::compute
    end

    %% Outputs
    subgraph "Outputs"
        Weights["Saved Weights (.pt)"]:::storage
        Vis["Visualizations (vis/)"]:::output
        BinOut["Binarized Images"]:::output
    end

    %% Demo
    subgraph "Demo"
        Notebook["demo.ipynb"]:::compute
        Assets["demo/"]:::storage
    end

    %% External Pretrained
    Pretrained["Google Drive Model Zoo"]:::external

    %% Flows
    Raw --> Proc
    Config --> Proc
    Proc --> Load
    Load --> Binae
    Utils --> Load
    Config --> Train
    Config --> Test
    Config --> Predict
    Binae --> Train
    External --> Binae
    Binae --> Test
    Binae --> Notebook
    Train --> Weights
    Train --> Vis
    Test --> BinOut
    Predict --> BinOut
    Weights --> Notebook
    Notebook --> Assets
    Pretrained --> Predict

    %% Click Events
    click Config "https://github.com/dali92002/docentr/blob/main/config.py"
    click Cog "https://github.com/dali92002/docentr/blob/main/cog.yaml"
    click Proc "https://github.com/dali92002/docentr/blob/main/process_dibco.py"
    click Load "https://github.com/dali92002/docentr/blob/main/load_data.py"
    click Utils "https://github.com/dali92002/docentr/blob/main/utils.py"
    click Init "https://github.com/dali92002/docentr/blob/main/models/__init__.py"
    click Binae "https://github.com/dali92002/docentr/blob/main/models/binae.py"
    click Train "https://github.com/dali92002/docentr/blob/main/train.py"
    click Test "https://github.com/dali92002/docentr/blob/main/test.py"
    click Predict "https://github.com/dali92002/docentr/blob/main/predict.py"
    click Notebook "https://github.com/dali92002/docentr/blob/main/demo.ipynb"
    click Assets "https://github.com/dali92002/docentr/tree/main/demo/"
    click Pretrained "https://github.com/dali92002/docentr/tree/main/git_images/"

    %% Styles
    classDef storage fill:#ccf,stroke:#333,stroke-width:1px
    classDef compute fill:#cfc,stroke:#333,stroke-width:1px
    classDef output fill:#fcf,stroke:#333,stroke-width:1px
    classDef config fill:#eee,stroke:#333,stroke-width:1px
    classDef external fill:#fcc,stroke:#333,stroke-width:1px
```
