# mwasifanwar/documind-ai architecture

> Architecture diagram of the GitHub repository mwasifanwar/documind-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mwasifanwar/documind-ai
- Repository: https://github.com/mwasifanwar/documind-ai

## Overview

##### Project type and purpose
- DocuMind-AI is a backend microservice that exposes a REST API for advanced OCR and document understanding.
- It ingests PDF/image documents, runs a multi-stage pipeline (preprocessing → OCR → layout analysis → classification → entity/table extraction → postprocessing), and returns structured JSON.

##### File structure
– api/
- endpoints.py → HTTP routes
- schemas.py   → request/response models
– core/
- ocr_engine.py        → text extraction
- layout_analyzer.py   → region detection & layout features
- document_classifier.py → routes documents through proper models
– models/
- transformer_model.py → NLU tasks (e.g., key-value extraction)
- entity_extractor.py  → named-entity recognition
- table_detector.py    → detect tabular regions
– processing/
- preprocessor.py    → image/PDF normalization
- table_processor.py → refine raw table data
- postprocessor.py   → merge all outputs into final payload
– utils/
- config.py         → central configuration loader
- file_handlers.py  → I/O abstraction (S3, local FS)
- visualization.py  → debugging & annotation visuals
– scripts/
- train.py       → model training orchestration
- evaluate.py    → model evaluation
- process_document.py → CLI wrapper for batch processing

##### README
- Confirms purpose: “Advanced OCR and document understanding”
- Highlights transformer-based models, layout understanding, table & entity extraction

##### Main components & their relationships
a. API Layer (api/)
b. Pipeline Orchestrator (core/)
c. Pre/Post-Processing (processing/)
d. Model Inference Units (models/)
e. Utilities & Config (utils/)
f. Training/Evaluation Subsystem (scripts/)
g. External Services: Model Storage, Document Store, Logging/Monitoring

Flow:
1) Client → API Endpoint (via HTTP)
2) API → Preprocessor (normalizes, converts to image tensors)
3) Preprocessor → OCR Engine → raw text + coordinates
4) OCR Output → Layout Analyzer & Document Classifier
5) Classifier routes to appropriate Model(s) (entity_extractor, table_detector, transformer_model)
6) Table Detector → Table Processor
7) All model outputs → Postprocessor (merges & formats)
8) API → returns structured JSON response

##### Architectural patterns & design principles
- Layered Architecture: clear separation between API, orchestration, inference, and utilities
- Modular Design: each model and processing step is replaceable/interchangeable
- Config-Driven: central utils/config.py for environment and resource settings
- Single-Responsibility: every module has a narrow focus

- Annotations for data formats at each handoff.
- A side sidebar listing external dependencies and where they integrate.

This approach ensures every essential architectural element of DocuMind-AI is captured, labeled, and visually distinct, giving any reader immediate clarity on how the system is structured and how data flows through it.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client
    subgraph "Client"
        C[/"Client"/]:::external
    end

    %% API Layer
    subgraph "API Layer"
        EP["Endpoints"]:::control
        SC["Schemas"]:::control
    end
    C -->|"HTTP Request"| EP
    EP -->|"validate request"| SC
    SC -->|"raw bytes"| PRE

    %% Pre-Processing
    PRE["Preprocessor"]:::processing
    click PRE "https://github.com/mwasifanwar/documind-ai/blob/main/processing/preprocessor.py"
    SC -->|"raw bytes"| PRE

    %% OCR & Core Pipeline
    OCR["OCR Engine"]:::control
    click OCR "https://github.com/mwasifanwar/documind-ai/blob/main/core/ocr_engine.py"
    PRE -->|"image tensor"| OCR

    LA["Layout Analyzer"]:::control
    click LA "https://github.com/mwasifanwar/documind-ai/blob/main/core/layout_analyzer.py"
    OCR -->|"text+layout JSON"| LA

    DC["Document Classifier"]:::control
    click DC "https://github.com/mwasifanwar/documind-ai/blob/main/core/document_classifier.py"
    OCR -->|"text+layout JSON"| DC

    %% Model Inference Units
    EE["Entity Extractor"]:::processing
    click EE "https://github.com/mwasifanwar/documind-ai/blob/main/models/entity_extractor.py"
    DC -->|"route to entity extractor"| EE

    TD["Table Detector"]:::processing
    click TD "https://github.com/mwasifanwar/documind-ai/blob/main/models/table_detector.py"
    DC -->|"route to table detector"| TD

    TM["Transformer Model"]:::processing
    click TM "https://github.com/mwasifanwar/documind-ai/blob/main/models/transformer_model.py"
    DC -->|"route to transformer model"| TM

    %% Table Processing
    TP["Table Processor"]:::processing
    click TP "https://github.com/mwasifanwar/documind-ai/blob/main/processing/table_processor.py"
    TD -->|"raw tables"| TP

    %% Post-Processing
    PP["Postprocessor"]:::processing
    click PP "https://github.com/mwasifanwar/documind-ai/blob/main/processing/postprocessor.py"
    EE -->|"entities JSON"| PP
    TM -->|"entities JSON"| PP
    TP -->|"tables JSON"| PP

    %% API Response
    RESP["Structured JSON"]:::data
    PP -->|"structured JSON"| RESP
    RESP -->|"HTTP Response"| C

    %% External Systems
    MS[(Model Store)]:::external
    DS[(Document Store)]:::external
    LOG[(Logging/Monitoring)]:::external

    EP -->|"store raw document"| DS
    PP -->|"store processed output"| DS
    EP -->|"log request"| LOG
    PP -->|"log response"| LOG

    %% Scripts
    subgraph "Scripts"
        T["train.py"]:::control
        E["evaluate.py"]:::control
        PD["process_document.py"]:::control
    end
    click T "https://github.com/mwasifanwar/documind-ai/blob/main/scripts/train.py"
    click E "https://github.com/mwasifanwar/documind-ai/blob/main/scripts/evaluate.py"
    click PD "https://github.com/mwasifanwar/documind-ai/blob/main/scripts/process_document.py"
    T -->|"save model"| MS
    E -->|"save evaluation"| MS
    PD -->|"batch process"| EP

    %% Utilities
    subgraph "Utilities"
        CFG["config.py"]:::utils
        FH["file_handlers.py"]:::utils
        VIS["visualization.py"]:::utils
    end
    click CFG "https://github.com/mwasifanwar/documind-ai/blob/main/utils/config.py"
    click FH "https://github.com/mwasifanwar/documind-ai/blob/main/utils/file_handlers.py"
    click VIS "https://github.com/mwasifanwar/documind-ai/blob/main/utils/visualization.py"
    CFG --> EP
    CFG --> OCR
    CFG --> PP
    FH --> PRE
    FH --> PP
    VIS --> LA

    %% Data artifacts
    classDef data fill:#F5F5DC,stroke:#333,stroke-width:1px;
    classDef control fill:#ADD8E6,stroke:#333,stroke-width:1px;
    classDef processing fill:#90EE90,stroke:#333,stroke-width:1px;
    classDef utils fill:#D3D3D3,stroke:#333,stroke-width:1px;
    classDef external fill:#FFA500,stroke:#333,stroke-width:1px;
```
