# vivek1918/document_verification_system architecture

> Architecture diagram of the GitHub repository vivek1918/document_verification_system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vivek1918/document_verification_system
- Repository: https://github.com/vivek1918/document_verification_system

## Overview

##### Project type and purpose
- This is a specialized, production-grade document verification pipeline (an open-source tool + microservice) designed for KYC.
- It ingests scanned documents, runs OCR, extracts structured fields, normalizes them, applies cross-document rules, logs metrics, and exposes functionality via a REST API and CLI.

##### File structure
- Top-level:
- api/ (FastAPI endpoints)
- run_pipeline.py & evaluate.py (CLI entrypoints)
- verifier/ (core pipeline modules)
- tests/ (unit tests)
- ocr_outputs/ (sample outputs for Mistral & enhanced)
- config.yml & requirements.txt (configuration/dependencies)
- verifier/ is subdivided into:
- ocr/ (Tesseract, EasyOCR, Mistral, preproc)
- extract/ (regex, LLM, Gemini, GROQ)
- normalize/ (cleaners, normalizers)
- verify/ (cross-document rules)
- io/ (storage abstraction)
- utils/ (logging)

##### README for additional insights
- Dual OCR engine strategy (Tesseract primary, EasyOCR fallback)
- Optional LLM integration (Gemini API or HF models)
- Seven verification rules across 3 doc types
- GPU support, comprehensive logging/metrics
- FastAPI + CLI interfaces

##### Essential architectural elements
- Cover every pipeline stage as a separate component
- Extensibility: add new extractor by plugging into the Extractor layer
- Config.yml at the center or top to indicate centralized configuration

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration
    config["config.yml"]:::config

    %% Interfaces Layer
    subgraph "Interfaces"
        CLI["CLI: run_pipeline.py"]:::iface
        EvalCLI["Evaluate CLI: evaluate.py"]:::iface
        API["API: FastAPI (api/main.py)"]:::iface
    end

    %% Core Pipeline
    subgraph "Pipeline Core"
        Preproc["Preprocessor\n(verifier/ocr/preproc.py)"]:::core

        subgraph "OCR Engines (Plugin)"
            Tesseract["TesseractOCR"]:::core
            EasyOCR["EasyOCR (CUDA)"]:::core
            Mistral["MistralOCR"]:::core
            MistralE["MistralOCR Enhanced"]:::core
        end

        subgraph "Extractors"
            Regex["Regex"]:::core
            LLM["LLM"]:::core
            Gemini["Gemini API"]:::core
            GROQ["GROQ"]:::core
        end

        subgraph "Normalizers"
            Cleaners["Cleaners"]:::core
            Normalizers["Normalizers"]:::core
        end

        Verifier["Verifier\n(verifier/verify/rules.py)"]:::core
    end

    %% Storage & Logging
    Storage["Storage/IO"]:::storage
    Logging["Metrics & Logging"]:::storage

    %% Data Flows
    CLI -->|"trigger"| Preproc
    EvalCLI -->|"reads results"| Storage
    API -->|"HTTP trigger"| Preproc

    config -.-> Preproc
    config -.-> CLI
    config -.-> EvalCLI
    config -.-> API

    Preproc -->|"clean image"| Tesseract
    Tesseract -->|"text output"| Regex
    Tesseract -.->|failure fallback| EasyOCR
    EasyOCR -->|"text output"| Regex
    Mistral -->|"text output"| Regex
    MistralE -->|"text output"| Regex

    Regex -->|"entities"| Cleaners
    LLM -->|"entities"| Cleaners
    Gemini -->|"entities"| Cleaners
    GROQ -->|"entities"| Cleaners

    Cleaners -->|"cleaned fields"| Normalizers
    Normalizers -->|"normalized data"| Verifier

    Verifier -->|"results"| Storage
    Verifier -->|"metrics"| Logging

    %% Click Events
    click CLI "https://github.com/vivek1918/document_verification_system/blob/main/run_pipeline.py"
    click EvalCLI "https://github.com/vivek1918/document_verification_system/blob/main/evaluate.py"
    click API "https://github.com/vivek1918/document_verification_system/blob/main/api/main.py"
    click config "https://github.com/vivek1918/document_verification_system/blob/main/config.yml"
    click Preproc "https://github.com/vivek1918/document_verification_system/blob/main/verifier/ocr/preproc.py"
    click Tesseract "https://github.com/vivek1918/document_verification_system/blob/main/verifier/ocr/tesseract_ocr.py"
    click EasyOCR "https://github.com/vivek1918/document_verification_system/blob/main/verifier/ocr/easyocr_ocr.py"
    click Mistral "https://github.com/vivek1918/document_verification_system/blob/main/verifier/ocr/mistral_ocr.py"
    click MistralE "https://github.com/vivek1918/document_verification_system/blob/main/verifier/ocr/mistral_ocr_enhanced.py"
    click Regex "https://github.com/vivek1918/document_verification_system/blob/main/verifier/extract/regex_extractors.py"
    click LLM "https://github.com/vivek1918/document_verification_system/blob/main/verifier/extract/llm_extractors.py"
    click Gemini "https://github.com/vivek1918/document_verification_system/blob/main/verifier/extract/gemini_extractors.py"
    click GROQ "https://github.com/vivek1918/document_verification_system/blob/main/verifier/extract/groq_extractors.py"
    click Cleaners "https://github.com/vivek1918/document_verification_system/blob/main/verifier/normalize/cleaners.py"
    click Normalizers "https://github.com/vivek1918/document_verification_system/blob/main/verifier/normalize/normalizers.py"
    click Verifier "https://github.com/vivek1918/document_verification_system/blob/main/verifier/verify/rules.py"
    click Storage "https://github.com/vivek1918/document_verification_system/blob/main/verifier/io/storage.py"
    click Logging "https://github.com/vivek1918/document_verification_system/blob/main/verifier/utils/logger.py"

    %% Styles
    classDef iface fill:#D0E6FF,stroke:#0066CC,color:#000000,stroke-width:1px;
    classDef core fill:#DFF7DF,stroke:#33AA33,color:#000000,stroke-width:1px;
    classDef storage fill:#FFE9CC,stroke:#FF9900,color:#000000,stroke-width:1px;
    classDef config fill:#FFF5CC,stroke:#CCCC00,color:#000000,stroke-width:1px;
```
