# nvidia/nv-ingest architecture

> Architecture diagram of the GitHub repository nvidia/nv-ingest, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia/nv-ingest
- Repository: https://github.com/nvidia/nv-ingest

## Overview

##### Project Type and Core Purpose
This is a microservice-based system designed for scalable document content and metadata extraction, with support for multiple document types (PDF, Word, PowerPoint) and content types (text, tables, charts, images).

##### Main Components

A. Client Layer:
- REST Client Interface
- CLI Client Interface
- Python Client Library

B. Core Service Layer:
- NV-Ingest Microservice
- Message Broker (Redis)
- Job Management System

C. Processing Pipeline:
- Document Extractors (PDF, DOCX, PPTX)
- Content Processors
- Text Extraction
- Table Detection
- Chart Detection
- Image Processing
- OCR Processing

D. NIM (NVIDIA Inference Microservices):
- YOLOX (Table/Chart Detection)
- PaddleOCR (Text Extraction)
- Deplot (Chart Processing)
- CACHED (Chart Processing)
- Embedding Service

E. Storage & Monitoring:
- Milvus Vector Database
- Prometheus/Grafana Monitoring
- OpenTelemetry Collector
- Zipkin Tracing

**[Client Layer]**
- ↓
**[API/Service Layer]**
- ↓
**[Processing Pipeline]**
- ↓
**[NIM Services]**
- ↓
**[Storage/Monitoring]**

##### Connection Types

- Double lines: High-bandwidth data transfer

##### Key Architectural Patterns to Highlight

a) Microservices Architecture:
- Clear service boundaries
- Independent scaling capabilities
- Service discovery mechanisms

b) Pipeline Processing:
- Sequential processing stages
- Parallel processing capabilities
- Data transformation points

c) Monitoring & Observability:
- Integration points with telemetry services
- Metrics collection paths
- Tracing connectivity

##### Additional Elements

- Component types
- Connection types

b) Annotations for:
- Scaling points
- GPU requirements
- Memory considerations
- Network requirements

##### Implementation Details

- REST API endpoints (port 7670)
- Redis message queue integration
- GPU allocation for different NIMs
- Document processing workflows
- Data persistence paths

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles and Classes
    classDef client fill:#3498db,stroke:#2980b9,color:white
    classDef core fill:#2ecc71,stroke:#27ae60,color:white
    classDef processor fill:#f1c40f,stroke:#f39c12,color:black
    classDef nim fill:#e74c3c,stroke:#c0392b,color:white
    classDef storage fill:#9b59b6,stroke:#8e44ad,color:white
    classDef broker fill:#1abc9c,stroke:#16a085,color:white

    %% Client Layer
    subgraph Clients
        REST["REST Client"]:::client
        CLI["CLI Interface"]:::client
        PythonLib["Python Client Library"]:::client
    end

    %% Core Service Layer
    subgraph CoreService
        NVIngest["NV-Ingest Microservice"]:::core
        Redis["Redis Message Broker"]:::broker
        JobMgmt["Job Management System"]:::core
        APILayer["API Layer"]:::core
    end

    %% Processing Pipeline
    subgraph Pipeline
        direction TB
        subgraph Extractors
            PDFExt["PDF Extractor"]:::processor
            DOCXExt["DOCX Extractor"]:::processor
            PPTXExt["PPTX Handler"]:::processor
        end
        
        subgraph Processors
            TableDet["Table Detection"]:::processor
            ChartProc["Chart Processing"]:::processor
            OCRProc["OCR Processing"]:::processor
            ImgProc["Image Processing"]:::processor
        end
    end

    %% NIM Services
    subgraph NIMs
        YOLOX["YOLOX"]:::nim
        PaddleOCR["PaddleOCR"]:::nim
        Deplot["Deplot"]:::nim
        CACHED["CACHED"]:::nim
        Embedding["Embedding Service"]:::nim
    end

    %% Storage & Monitoring
    subgraph Observability
        Milvus["Milvus Vector DB"]:::storage
        Prometheus["Prometheus"]:::storage
        OpenTel["OpenTelemetry"]:::storage
        Zipkin["Zipkin"]:::storage
    end

    %% Connections
    REST & CLI & PythonLib --> APILayer
    APILayer --> NVIngest
    NVIngest <--> Redis
    Redis <--> JobMgmt
    JobMgmt --> Extractors
    Extractors --> Processors
    Processors --> NIMs
    NIMs --> Milvus
    NVIngest -.-> OpenTel
    OpenTel -.-> Prometheus & Zipkin

    %% Click Events
    click REST "https://github.com/NVIDIA/nv-ingest/blob/main/client/src/nv_ingest_client/message_clients/rest/rest_client.py"
    click CLI "https://github.com/NVIDIA/nv-ingest/blob/main/client/src/nv_ingest_client/nv_ingest_cli.py"
    click PythonLib "https://github.com/NVIDIA/nv-ingest/blob/main/client/src/nv_ingest_client/client/client.py"
    click NVIngest "https://github.com/NVIDIA/nv-ingest/blob/main/src/microservice_entrypoint.py"
    click APILayer "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/api/v1/ingest.py"
    click Redis "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/util/message_brokers/redis/redis_client.py"
    click JobMgmt "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/schemas/ingest_job_schema.py"
    click PDFExt "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/modules/extractors/pdf_extractor.py"
    click DOCXExt "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/modules/extractors/docx_extractor.py"
    click PPTXExt "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/extraction_workflows/pptx/pptx_helper.py"
    click TableDet "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/util/nim/yolox.py"
    click ChartProc "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/stages/nim/chart_extraction.py"
    click OCRProc "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/util/nim/paddle.py"
    click ImgProc "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/util/image_processing/transforms.py"
    click Milvus "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/modules/sinks/vdb_task_sink.py"
    click OpenTel "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/modules/telemetry/otel_meter.py"
    click Zipkin "https://github.com/NVIDIA/nv-ingest/blob/main/src/nv_ingest/modules/telemetry/otel_tracer.py"
    click Prometheus "https://github.com/NVIDIA/nv-ingest/blob/main/config/prometheus.yaml"
```
