How nvidia/nv-ingest works
Project Type and Core Purpose
This is a microservice-based system designed for scalable document content and metadata extraction, with support for multiple document types (PDF, Word, PowerPoint) and content types (text, tables, charts, images).
Main Components
A. Client Layer:
• REST Client Interface
• CLI Client Interface
• Python Client Library
B. Core Service Layer:
• NV-Ingest Microservice
• Message Broker (Redis)
• Job Management System
C. Processing Pipeline:
• Document Extractors (PDF, DOCX, PPTX)
• Content Processors
• Text Extraction
• Table Detection
• Chart Detection
• Image Processing
• OCR Processing
D. NIM (NVIDIA Inference Microservices):
• YOLOX (Table/Chart Detection)
• PaddleOCR (Text Extraction)
• Deplot (Chart Processing)
• CACHED (Chart Processing)
• Embedding Service
E. Storage & Monitoring:
• Milvus Vector Database
• Prometheus/Grafana Monitoring
• OpenTelemetry Collector
• Zipkin Tracing
[Client Layer]
• ↓
[API/Service Layer]
• ↓
[Processing Pipeline]
• ↓
[NIM Services]
• ↓
[Storage/Monitoring]
Connection Types
• Double lines: High-bandwidth data transfer
Key Architectural Patterns to Highlight
a) Microservices Architecture:
• Clear service boundaries
• Independent scaling capabilities
• Service discovery mechanisms
b) Pipeline Processing:
• Sequential processing stages
• Parallel processing capabilities
• Data transformation points
c) Monitoring & Observability:
• Integration points with telemetry services
• Metrics collection paths
• Tracing connectivity
Additional Elements
• Component types
• Connection types
b) Annotations for:
• Scaling points
• GPU requirements
• Memory considerations
• Network requirements
Implementation Details
• REST API endpoints (port 7670)
• Redis message queue integration
• GPU allocation for different NIMs
• Document processing workflows
• Data persistence paths