# mandark-droid/genai_otel_instrument architecture

> Architecture diagram of the GitHub repository mandark-droid/genai_otel_instrument, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mandark-droid/genai_otel_instrument
- Repository: https://github.com/mandark-droid/genai_otel_instrument

## Overview

##### Project Type and Purpose
- This is an open-source Python library (genai-otel-instrument) that provides zero-code and programmatic OpenTelemetry instrumentation for GenAI/LLM applications and related services (databases, caches, message queues, vector DBs).
- It auto-instruments 15+ LLM providers (OpenAI, Anthropic, HuggingFace, etc.) and MCP (“Model Context Protocol”) tools (Postgres, Redis, Kafka, Pinecone, etc.), collects traces & metrics (including cost and GPU usage), enriches spans with cost attributes, and exports them via OTLP to any OpenTelemetry-compatible backend.

##### Key Architectural Elements from File Structure
- genai_otel/auto_instrument.py & cli.py – entry points for instrumentation (API or CLI wrapper).
- genai_otel/config.py – holds configuration logic (env vars, programmatic).
- genai_otel/instrumentors/*.py – plugin modules that hook into each LLM provider SDK (monkey-patch or decorator pattern).
- genai_otel/mcp_instrumentors/*.py – similar plugin modules for MCP tools (DBs, caches, APIs, vector DBs).
- cost_calculator.py, cost_enrichment_processor.py – logic to compute and enrich spans with cost metrics.
- gpu_metrics.py – collects GPU utilization & power metrics periodically.
- logging_config.py, metrics.py – auxiliary modules.
- example_usage.py & examples/demo – a reference app + Docker-Compose for demo telemetry stack (OTEL Collector, Prometheus, Grafana, OpenSearch).

##### Information from README
- Zero-code setup via environment variables or one-line API call or CLI wrapper.
- Deep lists of supported providers & MCP tools.
- Exposes spans & metrics with rich attributes per OpenTelemetry semantic conventions (GenAI).
- Configurable exporters, timeouts, semantic conventions, session/user extraction, RAG pipeline helpers.

a. Main Components
- User Application (any Python app using LLMs, DBs, caches, vector DBs)
- genai-otel Instrumentation Library
– Entry Point: genai_otel.instrument() or genai-instrument CLI
– Config Manager (reads env vars / programmatic)
– Instrumentor Manager (discovers & registers instrumentors)
– Instrumentors
· LLM Instrumentors (openai, anthropic, huggingface, etc.)
· MCP Instrumentors (Postgres, Redis, Kafka, Pinecone, etc.)
– Cost Calculator & Enrichment Processor
– GPU Metrics Collector
- OpenTelemetry SDK Pipeline
– Span & Metric Processors (CostEnrichmentSpanProcessor, BatchSpanProcessor)
– OTLP Exporter
- External Dependencies
– LLM Provider APIs (OpenAI, Anthropic, Google AI, etc.)
– MCP Services (Postgres, Redis, Kafka, Vector DBs)
- Observability Backends
– OTEL Collector / Direct OTLP Endpoint
– Tracing Backends (Jaeger, Zipkin, Elastic APM, etc.)
– Metrics Backends (Prometheus + Grafana, OpenSearch + Kibana, Datadog, etc.)

b. Relationships & Data Flows
1. User App calls LLM/DB client → Instrumentor intercepts and starts a span (attributes: model, tokens, SQL, etc.)
2. Instrumentor calls original SDK function → receives response/result
3. Instrumentor captures response attributes (latency, token counts), computes cost via Cost Calculator
4. Span & metrics (counters, histograms, gauges) emitted to OpenTelemetry SDK
5. SpanProcessor enriches spans with cost and GPU attributes, batches them
6. OTLP Exporter transmits telemetry to configured endpoint
7. Telemetry backend stores & visualizes traces/metrics (e.g., Grafana dashboards in examples/demo)

c. Architectural Patterns & Principles
- Plugin/Instrumentor Pattern: each provider or service has its own module implementing a standard interface (monkey-patch or decorator).
- OpenTelemetry Pipeline: separation of concern between instrumentation, processing, and exporting.
- Zero-Code vs. Code-Based API: entry points enable both environment-driven and API-driven configurations.
- Config-Driven Behavior: uses environment variables and a unified config class for feature toggles (GPU, cost tracking, MCP).

d. Key Technologies & Frameworks
- Python 3.9+
- OpenTelemetry SDK (tracing & metrics)
- OTLP exporter (HTTP/Protobuf or gRPC)
- LLM SDKs (openai, anthropic, cohere, etc.)
- DB/Cache clients (psycopg2, redis-py, kafka-python, pinecone, etc.)
- Prometheus, Grafana, OpenSearch for demos

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Application
    subgraph "User Application"
        App["User Application"]
    end

    %% genai-otel Library
    subgraph "genai-otel Library" 
        direction TB
        subgraph "Entry Points"
            API["Auto-Instrument API"]:::lib
            CLI["CLI Wrapper"]:::lib
        end
        subgraph "Config & Management"
            Config["Config Manager"]:::lib
            Manager["Instrumentor Manager"]:::lib
        end
        subgraph "Instrumentors"
            direction TB
            subgraph "LLM Instrumentors"
                baseLLM["Base Instrumentor"]:::lib
                AAnthropic["AnthropicInstrumentor"]:::lib
                AAnyScale["AnyscaleInstrumentor"]:::lib
                AWSB["AWS BedrockInstrumentor"]:::lib
                AZOpenAI["Azure OpenAIInstrumentor"]:::lib
                Cohere["CohereInstrumentor"]:::lib
                GgAI["Google AIInstrumentor"]:::lib
                Groq["GroqInstrumentor"]:::lib
                HF["HuggingFaceInstrumentor"]:::lib
                LangChain["LangChainInstrumentor"]:::lib
                Llama["LlamaIndexInstrumentor"]:::lib
                Mistral["MistralAIInstrumentor"]:::lib
                Ollama["OllamaInstrumentor"]:::lib
                OpenAI["OpenAIInstrumentor"]:::lib
                Replicate["ReplicateInstrumentor"]:::lib
                Together["TogetherAIInstrumentor"]:::lib
                Vertex["VertexAIInstrumentor"]:::lib
            end
            subgraph "MCP Instrumentors"
                baseMCP["Base MCP Instrumentor"]:::lib
                APIInst["APIInstrumentor"]:::lib
                DBInst["DatabaseInstrumentor"]:::lib
                Kafka["KafkaInstrumentor"]:::lib
                Redis["RedisInstrumentor"]:::lib
                VDB["VectorDBInstrumentor"]:::lib
            end
        end
        subgraph "Processing & Metrics"
            CostCalc["Cost Calculator"]:::lib
            CostEnrich["Cost Enrichment Processor"]:::lib
            GPU["GPU Metrics Collector"]:::lib
        end
    end

    %% OpenTelemetry Pipeline
    subgraph "OpenTelemetry SDK Pipeline"
        direction TB
        SpanProc["Span Processors"]:::pipeline
        MetricsProc["Metric Processors"]:::pipeline
        OTLP["OTLP Exporter"]:::pipeline
    end

    %% External Dependencies
    subgraph "External Dependencies"
        direction TB
        LLMAPIs["LLM Provider APIs"]:::external
        MCPServices["MCP Services"]:::external
    end

    %% Observability Backends
    subgraph "Observability Backends"
        direction TB
        OTELColl["OTEL Collector"]:::backend
        TraceBackends["Tracing Backends"]:::backend
        MetricsBackends["Metrics Backends"]:::backend
        Demo["Demo Stack (Docker-Compose)"]:::backend
    end

    %% Connections
    App -->|"initialize"| API
    App -->|"run CLI"| CLI
    API --> Config
    CLI --> Config
    Config --> Manager
    Manager --> baseLLM
    Manager --> baseMCP
    baseLLM --> AAnthropic
    baseLLM --> AAnyScale
    baseLLM --> AWSB
    baseLLM --> AZOpenAI
    baseLLM --> Cohere
    baseLLM --> GgAI
    baseLLM --> Groq
    baseLLM --> HF
    baseLLM --> LangChain
    baseLLM --> Llama
    baseLLM --> Mistral
    baseLLM --> Ollama
    baseLLM --> OpenAI
    baseLLM --> Replicate
    baseLLM --> Together
    baseLLM --> Vertex
    baseMCP --> APIInst
    baseMCP --> DBInst
    baseMCP --> Kafka
    baseMCP --> Redis
    baseMCP --> VDB
    App -->|calls SDK| AAnthropic
    App -->|calls SDK| APIInst
    AAnthropic -->|"span start"| SpanProc
    APIInst -->|"span start"| SpanProc
    SpanProc --> CostEnrich
    SpanProc --> OTLP
    MetricsProc --> OTLP
    AAnthropic --> CostCalc
    APIInst --> CostCalc
    CostCalc --> CostEnrich
    GPU --> MetricsProc
    OTLP --> OTELColl
    OTELColl --> TraceBackends
    OTELColl --> MetricsBackends
    OTELColl --> Demo
    LLMAPIs --> AAnthropic
    LLMAPIs --> AAnyScale
    LLMAPIs --> AWSB
    LLMAPIs --> AZOpenAI
    LLMAPIs --> Cohere
    LLMAPIs --> GgAI
    LLMAPIs --> Groq
    LLMAPIs --> HF
    LLMAPIs --> LangChain
    LLMAPIs --> Llama
    LLMAPIs --> Mistral
    LLMAPIs --> Ollama
    LLMAPIs --> OpenAI
    LLMAPIs --> Replicate
    LLMAPIs --> Together
    LLMAPIs --> Vertex
    MCPServices --> APIInst
    MCPServices --> DBInst
    MCPServices --> Kafka
    MCPServices --> Redis
    MCPServices --> VDB

    %% Click Events
    click API "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/auto_instrument.py"
    click CLI "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/cli.py"
    click Config "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/config.py"
    click Manager "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/mcp_instrumentors/manager.py"
    click baseLLM "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/base.py"
    click AAnthropic "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/anthropic_instrumentor.py"
    click AAnyScale "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/anyscale_instrumentor.py"
    click AWSB "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/aws_bedrock_instrumentor.py"
    click AZOpenAI "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/azure_openai_instrumentor.py"
    click Cohere "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/cohere_instrumentor.py"
    click GgAI "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/google_ai_instrumentor.py"
    click Groq "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/groq_instrumentor.py"
    click HF "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/huggingface_instrumentor.py"
    click LangChain "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/langchain_instrumentor.py"
    click Llama "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/llamaindex_instrumentor.py"
    click Mistral "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/mistralai_instrumentor.py"
    click Ollama "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/ollama_instrumentor.py"
    click OpenAI "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/openai_instrumentor.py"
    click Replicate "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/replicate_instrumentor.py"
    click Together "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/togetherai_instrumentor.py"
    click Vertex "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/instrumentors/vertexai_instrumentor.py"
    click baseMCP "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/mcp_instrumentors/base.py"
    click APIInst "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/mcp_instrumentors/api_instrumentor.py"
    click DBInst "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/mcp_instrumentors/database_instrumentor.py"
    click Kafka "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/mcp_instrumentors/kafka_instrumentor.py"
    click Redis "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/mcp_instrumentors/redis_instrumentor.py"
    click VDB "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/mcp_instrumentors/vector_db_instrumentor.py"
    click CostCalc "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/cost_calculator.py"
    click CostEnrich "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/cost_enrichment_processor.py"
    click GPU "https://github.com/mandark-droid/genai_otel_instrument/blob/main/genai_otel/gpu_metrics.py"
    click OTELColl "https://github.com/mandark-droid/genai_otel_instrument/tree/main/examples/demo"

    %% Styles
    classDef lib fill:#e0f7e9,stroke:#2d7c49,stroke-width:1px
    classDef external fill:#e3f2fd,stroke:#1565c0,stroke-width:1px
    classDef pipeline fill:#fff3e0,stroke:#ef6c00,stroke-width:1px
    classDef backend fill:#f3e5f5,stroke:#6a1b9a,stroke-width:1px
```
