# khanasif1/aoai-observability-demo architecture

> Architecture diagram of the GitHub repository khanasif1/aoai-observability-demo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/khanasif1/aoai-observability-demo
- Repository: https://github.com/khanasif1/aoai-observability-demo

## Overview

##### Project type and purpose
- This is a full-stack observability demo solution for Azure OpenAI services.
- Purpose: track per-user consumption, compute billing, and surface both real-time and historical costs.

##### Main components
a. Azure API Management (APIM)
- Entry point for all client requests
- Handles user authentication via managed identities and routes to downstream services
b. Azure Functions
- azFn-AoAiModel-Pricing: scrapes Azure OpenAI pricing, selects model/version price, returns JSON
- azFn-AppInsightAPI: reads usage & cost from Application Insights, exposes REST API
c. OpenTelemetry Console App
- Instrumentation for distributed tracing, metrics, and logging
- Correlates APIM calls with function executions, pushes telemetry to Application Insights
d. Application Insights
- Central store for traces, metrics, and logs emitted by functions and OpenTelemetry
e. Streamlit UI
- Front-end dashboard
- Calls azFn-AppInsightAPI to fetch cost/usage data, displays interactive charts & tables
f. External dependencies
- Azure OpenAI resource (model endpoints)
- Pricing webpage (scraped by azFn-AoAiModel-Pricing)

##### Relationships & data flows
1. Client → APIM: request model inference or cost lookup
2. APIM → Managed Identity auth → Azure Functions
3. azFn-AoAiModel-Pricing fetches pricing → returns JSON → logs usage/cost to Application Insights
4. OpenTelemetry console app subscribes/traces the invocation paths (APIM → Functions) → sends telemetry
5. azFn-AppInsightAPI queries Application Insights for aggregated usage/cost data → returns to callers
6. Streamlit UI → invokes azFn-AppInsightAPI → renders user-specific cost analytics

##### Architectural patterns & design principles
- Micro-services (each Azure Function is a lightweight service)
- Serverless compute for scale and cost efficiency
- Observability-first: distributed tracing + centralized logging/metrics
- Clear separation of concerns: pricing scraper, telemetry collector, data API, visualization

##### Key technologies & frameworks
- Azure API Management
- Azure Functions (Python)
- Python OpenTelemetry SDK
- Application Insights
- Streamlit (Python UI)
- Scrapy or simple HTTP client for pricing page

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client"
        Client["Client Application"]:::external
    end

    subgraph "API Gateway"
        APIM["Azure API Management"]:::apiGateway
    end

    subgraph "Serverless Backends"
        pricingFn["azFn-AoAiModel-Pricing"]:::functions
        appInsightFn["azFn-AppInsightAPI"]:::functions
        scraper["Pricing Scraper"]:::functions
    end

    subgraph "Observability"
        otel["OpenTelemetry Console App"]:::observ
        appInsights["Application Insights"]:::observ
    end

    subgraph "UI"
        streamlit["Streamlit UI (Cost Dashboard)"]:::ui
    end

    subgraph "External Services"
        azureOpenAI["Azure OpenAI Endpoint"]:::external
        pricingPage["Azure OpenAI Pricing Page"]:::external
    end

    %% Data Flows
    Client -->|"REST call"| APIM
    APIM -->|"Managed identity auth"| pricingFn
    APIM -->|"Managed identity auth"| appInsightFn
    pricingFn -->|"HTTPS GET"| pricingPage
    pricingFn -->|"JSON Response"| APIM
    pricingFn -->|"Logs usage/cost"| appInsights
    scraper -->|"Scheduled Scrape"| pricingPage
    scraper -->|"Logs metrics"| appInsights
    appInsightFn -->|"Queries usage/cost"| appInsights
    appInsightFn -->|"JSON Response"| APIM
    APIM -->|"REST response"| Client

    APIM -->|"OTLP trace"| otel
    pricingFn -->|"OTLP trace"| otel
    appInsightFn -->|"OTLP trace"| otel
    otel -->|"Push telemetry"| appInsights

    streamlit -->|"REST call"| appInsightFn
    appInsightFn -->|"JSON data"| streamlit

    %% Click Events
    click pricingFn "https://github.com/khanasif1/aoai-observability-demo/blob/main/observability/azFn-AoAiModel-Pricing/function_app.py"
    click appInsightFn "https://github.com/khanasif1/aoai-observability-demo/blob/main/observability/azFn-AppInsightAPI/function_app.py"
    click otel "https://github.com/khanasif1/aoai-observability-demo/tree/main/observability/opentelemetry/"
    click streamlit "https://github.com/khanasif1/aoai-observability-demo/blob/main/observability/ui/costAnalysisView.py"
    click scraper "https://github.com/khanasif1/aoai-observability-demo/blob/main/observability/scrapeAoAIPricing/scrapy.py"
    click appInsights "https://github.com/khanasif1/aoai-observability-demo/blob/main/observability/ArchitectureDesign/AOAI_MicroBilling.drawio"

    %% Styles
    classDef apiGateway fill:#cce5ff,stroke:#3399ff
    classDef functions fill:#d4edda,stroke:#28a745
    classDef observ fill:#fff3cd,stroke:#ffc107
    classDef ui fill:#e2e3ff,stroke:#6f42c1
    classDef external fill:none,stroke:#999,stroke-dasharray:5 5
```
