How khanasif1/aoai-observability-demo works
Project type and purpose
• This is a full-stack observability demo solution for Azure OpenAI services.
• Purpose: track per-user consumption, compute billing, and surface both real-time and historical costs.
Main components
a. Azure API Management (APIM)
• Entry point for all client requests
• Handles user authentication via managed identities and routes to downstream services
b. Azure Functions
• azFn-AoAiModel-Pricing: scrapes Azure OpenAI pricing, selects model/version price, returns JSON
• azFn-AppInsightAPI: reads usage & cost from Application Insights, exposes REST API
c. OpenTelemetry Console App
• Instrumentation for distributed tracing, metrics, and logging
• Correlates APIM calls with function executions, pushes telemetry to Application Insights
d. Application Insights
• Central store for traces, metrics, and logs emitted by functions and OpenTelemetry
e. Streamlit UI
• Front-end dashboard
• Calls azFn-AppInsightAPI to fetch cost/usage data, displays interactive charts & tables
f. External dependencies
• Azure OpenAI resource (model endpoints)
• Pricing webpage (scraped by azFn-AoAiModel-Pricing)
Relationships & data flows
1. Client → APIM: request model inference or cost lookup
2. APIM → Managed Identity auth → Azure Functions
3. azFn-AoAiModel-Pricing fetches pricing → returns JSON → logs usage/cost to Application Insights
4. OpenTelemetry console app subscribes/traces the invocation paths (APIM → Functions) → sends telemetry
5. azFn-AppInsightAPI queries Application Insights for aggregated usage/cost data → returns to callers
6. Streamlit UI → invokes azFn-AppInsightAPI → renders user-specific cost analytics
Architectural patterns & design principles
• Micro-services (each Azure Function is a lightweight service)
• Serverless compute for scale and cost efficiency
• Observability-first: distributed tracing + centralized logging/metrics
• Clear separation of concerns: pricing scraper, telemetry collector, data API, visualization
Key technologies & frameworks
• Azure API Management
• Azure Functions (Python)
• Python OpenTelemetry SDK
• Application Insights
• Streamlit (Python UI)
• Scrapy or simple HTTP client for pricing page