# azure-samples/ai-hub-gateway-solution-accelerator architecture

> Architecture diagram of the GitHub repository azure-samples/ai-hub-gateway-solution-accelerator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/azure-samples/ai-hub-gateway-solution-accelerator
- Repository: https://github.com/azure-samples/ai-hub-gateway-solution-accelerator

## Overview

#### 1. Purpose and entry points
Citadel Governance Hub is an Azure solution accelerator for a centralized AI runtime governance plane. Azure API Management (APIM) fronts LLMs, AI services, MCP tools, and A2A agents with declarative security, routing, policy enforcement, and usage tracking.

Initial deployment is driven by Azure Developer CLI (`azure.yaml`) into the Bicep root deployment: `bicep/infra/main.bicep` with `bicep/infra/main.bicepparam`. CI/CD is defined in `.azdo/pipelines/azure-dev.yml`.

#### 2. Landing-zone composition
The root Bicep composition provisions the shared hub dependencies and delegates resource construction to reusable modules:
- `bicep/infra/main.bicep`
- `bicep/infra/resources.bicep`
- `bicep/infra/modules/apim/apim.bicep`

Material Azure services include APIM, Microsoft Foundry/AI Services and model deployments, API Center, Event Hub, Cosmos DB, Logic Apps, Key Vault, Managed Redis, Log Analytics, Application Insights, managed identities, VNets, private endpoints, and DNS. Networking supports new or existing VNets and optional private connectivity through `bicep/infra/modules/networking/vnet.bicep` and `bicep/infra/modules/networking/private-endpoint.bicep`.

#### 3. APIM is the runtime control boundary
Clients and agent execution planes call APIM rather than AI backends directly. APIM APIs import OpenAPI definitions and apply policy fragments for authentication, model access, routing, quotas/throttling, PII handling, content-safety integration, caching, and telemetry.

Core gateway definition paths:
- `bicep/infra/modules/apim/apim.bicep`
- `bicep/infra/modules/apim/universal-llm-api.bicep`
- `bicep/infra/modules/apim/policy-fragments.bicep`

The Universal LLM and Unified AI APIs provide normalized client-facing surfaces, while backend details remain controlled behind APIM.

#### 4. LLM supply-side onboarding and routing
Backend contracts independently onboard Azure OpenAI/Foundry and compatible external providers, then construct APIM backends and pools. Policies resolve requested models or aliases, select target pools, authorize backend calls, and enable resiliency behavior such as balancing/failover.

Key implementation paths:
- `bicep/infra/llm-backend-onboarding/main.bicep`
- `bicep/infra/llm-backend-onboarding/modules/llm-backend-pools.bicep`
- `bicep/infra/modules/apim/policies/universal-llm-api-policy-v2.xml`

This separation lets platform teams alter AI capacity and routing without changing consumer contracts.

#### 5. Consumer access and asset publication contracts
Access contracts implement the demand boundary: each use case/environment can receive APIM products, subscriptions, policies, and credential delivery via Key Vault or Foundry connections. Publish contracts separately expose MCP tools and A2A agents through protected APIM endpoints; publication does not grant consumption access.

Key paths:
- `bicep/infra/citadel-access-contracts/main.bicep`
- `bicep/infra/citadel-access-contracts/modules/apimProduct.bicep`
- `bicep/infra/citadel-publish-contracts/main.bicep`

Identity is based on Entra ID, managed identities, APIM subscriptions, JWT/OAuth policies, and Key Vault secret storage.

#### 6. Usage, observability, and analytics flow
APIM policies emit AI/LLM/MCP/A2A and governance events to Event Hub. Logic App workflows consume event streams and ingest usage records into Cosmos DB; Log Analytics and Application Insights provide platform telemetry, dashboards, and alerting. Power BI artifacts consume retained usage/cost data.

Core paths:
- `bicep/infra/modules/apim/policies/frag-llm-usage.xml`
- `src/usage-ingestion-logicapp/llm-usage-ingestion/workflow.json`
- `bicep/infra/modules/cosmos-db/cosmos-db.bicep`

#### 7. Lifecycle and operational boundaries
The root deployment is intended for initial provisioning. Subsequent gateway releases are applied in place through the APIM Gateway Upgrade module, preserving the landing-zone infrastructure. `release.json` is exposed by APIM’s version API and includes independently tracked release scopes.

Key paths:
- `bicep/infra/apim-gateway-upgrade/main.bicep`
- `bicep/infra/modules/apim/version-api.bicep`
- `release.json`

Main entry points: [`bicep/infra/main.bicep`](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/main.bicep) (Root Bicep deployment), [`bicep/infra/modules/apim/apim.bicep`](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/apim.bicep) (API Management), [`bicep/infra/modules/apim/policy-fragments.bicep`](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/policy-fragments.bicep) (Gateway policy fragments), [`bicep/infra/llm-backend-onboarding/modules/llm-backend-pools.bicep`](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/llm-backend-onboarding/modules/llm-backend-pools.bicep) (Backend pools & routing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_delivery["Delivery"]
  node_azd{{"Azure Developer CLI<br/>deployment entry<br/>[azure.yaml]"}}
  node_pipeline["Azure DevOps pipeline<br/>CI/CD<br/>[azure-dev.yml]"]
  node_upgrade["APIM Gateway Upgrade<br/>in-place gateway release<br/>[main.bicep]"]
  node_release["Release manifest<br/>release metadata<br/>[release.json]"]
end

subgraph group_landing["Landing Zone"]
  node_root["Root Bicep deployment<br/>Bicep composition<br/>[main.bicep]"]
  node_params["Deployment parameters<br/>Bicep parameters<br/>[main.bicepparam]"]
  node_resources["Shared hub resources<br/>Bicep resources<br/>[resources.bicep]"]
  node_network["Network &amp; private access<br/>network modules<br/>[vnet.bicep]"]
end

subgraph group_gateway["Runtime Governance"]
  node_apim["API Management<br/>runtime gateway<br/>[apim.bicep]"]
  node_api_surfaces["Universal LLM &amp; Unified AI APIs<br/>APIM APIs"]
  node_policies["Gateway policy fragments<br/>APIM policies"]
  node_backend_onboarding["LLM backend onboarding<br/>Bicep composition<br/>[main.bicep]"]
  node_backend_pools["Backend pools &amp; routing<br/>APIM backends"]
  node_version_api["Release version API<br/>APIM version endpoint<br/>[version-api.bicep]"]
end

subgraph group_contracts["Contracts"]
  node_access["Consumer access contracts<br/>access provisioning<br/>[main.bicep]"]
  node_products["APIM products<br/>consumer entitlement<br/>[apimProduct.bicep]"]
  node_publish["Asset publish contracts<br/>publication provisioning<br/>[main.bicep]"]
end

subgraph group_insights["Usage &amp; Observability"]
  node_eventhub["Event Hub<br/>governance event stream"]
  node_usage_policy["Usage event policy<br/>APIM policy<br/>[frag-llm-usage.xml]"]
  node_ingestion["Usage ingestion workflow<br/>Logic App workflow<br/>[workflow.json]"]
  node_cosmos[("Cosmos DB usage store<br/>usage data store<br/>[cosmos-db.bicep]")]
end

node_azd -->|"deploys"| node_root
node_params -->|"parameterizes"| node_root
node_pipeline -->|"automates deployment"| node_root
node_root -->|"provisions"| node_resources
node_root -->|"configures"| node_network
node_root -->|"provisions"| node_apim
node_apim -->|"hosts"| node_api_surfaces
node_policies -->|"governs"| node_api_surfaces
node_backend_onboarding -->|"constructs"| node_backend_pools
node_backend_pools -->|"routes backend calls"| node_api_surfaces
node_access -->|"creates"| node_products
node_products -->|"grants consumer access"| node_apim
node_publish -->|"publishes protected assets"| node_apim
node_usage_policy -->|"emits events"| node_eventhub
node_policies -->|"includes"| node_usage_policy
node_eventhub -->|"triggers"| node_ingestion
node_ingestion -->|"ingests usage"| node_cosmos
node_upgrade -->|"updates in place"| node_apim
node_version_api -->|"serves"| node_release
node_apim -->|"hosts"| node_version_api

click node_azd "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/azure.yaml"
click node_root "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/main.bicep"
click node_params "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/main.bicepparam"
click node_pipeline "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/.azdo/pipelines/azure-dev.yml"
click node_resources "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/resources.bicep"
click node_network "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/networking/vnet.bicep"
click node_apim "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/apim.bicep"
click node_policies "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/policy-fragments.bicep"
click node_backend_onboarding "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/llm-backend-onboarding/main.bicep"
click node_backend_pools "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/llm-backend-onboarding/modules/llm-backend-pools.bicep"
click node_access "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/citadel-access-contracts/main.bicep"
click node_products "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/citadel-access-contracts/modules/apimProduct.bicep"
click node_publish "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/citadel-publish-contracts/main.bicep"
click node_usage_policy "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/policies/frag-llm-usage.xml"
click node_ingestion "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/src/usage-ingestion-logicapp/llm-usage-ingestion/workflow.json"
click node_cosmos "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/cosmos-db/cosmos-db.bicep"
click node_upgrade "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/apim-gateway-upgrade/main.bicep"
click node_version_api "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/version-api.bicep"
click node_release "https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/release.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_azd,node_pipeline,node_upgrade,node_release toneBlue
class node_root,node_params,node_resources,node_network toneAmber
class node_apim,node_api_surfaces,node_policies,node_backend_onboarding,node_backend_pools,node_version_api toneMint
class node_access,node_products,node_publish toneRose
class node_eventhub,node_usage_policy,node_ingestion,node_cosmos toneIndigo
```

## Components

### Delivery

- [Azure Developer CLI](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/azure.yaml): deployment entry · `azure.yaml`
- [Azure DevOps pipeline](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/.azdo/pipelines/azure-dev.yml): CI/CD · `.azdo/pipelines/azure-dev.yml`
- [APIM Gateway Upgrade](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/apim-gateway-upgrade/main.bicep): in-place gateway release · Updates gateway assets without reprovisioning the landing zone. · `bicep/infra/apim-gateway-upgrade/main.bicep`
- [Release manifest](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/release.json): release metadata · `release.json`

### Landing Zone

- [Root Bicep deployment](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/main.bicep): Bicep composition · `bicep/infra/main.bicep`
- [Deployment parameters](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/main.bicepparam): Bicep parameters · `bicep/infra/main.bicepparam`
- [Shared hub resources](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/resources.bicep): Bicep resources · Provisions AI, event, data, identity, and monitoring services. · `bicep/infra/resources.bicep`
- [Network & private access](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/networking/vnet.bicep): network modules · Supports new or existing VNets and optional private connectivity. · `bicep/infra/modules/networking/vnet.bicep`

### Runtime Governance

- [API Management](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/apim.bicep): runtime gateway · Central control boundary for AI, MCP, and A2A traffic. · `bicep/infra/modules/apim/apim.bicep`
- **Universal LLM & Unified AI APIs**: APIM APIs · Normalized consumer-facing AI surfaces.
- [Gateway policy fragments](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/policy-fragments.bicep): APIM policies · Enforces identity, quotas, PII, safety, caching, and telemetry. · `bicep/infra/modules/apim/policy-fragments.bicep`
- [LLM backend onboarding](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/llm-backend-onboarding/main.bicep): Bicep composition · Onboards Foundry, Azure OpenAI, and compatible providers. · `bicep/infra/llm-backend-onboarding/main.bicep`
- [Backend pools & routing](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/llm-backend-onboarding/modules/llm-backend-pools.bicep): APIM backends · Resolves models and aliases, then balances or fails over targets. · `bicep/infra/llm-backend-onboarding/modules/llm-backend-pools.bicep`
- [Release version API](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/version-api.bicep): APIM version endpoint · Serves independently tracked release scopes. · `bicep/infra/modules/apim/version-api.bicep`

### Contracts

- [Consumer access contracts](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/citadel-access-contracts/main.bicep): access provisioning · Creates products, subscriptions, policies, and credential delivery. · `bicep/infra/citadel-access-contracts/main.bicep`
- [APIM products](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/citadel-access-contracts/modules/apimProduct.bicep): consumer entitlement · `bicep/infra/citadel-access-contracts/modules/apimProduct.bicep`
- [Asset publish contracts](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/citadel-publish-contracts/main.bicep): publication provisioning · Exposes MCP tools and A2A agents without granting consumption access. · `bicep/infra/citadel-publish-contracts/main.bicep`

### Usage & Observability

- **Event Hub**: governance event stream · Receives APIM AI, MCP, A2A, and governance events.
- [Usage event policy](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/apim/policies/frag-llm-usage.xml): APIM policy · `bicep/infra/modules/apim/policies/frag-llm-usage.xml`
- [Usage ingestion workflow](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/src/usage-ingestion-logicapp/llm-usage-ingestion/workflow.json): Logic App workflow · `src/usage-ingestion-logicapp/llm-usage-ingestion/workflow.json`
- [Cosmos DB usage store](https://github.com/azure-samples/ai-hub-gateway-solution-accelerator/blob/main/bicep/infra/modules/cosmos-db/cosmos-db.bicep): usage data store · `bicep/infra/modules/cosmos-db/cosmos-db.bicep`

## Connections

- Azure Developer CLI → Root Bicep deployment: deploys
- Deployment parameters → Root Bicep deployment: parameterizes
- Azure DevOps pipeline → Root Bicep deployment: automates deployment
- Root Bicep deployment → Shared hub resources: provisions
- Root Bicep deployment → Network & private access: configures
- Root Bicep deployment → API Management: provisions
- API Management → Universal LLM & Unified AI APIs: hosts
- Gateway policy fragments → Universal LLM & Unified AI APIs: governs
- LLM backend onboarding → Backend pools & routing: constructs
- Backend pools & routing → Universal LLM & Unified AI APIs: routes backend calls
- Consumer access contracts → APIM products: creates
- APIM products → API Management: grants consumer access
- Asset publish contracts → API Management: publishes protected assets
- Usage event policy → Event Hub: emits events
- Gateway policy fragments → Usage event policy: includes
- Event Hub → Usage ingestion workflow: triggers
- Usage ingestion workflow → Cosmos DB usage store: ingests usage
- APIM Gateway Upgrade → API Management: updates in place
- Release version API → Release manifest: serves
- API Management → Release version API: hosts
