How azure-samples/ai-hub-gateway-solution-accelerator works
1. Purpose and entry points
Citadel Governance Hub is an Azure solution accelerator for a centralized AI runtime governance plane. Azure API Management (APIM) fronts LLMs, AI services, MCP tools, and A2A agents with declarative security, routing, policy enforcement, and usage tracking.
Initial deployment is driven by Azure Developer CLI (azure.yaml) into the Bicep root deployment: bicep/infra/main.bicep with bicep/infra/main.bicepparam. CI/CD is defined in .azdo/pipelines/azure-dev.yml.
2. Landing-zone composition
The root Bicep composition provisions the shared hub dependencies and delegates resource construction to reusable modules:
• bicep/infra/main.bicep
• bicep/infra/resources.bicep
• bicep/infra/modules/apim/apim.bicep
Material Azure services include APIM, Microsoft Foundry/AI Services and model deployments, API Center, Event Hub, Cosmos DB, Logic Apps, Key Vault, Managed Redis, Log Analytics, Application Insights, managed identities, VNets, private endpoints, and DNS. Networking supports new or existing VNets and optional private connectivity through bicep/infra/modules/networking/vnet.bicep and bicep/infra/modules/networking/private-endpoint.bicep.
3. APIM is the runtime control boundary
Clients and agent execution planes call APIM rather than AI backends directly. APIM APIs import OpenAPI definitions and apply policy fragments for authentication, model access, routing, quotas/throttling, PII handling, content-safety integration, caching, and telemetry.
Core gateway definition paths:
• bicep/infra/modules/apim/apim.bicep
• bicep/infra/modules/apim/universal-llm-api.bicep
• bicep/infra/modules/apim/policy-fragments.bicep
The Universal LLM and Unified AI APIs provide normalized client-facing surfaces, while backend details remain controlled behind APIM.
4. LLM supply-side onboarding and routing
Backend contracts independently onboard Azure OpenAI/Foundry and compatible external providers, then construct APIM backends and pools. Policies resolve requested models or aliases, select target pools, authorize backend calls, and enable resiliency behavior such as balancing/failover.
Key implementation paths:
• bicep/infra/llm-backend-onboarding/main.bicep
• bicep/infra/llm-backend-onboarding/modules/llm-backend-pools.bicep
• bicep/infra/modules/apim/policies/universal-llm-api-policy-v2.xml
This separation lets platform teams alter AI capacity and routing without changing consumer contracts.
5. Consumer access and asset publication contracts
Access contracts implement the demand boundary: each use case/environment can receive APIM products, subscriptions, policies, and credential delivery via Key Vault or Foundry connections. Publish contracts separately expose MCP tools and A2A agents through protected APIM endpoints; publication does not grant consumption access.
Key paths:
• bicep/infra/citadel-access-contracts/main.bicep
• bicep/infra/citadel-access-contracts/modules/apimProduct.bicep
• bicep/infra/citadel-publish-contracts/main.bicep
Identity is based on Entra ID, managed identities, APIM subscriptions, JWT/OAuth policies, and Key Vault secret storage.
6. Usage, observability, and analytics flow
APIM policies emit AI/LLM/MCP/A2A and governance events to Event Hub. Logic App workflows consume event streams and ingest usage records into Cosmos DB; Log Analytics and Application Insights provide platform telemetry, dashboards, and alerting. Power BI artifacts consume retained usage/cost data.
Core paths:
• bicep/infra/modules/apim/policies/frag-llm-usage.xml
• src/usage-ingestion-logicapp/llm-usage-ingestion/workflow.json
• bicep/infra/modules/cosmos-db/cosmos-db.bicep
7. Lifecycle and operational boundaries
The root deployment is intended for initial provisioning. Subsequent gateway releases are applied in place through the APIM Gateway Upgrade module, preserving the landing-zone infrastructure. release.json is exposed by APIM’s version API and includes independently tracked release scopes.
Key paths:
• bicep/infra/apim-gateway-upgrade/main.bicep
• bicep/infra/modules/apim/version-api.bicep
• release.json
Main entry points: main.bicep Root Bicep deployment · apim.bicep API Management · policy-fragments.bicep Gateway policy fragments · llm-backend-pools.bicep Backend pools & routing
Components 21
Delivery
- Azure Developer CLI
azure.yaml - Azure DevOps pipeline
.azdo/pipelines/azure-dev.yml - APIM Gateway Upgrade · Updates gateway assets without reprovisioning the landing zone.
bicep/infra/apim-gateway-upgrade/main.bicep - Release manifest
release.json
Landing Zone
- Root Bicep deployment
bicep/infra/main.bicep - Deployment parameters
bicep/infra/main.bicepparam - Shared hub resources · Provisions AI, event, data, identity, and monitoring services.
bicep/infra/resources.bicep - Network & private access · Supports new or existing VNets and optional private connectivity.
bicep/infra/modules/networking/vnet.bicep
Runtime Governance
- API Management · Central control boundary for AI, MCP, and A2A traffic.
bicep/infra/modules/apim/apim.bicep - Universal LLM & Unified AI APIs · Normalized consumer-facing AI surfaces.
- Gateway policy fragments · Enforces identity, quotas, PII, safety, caching, and telemetry.
bicep/infra/modules/apim/policy-fragments.bicep - LLM backend onboarding · Onboards Foundry, Azure OpenAI, and compatible providers.
bicep/infra/llm-backend-onboarding/main.bicep - Backend pools & routing · Resolves models and aliases, then balances or fails over targets.
bicep/infra/llm-backend-onboarding/modules/llm-backend-pools.bicep - Release version API · Serves independently tracked release scopes.
bicep/infra/modules/apim/version-api.bicep
Contracts
- Consumer access contracts · Creates products, subscriptions, policies, and credential delivery.
bicep/infra/citadel-access-contracts/main.bicep - APIM products
bicep/infra/citadel-access-contracts/modules/apimProduct.bicep - Asset publish contracts · Exposes MCP tools and A2A agents without granting consumption access.
bicep/infra/citadel-publish-contracts/main.bicep
Usage & Observability
- Event Hub · Receives APIM AI, MCP, A2A, and governance events.
- Usage event policy
bicep/infra/modules/apim/policies/frag-llm-usage.xml - Usage ingestion workflow
src/usage-ingestion-logicapp/llm-usage-ingestion/workflow.json - Cosmos DB usage store
bicep/infra/modules/cosmos-db/cosmos-db.bicep
Connections 20
- Azure Developer CLI → Root Bicep deployment: deploys
- Deployment parameters → Root Bicep deployment: parameterizes
- Azure DevOps pipeline → Root Bicep deployment: automates deployment
- Root Bicep deployment → Shared hub resources: provisions
- Root Bicep deployment → Network & private access: configures
- Root Bicep deployment → API Management: provisions
- API Management → Universal LLM & Unified AI APIs: hosts
- Gateway policy fragments → Universal LLM & Unified AI APIs: governs
- LLM backend onboarding → Backend pools & routing: constructs
- Backend pools & routing → Universal LLM & Unified AI APIs: routes backend calls
- Consumer access contracts → APIM products: creates
- APIM products → API Management: grants consumer access
- Asset publish contracts → API Management: publishes protected assets
- Usage event policy → Event Hub: emits events
- Gateway policy fragments → Usage event policy: includes
- Event Hub → Usage ingestion workflow: triggers
- Usage ingestion workflow → Cosmos DB usage store: ingests usage
- APIM Gateway Upgrade → API Management: updates in place
- Release version API → Release manifest: serves
- API Management → Release version API: hosts