How azure-samples/azure-devops-copilot-extension works
Project Type and Purpose
– Azure DevOps Co-Pilot is a full-stack proof-of-concept extension for Azure DevOps that leverages Azure OpenAI + Semantic Kernel.
– It comprises three layers:
• A TypeScript/React Azure DevOps extension (the “frontend”)
• A .NET Core Web API backend that orchestrates Semantic Kernel and OpenAI calls
• An Azure infrastructure layer (Bicep/ARM) that hosts and secures the services
High-Level Components
a. Azure DevOps Extension (azdo folder)
• UI Components (React/TSX) – dialogs for user stories, test cases, configuration
• Services (CopilotService.ts, ConfigurationService.ts) – wrap calls to the backend via REST
• Azure DevOps SDK client (azure-devops-extension-api)
b. API Backend (api/src & api/Mapache.PromptEvalDotNet)
• Controllers – AzDoController, TestCaseController, UserStoryController
• Middleware – API key validation, exception filters
• Prompt evaluation libraries (Mapache.PromptEvalDotNet) – implement Coherence, Relevance, Groundedness evaluators using Semantic Kernel
• Azure OpenAI / Semantic Kernel integration
c. Azure Infrastructure (infra folder)
• Bicep modules for Container Apps, Managed Environments, RBAC locks, Resource Groups
• CI/CD definitions (GitHub Actions workflows, azure-deploy.json)
d. External Services
• Azure OpenAI / Semantic Kernel
• Azure DevOps REST API
• Azure Container Apps / Managed Environment
Relationships and Data Flows
– User interacts with the Azure DevOps UI extension → extension calls backend via HTTPS (CopilotService)
– Backend Controller receives HTTP request → Middleware authenticates (ApiKeyValidator)
– Controller invokes PromptEvaluatorFactory / Semantic Kernel → issues prompts to Azure OpenAI
– Results returned to Controller → JSON response sent back to extension
– Extension updates Azure DevOps work items via Azure DevOps SDK calls
Architectural Patterns & Design Principles
– Clear separation of concerns: Extension UI vs Backend API vs Infra-as-Code
– Modular prompt evaluators (factory pattern) to add new AI evaluation metrics
– Middleware pipeline in ASP.NET Core for authentication and exception handling
– IaC idempotency via Bicep modules and nested role assignments