# microsoft/conversation-knowledge-mining-solution-accelerator architecture

> Architecture diagram of the GitHub repository microsoft/conversation-knowledge-mining-solution-accelerator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/conversation-knowledge-mining-solution-accelerator
- Repository: https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator

## Overview

##### Project type and purpose
- This is a full-stack “solution accelerator” that ingests, processes, and visualizes conversational data.
- Key goals:
- Transform audio/text conversation data into searchable indexes.
- Extract topics, key phrases, perform speech-to-text, and support interactive chat.
- Deploy end-to-end on Azure using App Service, Container Apps, AI services, databases, and search.

##### File structure to locate main components
- `/src/App` → React front-end (WebApp.Dockerfile, package.json, `src/*`)
- `/src/api` → Python API (FastAPI/Flask style `app.py`, `api_routes.py`, models, services)
- `/infra` → Bicep templates provisioning Azure resources (App Service, Cosmos DB, SQL DB, Storage, Key Vault, AI services, Container Registry)
- `/infra/scripts` → Data-processing scripts for search index, Azure AI Foundry (notebooks, Python, PowerShell, shell scripts)
- CI/CD: `.github/workflows` and `.azdo/pipelines` for build, test, deploy.

##### README for architecture insights
- The README contains a “Solution architecture” image showing high-level flows.
- Services used: Azure AI Foundry, Azure AI Content Understanding, Azure OpenAI Service, Azure AI Search, Storage, SQL, Cosmos DB, Key Vault, Managed Identity.

##### Main components
a. User Interface Layer
- React SPA deployed to Azure App Service (or Container App)
- Public web endpoint (HTTPS)
b. API Layer
- Python API container (App Service or Container App)
- Exposes REST endpoints: `/api/chat`, `/api/history`, `/api/chart`
c. Data Stores
- Azure SQL Database (structured metadata)
- Azure Cosmos DB (history, user sessions)
- Azure Blob Storage (raw audio/text files, intermediate data)
d. AI Services
- Azure AI Content Understanding (topic modeling, key-phrase extraction)
- Azure AI Foundry (processing pipelines, orchestration notebooks)
- Azure OpenAI Service (chat generation, embeddings)
- Azure AI Search (search index for processed conversation data)
e. Auxiliary Services
- Azure Key Vault (secrets, connection strings)
- Azure Container Registry (for Docker images)
- Managed Identity (secure access between services)
- GitHub Actions / Azure DevOps pipelines (CI/CD)
f. Data-Processing Scripts
- Azure CLI / PowerShell / Python scripts triggered post-deployment (index creation, data ingestion)
- Microsoft Fabric notebooks for advanced data prep (optional sub-component)

##### Relationships and interactions
1. User’s browser → HTTPS → React SPA
2. React SPA → REST calls → Python API
3. Python API:
- Reads/writes SQL & Cosmos DB via service clients
- Calls AI services:
- Content Understanding to extract topics/key phrases
- OpenAI for embeddings and chat responses
- Pushes processed data to Azure AI Search index
4. Data-ingestion pipeline scripts:
- Pull raw data from Blob Storage
- Use AI Foundry pipelines to process and output JSON
- Call Azure Search REST API to create or update the index
5. Deployment pipelines (GitHub Actions/Azure DevOps) → execute Bicep templates → provision infra → build & push Docker images → deploy App Service/Container App → run post-deployment scripts

##### Architectural patterns & design principles
- Separation of concerns: UI vs API vs Data Processing vs Infra provisioning.
- Microservices-style: API in its own container, UI in its own container.
- Infrastructure as Code: Bicep modules for repeatable, parameterized deployments.
- Managed Identity & Key Vault: no secrets in code, secure service-to-service auth.
- Eventual consistency: search index reflects processed data asynchronously.

##### Essential elements
- Clear labels: e.g., “Azure App Service (React)”, “Python API Container”, “Azure OpenAI: gpt-4o-mini”.
- A note highlighting optional advanced components (Fabric notebooks) but keeping primary focus on core solution accelerator.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend Layer
    subgraph "Frontend Layer"
        direction TB
        ReactApp["Azure App Service (React SPA)"]:::azurePaaS
    end

    %% Backend Layer
    subgraph "Backend Layer"
        direction TB
        PythonAPI["Python API Container"]:::azurePaaS
        APIRoutes["api_routes.py"]:::container
        HistoryRoutes["history_routes.py"]:::container
        AppMain["app.py"]:::container
        SQService["sqldb_service.py"]:::container
        CosmosService["cosmosdb_service.py"]:::container
        ChatHelper["chat_helper.py"]:::container
        ChatService["chat_service.py"]:::container
    end

    %% Data & AI Layer
    subgraph "Data & AI Layer"
        direction TB
        SQLDB["Azure SQL Database"]:::database
        CosmosDB["Azure Cosmos DB"]:::database
        BlobStorage["Azure Blob Storage"]:::database
        AIContent["Azure AI Content Understanding"]:::aiService
        AIFoundry["Azure AI Foundry"]:::aiService
        OpenAIService["Azure OpenAI Service"]:::aiService
        AISearch["Azure AI Search"]:::aiService
        KeyVault["Azure Key Vault"]:::azurePaaS
        ManagedIdentity["Managed Identity"]:::azurePaaS
    end

    %% Deployment Layer
    subgraph "Deployment"
        direction TB
        BicepInfra["Bicep Templates"]:::container
        DeployAppService["deploy_app_service.bicep"]:::container
        DeployFEImage["deploy_frontend_docker.bicep"]:::container
        DeployBEImage["deploy_backend_docker.bicep"]:::container
        DeploySQL["deploy_sql_db.bicep"]:::container
        DeployCosmos["deploy_cosmos_db.bicep"]:::container
        DeployStorage["deploy_storage_account.bicep"]:::container
        DeployAIFoundry["deploy_ai_foundry.bicep"]:::container
        DeployKeyVault["deploy_keyvault.bicep"]:::container
        DeployIdentity["deploy_managed_identity.bicep"]:::container
        DeployPostScripts["deploy_post_deployment_scripts.bicep"]:::container
        ScriptsIndex["index_scripts/"]:::container
        ScriptsFabric["fabric_scripts/"]:::container
        CI_GH[".github/workflows/"]:::container
        CI_AZDO["azure-dev.yml"]:::container
        FetchImageModule["fetch-container-image.bicep"]:::container
    end

    %% Config & Samples
    subgraph "Configuration & Samples"
        direction TB
        EnvSample[".env.sample"]:::container
        FrontendConfig["config.json"]:::container
    end

    %% Connections
    ReactApp -->|"HTTPS"| PythonAPI
    PythonAPI -->|"reads/writes"| SQLDB
    PythonAPI -->|"reads/writes"| CosmosDB
    PythonAPI -->|"stores raw/intermediate"| BlobStorage
    PythonAPI -->|"calls"| AIContent
    PythonAPI -->|"calls"| OpenAIService
    PythonAPI -->|"pushes to"| AISearch
    PythonAPI -->|"access via"| KeyVault
    PythonAPI -->|"auth via"| ManagedIdentity

    %% API internals
    PythonAPI --> AppMain
    AppMain --> APIRoutes
    AppMain --> HistoryRoutes
    PythonAPI --> SQService
    PythonAPI --> CosmosService
    PythonAPI --> ChatHelper
    PythonAPI --> ChatService

    %% Deployment flows
    CI_GH -->|deploy| BicepInfra
    CI_AZDO -->|deploy| BicepInfra
    BicepInfra --> DeployAppService
    BicepInfra --> DeployFEImage
    BicepInfra --> DeployBEImage
    BicepInfra --> DeploySQL
    BicepInfra --> DeployCosmos
    BicepInfra --> DeployStorage
    BicepInfra --> DeployAIFoundry
    BicepInfra --> DeployKeyVault
    BicepInfra --> DeployIdentity
    BicepInfra --> DeployPostScripts
    DeployPostScripts -->|triggers| ScriptsIndex
    DeployPostScripts -->|triggers| ScriptsFabric

    %% Config & samples usage
    AppMain --> EnvSample
    ReactApp --> FrontendConfig

    %% Class Definitions
    classDef azurePaaS fill:#ADD8E6,stroke:#0366d6,stroke-width:1px;
    classDef database fill:#90EE90,stroke:#228B22,stroke-width:1px;
    classDef aiService fill:#FFA500,stroke:#DAA520,stroke-width:1px;
    classDef container fill:#D3D3D3,stroke:#696969,stroke-width:1px;

    %% Click Events
    click ReactApp "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/tree/main/src/App/"
    click PythonAPI "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/tree/main/src/api/"
    click APIRoutes "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/src/api/api/api_routes.py"
    click HistoryRoutes "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/src/api/api/history_routes.py"
    click AppMain "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/src/api/app.py"
    click SQService "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/src/api/common/database/sqldb_service.py"
    click CosmosService "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/src/api/common/database/cosmosdb_service.py"
    click ChatHelper "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/src/api/helpers/chat_helper.py"
    click ChatService "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/src/api/services/chat_service.py"
    click DeployAppService "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_app_service.bicep"
    click DeployFEImage "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_frontend_docker.bicep"
    click DeployBEImage "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_backend_docker.bicep"
    click DeploySQL "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_sql_db.bicep"
    click DeployCosmos "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_cosmos_db.bicep"
    click DeployStorage "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_storage_account.bicep"
    click DeployAIFoundry "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_ai_foundry.bicep"
    click DeployKeyVault "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_keyvault.bicep"
    click DeployIdentity "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_managed_identity.bicep"
    click DeployPostScripts "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/deploy_post_deployment_scripts.bicep"
    click ScriptsIndex "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/tree/main/infra/scripts/index_scripts/"
    click ScriptsFabric "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/tree/main/infra/scripts/fabric_scripts/"
    click CI_GH "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/tree/main/.github/workflows/"
    click CI_AZDO "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/.azdo/pipelines/azure-dev.yml"
    click FetchImageModule "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/infra/modules/fetch-container-image.bicep"
    click EnvSample "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/src/api/.env.sample"
    click FrontendConfig "https://github.com/microsoft/conversation-knowledge-mining-solution-accelerator/blob/main/src/App/public/config/config.json"
```
