# microsoft/genaiops-azureaisdk-template architecture

> Architecture diagram of the GitHub repository microsoft/genaiops-azureaisdk-template, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/genaiops-azureaisdk-template
- Repository: https://github.com/microsoft/genaiops-azureaisdk-template

## Overview

##### Project Type and Purpose
- This is a full-stack AI experiment accelerator designed to manage end-to-end generative AI experiments; it covers local experimentation, evaluation, and automated deployment via GitHub Actions.
- It integrates local development environments (with Python/Conda setups), CI/CD using GitHub workflows, and infrastructure provisioning using Terraform for Azure.

##### Analyzing the File Structure
- Top-level directories such as:
– .github: Contains GitHub Actions workflows for CI/CD, experiment validations, and infrastructure deployments.
– docs: Documentation and guides covering setup, execution, and best practices.
– infra: Terraform scripts and Azure CLI scripts for provisioning Azure infrastructure (storage, AI services, etc.).
– lib: Reusable code like custom evaluators.
– llmops: Core scripts for running experiments.
– math_coding and math_coding_agent: Use case folders that encapsulate separate flows, evaluations, deployments, and data.
– tests: Unit tests for validating main scripts.
- The clear separation into “use case” folders (each containing subdirectories for data, flows, evaluations, and deployment) indicates an architecture that supports scalable experimentation for multiple AI use cases.

##### Key Architectural Components
a. Frontend/Interface & Local Development:
– The local environment integration (VS Code, Conda setup) and documentation explain that users interact via a local development environment.
b. Experimentation Engine:
– Components in llmops and use case folders (math_coding, math_coding_agent) manage experiment logic, configurations, and flow definitions.
– The experiment YAML files define the core experiment’s orchestration – including flow definitions, evaluation functions, and environment variables.
c. Evaluation & Monitoring:
– Custom evaluators in lib and different evaluators in the use case folders handle offline and online evaluation.
– There is integrated observability where logs, metrics, and evaluations are routinely tracked.
d. CI/CD and Orchestration:
– GitHub Actions workflows (inside .github/workflows) automate testing (PR validations) and deployment (dev workflows).
– Workflows for infra deployment automate provisioning through Terraform scripts located in infra/terraform.
e. Infrastructure & External Integration:
– Infra folder contains scripts (Terraform, Azure CLI) that set up Azure resources (e.g., Function Apps, Storage, Key Vault).
– Integration with Azure AI Foundry and Azure AI services is done via pre-configured connections specified in the experiment YAML.

##### Relationships and Interactions
– Local Development Environment → Experiment Engine (llmops/math_coding)
– Experiment Engine → GitHub Actions (pr and dev workflows triggering validations and deployments)
– Deployment Pipelines → Infrastructure (via Terraform in the infra folder)
– Experiment Engine accesses external services (Azure AI Foundry via defined connections in experiment YAML) using environment variables for credentials.
- Reusability aspects (e.g., the shared evaluator components in lib that serve multiple use cases).

##### Recommended Elements
- Use clear labels for each component:
- “Local Development Environment”
- “Experiment Engine” (with inner sub-components for flows, evaluations, and experiments)
- “CI/CD Pipelines (GitHub Actions)” – differentiate PR validation and Dev workflows
- “Azure Infrastructure” – Terraform-based deployment and Azure AI services
- “External Services” – AI Agents, OpenAI connections, and monitoring/logging endpoints.
- How data (e.g., dataset inputs, evaluation results) flows from experiment initiation to AI service calls.
- Deployment triggers, such as pushing code to GitHub initiating CI/CD workflows.
- Rectangles for components (folders/modules).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Local Development Environment
    subgraph "Local Development Environment"
        LDE1["Docs & Guides"]:::local
        LDE2[".env Sample"]:::local
    end

    %% Experimentation Engine
    subgraph "Experimentation Engine"
        EE1["Core Experiment Scripts (llmops)"]:::experiment
        EE2["Math Coding Use Case"]:::experiment
        EE3["Math Coding Agent Use Case"]:::experiment
    end

    %% Evaluation & Monitoring
    subgraph "Evaluation & Monitoring"
        EM1["Shared Evaluators (lib)"]:::evaluation
        EM2["Math Coding Evaluations"]:::evaluation
        EM3["Math Coding Agent Evaluations"]:::evaluation
    end

    %% CI/CD Pipelines
    subgraph "CI/CD Pipelines"
        CI1["GitHub Actions Workflows"]:::ci
        CI2["GitHub Actions Additional"]:::ci
    end

    %% Deployment & Infrastructure
    subgraph "Deployment & Infrastructure"
        DI1["Terraform Scripts"]:::infra
        DI2["Azure CLI Scripts"]:::infra
    end

    %% External Integration
    subgraph "External Integration"
        EI1["Azure AI Foundry"]:::external
        EI2["Function App"]:::external
    end

    %% Relationships and Data Flows
    LDE1 -->|"triggers"| EE1
    LDE2 -->|"provides-config"| EE1

    EE1 -->|"runs-experiment"| EE2
    EE1 -->|"runs-experiment"| EE3

    EE1 -->|"logs-evaluation"| EM1
    EE2 -->|"evaluates"| EM2
    EE3 -->|"evaluates"| EM3

    EE1 -->|"pushes-code"| CI1

    CI1 -->|"deploys"| DI1
    CI1 -->|"deploys"| DI2

    DI1 -->|"provisions"| EI1
    DI1 -->|"provisions"| EI2

    EE1 -->|"calls-API"| EI1

    EM1 -->|"reports-metrics"| EI2

    %% Click Events for Local Development Environment
    click LDE1 "https://github.com/microsoft/genaiops-azureaisdk-template/tree/main/docs/guides"
    click LDE2 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/.env.sample"

    %% Click Events for Experimentation Engine
    click EE1 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/llmops/experiment.py"
    click EE2 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/math_coding/experiment.yaml"
    click EE3 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/math_coding_agent/experiment.yaml"

    %% Click Events for Evaluation & Monitoring
    click EM1 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/lib/agent_eval/agent_score.py"
    click EM2 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/math_coding/evaluations/eval_f1_score.py"
    click EM3 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/math_coding_agent/evaluations/eval_agent_score.py"

    %% Click Events for CI/CD Pipelines
    click CI1 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/.github/workflows/infra_deploy_terraform_workflow.yaml"
    click CI2 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/.github/actions/execute_script/action.yml"

    %% Click Events for Deployment & Infrastructure
    click DI1 "https://github.com/microsoft/genaiops-azureaisdk-template/tree/main/infra/terraform"
    click DI2 "https://github.com/microsoft/genaiops-azureaisdk-template/tree/main/infra/azcli"

    %% Click Events for External Integration
    click EI1 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/infra/terraform/ai_foundry.tf"
    click EI2 "https://github.com/microsoft/genaiops-azureaisdk-template/blob/main/infra/terraform/function_app.tf"

    %% Styles
    classDef local fill:#AED6F1,stroke:#333,stroke-width:2px;
    classDef experiment fill:#A9DFBF,stroke:#333,stroke-width:2px;
    classDef evaluation fill:#F9E79F,stroke:#333,stroke-width:2px;
    classDef ci fill:#F5B7B1,stroke:#333,stroke-width:2px;
    classDef infra fill:#D7BDE2,stroke:#333,stroke-width:2px;
    classDef external fill:#FAD7A0,stroke:#333,stroke-width:2px;
```
