# houssemdellai/ai-gateway architecture

> Architecture diagram of the GitHub repository houssemdellai/ai-gateway, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/houssemdellai/ai-gateway
- Repository: https://github.com/houssemdellai/ai-gateway

## Overview

##### Main Components

a. Core Infrastructure Layer:
- Azure API Management (Central Component)
- Azure OpenAI Service Endpoints
- Azure Monitor/Application Insights
- Azure Identity & Access Management

b. Feature Modules Layer:
- Load Balancing Module
- Token Management Module
- Caching System
- Security & Access Control
- Logging & Monitoring
- Content Filtering
- Vector Search Capabilities

c. Integration Layer:
- Function Apps
- Event Hub
- Cosmos DB
- Azure AI Search
- Azure AI Content Safety

##### Relationships and Data Flow

- Client requests flowing through APIM
- Request routing to appropriate OpenAI endpoints
- Monitoring and logging flows
- Security validation paths
- Cache interaction paths
- Vector search operations

##### Design Approach

**Top Layer: Client Applications**

**Middle Layer: API Management Gateway**
- Security & Authentication
- Load Balancing
- Rate Limiting
- Caching
- Monitoring

**Bottom Layer: Backend Services**
- Azure OpenAI Services
- Supporting Services (Functions, Search, etc.)
- Storage & Databases

##### Implementation Details

- Built-in load balancing capabilities
- Token rate limiting mechanisms
- Semantic caching implementation
- Access control flows
- Streaming capabilities
- Vector search integration
- Content filtering mechanisms

##### Specific Features to Emphasize

- Backend pool load balancing
- Token rate limiting and metrics
- Semantic caching
- Access control mechanisms
- GPT-4 integration
- Function calling capabilities
- Content filtering
- Vector searching

##### Documentation Elements

Include:
- Version information
- Key metrics and SLA points
- Security compliance indicators
- Integration points

Remember to include the Well-Architected Framework pillars (Security, Reliability, Performance, Operations, and Costs) as they apply to different components of the system.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client["Client Layer"]
        CL[Client Applications]
    end

    subgraph Gateway["API Management Gateway"]
        APIM[Azure API Management]:::apim
        click APIM "https://github.com/HoussemDellai/AI-Gateway/blob/main/modules/apim/v1/apim.bicep"

        subgraph Security["Security Components"]
            AC[Access Control]:::security
            click AC "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/access-controlling"
            CF[Content Filter]:::security
            click CF "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/content-filtering"
        end

        subgraph Core["Core Features"]
            LB[Load Balancer]:::core
            click LB "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/backend-pool-load-balancing"
            TM[Token Management]:::core
            click TM "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/token-rate-limiting"
            SC[Semantic Cache]:::core
            click SC "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/semantic-caching"
            MR[Model Router]:::core
            click MR "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/model-routing"
            RS[Response Streaming]:::core
            click RS "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/response-streaming"
        end

        subgraph Monitoring["Monitoring & Logging"]
            AL[Application Logging]:::monitor
            click AL "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/built-in-logging"
            TMe[Token Metrics]:::monitor
            click TMe "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/token-metrics-emitting"
            AppI[Application Insights]:::monitor
            click AppI "https://github.com/HoussemDellai/AI-Gateway/blob/main/modules/monitor/v1/appinsights.bicep"
        end
    end

    subgraph Backend["Backend Services"]
        subgraph AI["AI Services"]
            AO[Azure OpenAI]:::ai
            click AO "https://github.com/HoussemDellai/AI-Gateway/blob/main/modules/cognitive-services/v1/openai.bicep"
            VS[Vector Search]:::ai
            click VS "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/vector-searching"
            PF[Prompt Flow]:::ai
            click PF "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/prompt-flow"
        end

        subgraph Integration["Integration Services"]
            FA[Function Apps]:::integration
            click FA "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/function-calling"
            MS[Message Storage]:::integration
            click MS "https://github.com/HoussemDellai/AI-Gateway/tree/main/labs/message-storing"
        end
    end

    %% Connections
    CL --> APIM
    APIM --> Security
    APIM --> Core
    APIM --> Monitoring
    Security --> AI
    Core --> AI
    Core --> Integration
    AI --> Integration

    %% Styles
    classDef apim fill:#1E88E5,stroke:#0D47A1,color:white
    classDef security fill:#FDD835,stroke:#F57F17,color:black
    classDef core fill:#43A047,stroke:#1B5E20,color:white
    classDef monitor fill:#8E24AA,stroke:#4A148C,color:white
    classDef ai fill:#00ACC1,stroke:#006064,color:white
    classDef integration fill:#FB8C00,stroke:#E65100,color:white

    %% Legend
    subgraph Legend
        L1[API Management]:::apim
        L2[Security Components]:::security
        L3[Core Features]:::core
        L4[Monitoring]:::monitor
        L5[AI Services]:::ai
        L6[Integration]:::integration
    end
```
