How envoyproxy/ai-gateway works
Main Components
a. Core Components:
• Envoy Gateway Integration
• Controller Component
• External Processor (ExtProc)
• API Layer (v1alpha1)
• Backend Services Integration
• Security Layer
b. Key Architectural Elements:
• Custom Resource Definitions (CRDs)
• LLM Backend Management
• LLM Route Management
• Request Translation Layer
• Authentication/Authorization System
Client Layer
• Incoming client requests
• Multiple client types (applications, services)
Gateway Layer
• Envoy Gateway component
• External Processor integration
• Route management
Processing Layer
• Controller component
• Request translation
• Backend authentication
• Request routing
Backend Integration Layer
• LLM service providers (AWS Bedrock, OpenAI)
• Security policies
• Backend configurations
Use these specific visual elements:
• Components: Use rectangles with rounded corners
• API Endpoints: Hexagons
• External Services: Clouds
• Blue: Core Envoy components
• Green: Processing components
• Yellow: Security components
• Purple: External services
• Gray: Infrastructure components
Key Relationships to Highlight
a. Control Plane:
• How the controller manages LLMBackends and LLMRoutes
• CRD management flow
• Configuration update paths
b. Data Plane:
• Request flow from clients through Envoy
• Translation process for different backends
• Security policy enforcement points
Important Implementation Details
Specific components based on the file structure:
• API Schema Translations (from internal/apischema)
• Backend Authentication Methods (from internal/extproc/backendauth)
• Request Translation Layer (from internal/extproc/translator)
• Route Management (from internal/extproc/router)
Specific Implementation Notes
• The separation between controller and extproc components
• The translation layer for different LLM providers
• The security policy implementation
• The custom resource definitions (from manifests/charts)