# pranavatla/gate architecture

> Architecture diagram of the GitHub repository pranavatla/gate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pranavatla/gate
- Repository: https://github.com/pranavatla/gate
- Video tour (about a minute): https://gitdiagram.com/pranavatla/gate/video

## Overview

gate.atla.in is a multi-tenant LLM gateway: tenant applications send a unified chat request using a virtual key, and the gateway applies governance and operational controls before returning a provider response. The chat handler owns the request workflow, including prompt/OKF enrichment, policy checks, cache and budget decisions, provider failover, and audit recording. Separate capabilities cover agent governance, administration, evaluations, and operational statistics. The README documents the broader product flow; some subsystem links below are therefore documented rather than verified against unsampled implementation.

Main entry points: [`app/main.py`](https://github.com/pranavatla/gate/blob/main/app/main.py) (Chat API), [`app/router.py`](https://github.com/pranavatla/gate/blob/main/app/router.py) (Provider Router), [`app/providers/http.py`](https://github.com/pranavatla/gate/blob/main/app/providers/http.py) (Provider HTTP), [`app/auth.py`](https://github.com/pranavatla/gate/blob/main/app/auth.py) (Tenant Authentication)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_governance["Tenant Governance"]
  node_tenant_auth["Tenant Authentication<br/>[auth.py]"]
  node_key_hash["Key Hashing<br/>[keys.py]"]
  node_policy["Request Policy<br/>[policy.py]"]
  node_rate_limit["Rate Limits<br/>[ratelimit.py]"]
  node_agent_governance["Agent Governance<br/>[agents.py]"]
  node_budget["Budget Controls<br/>[budget.py]"]
  node_admin["Tenant Administration<br/>[admin.py]"]
end

subgraph group_request["Chat Request"]
  node_chat_api["Chat API<br/>[main.py]"]
  node_prompt_registry["Prompt Registry<br/>[prompts.py]"]
  node_okf["OKF Bundles<br/>[okf.py]"]
  node_feature_flags["Feature Flags<br/>[flags.py]"]
  node_semantic_cache["Semantic Cache<br/>[cache.py]"]
end

subgraph group_providers["Provider Execution"]
  node_failover["Failover and Breakers<br/>[failover.py]"]
  node_provider_router["Provider Router<br/>[router.py]"]
  node_anthropic_adapter["Anthropic Adapter"]
  node_openai_adapter["OpenAI Adapter<br/>[openai_adapter.py]"]
  node_gemini_adapter["Gemini Adapter<br/>[gemini_adapter.py]"]
  node_bedrock_adapter["Bedrock Adapter<br/>[bedrock_adapter.py]"]
  node_provider_http["Provider HTTP<br/>[http.py]"]
end

subgraph group_quality["Quality and Insight"]
  node_eval_cases["Eval Case Import and Generation<br/>[evalimport.py]"]
  node_eval_generation["OKF Eval Generation<br/>[evalgen.py]"]
  node_eval_runner["Eval Runner<br/>[evalrun.py]"]
  node_eval_scoring["Eval Scoring<br/>[scorers.py]"]
  node_eval_compare["Eval Comparison<br/>[evalcheck.py]"]
  node_audit["Audit Trail<br/>[audit.py]"]
  node_statistics["Gateway Statistics<br/>[stats.py]"]
  node_quality_statistics["Eval Statistics<br/>[stats_quality.py]"]
  node_tracing["Optional Tracing<br/>[tracing.py]"]
end

subgraph group_state["Runtime State"]
  node_postgres[("PostgreSQL and pgvector")]
  node_redis[("Redis")]
end

node_tenant_app(("Tenant App"))
node_provider_services(("Model Providers"))

node_tenant_app -->|"sends chat"| node_chat_api
node_chat_api -->|"authenticates"| node_tenant_auth
node_tenant_auth -->|"hashes key"| node_key_hash
node_tenant_auth -->|"reads tenant"| node_postgres
node_chat_api -->|"checks limits"| node_rate_limit
node_rate_limit -->|"uses token bucket"| node_redis
node_chat_api -->|"applies policy"| node_policy
node_chat_api -->|"loads context"| node_okf
node_chat_api -->|"resolves prompt"| node_prompt_registry
node_chat_api -->|"checks flags"| node_feature_flags
node_chat_api -->|"checks agent"| node_agent_governance
node_chat_api -->|"checks cache"| node_semantic_cache
node_semantic_cache -->|"reads and writes"| node_postgres
node_chat_api -->|"decides budget"| node_budget
node_chat_api -->|"requests completion"| node_failover
node_failover -->|"routes model"| node_provider_router
node_provider_router -->|"dispatches"| node_anthropic_adapter
node_provider_router -->|"dispatches"| node_openai_adapter
node_provider_router -->|"dispatches"| node_gemini_adapter
node_provider_router -->|"dispatches"| node_bedrock_adapter
node_anthropic_adapter -->|"posts request"| node_provider_http
node_openai_adapter -->|"posts request"| node_provider_http
node_gemini_adapter -->|"posts request"| node_provider_http
node_bedrock_adapter -->|"posts request"| node_provider_http
node_provider_http -->|"calls provider"| node_provider_services
node_chat_api -->|"records outcome"| node_audit
node_audit -->|"appends event"| node_postgres
node_eval_cases -.->|"stores cases"| node_postgres
node_eval_generation -.->|"generates cases"| node_eval_cases
node_eval_runner -->|"sends eval requests"| node_chat_api
node_eval_runner -->|"scores answers"| node_eval_scoring
node_eval_scoring -->|"provides scores"| node_eval_compare
node_eval_compare -.->|"uses run history"| node_postgres
node_chat_api -->|"serves stats"| node_statistics
node_chat_api -->|"serves eval stats"| node_quality_statistics
node_statistics -->|"reads metrics"| node_postgres
node_quality_statistics -->|"reads eval data"| node_postgres
node_admin -.->|"manages tenants"| node_postgres
node_chat_api -->|"annotates requests"| node_tracing

click node_chat_api "https://github.com/pranavatla/gate/blob/main/app/main.py"
click node_tenant_auth "https://github.com/pranavatla/gate/blob/main/app/auth.py"
click node_key_hash "https://github.com/pranavatla/gate/blob/main/app/keys.py"
click node_policy "https://github.com/pranavatla/gate/blob/main/app/policy.py"
click node_rate_limit "https://github.com/pranavatla/gate/blob/main/app/ratelimit.py"
click node_agent_governance "https://github.com/pranavatla/gate/blob/main/app/agents.py"
click node_prompt_registry "https://github.com/pranavatla/gate/blob/main/app/prompts.py"
click node_okf "https://github.com/pranavatla/gate/blob/main/app/okf.py"
click node_feature_flags "https://github.com/pranavatla/gate/blob/main/app/flags.py"
click node_semantic_cache "https://github.com/pranavatla/gate/blob/main/app/cache.py"
click node_budget "https://github.com/pranavatla/gate/blob/main/app/budget.py"
click node_failover "https://github.com/pranavatla/gate/blob/main/app/failover.py"
click node_provider_router "https://github.com/pranavatla/gate/blob/main/app/router.py"
click node_anthropic_adapter "https://github.com/pranavatla/gate/blob/main/app/providers/anthropic_adapter.py"
click node_openai_adapter "https://github.com/pranavatla/gate/blob/main/app/providers/openai_adapter.py"
click node_gemini_adapter "https://github.com/pranavatla/gate/blob/main/app/providers/gemini_adapter.py"
click node_bedrock_adapter "https://github.com/pranavatla/gate/blob/main/app/providers/bedrock_adapter.py"
click node_provider_http "https://github.com/pranavatla/gate/blob/main/app/providers/http.py"
click node_eval_cases "https://github.com/pranavatla/gate/blob/main/app/evalimport.py"
click node_eval_generation "https://github.com/pranavatla/gate/blob/main/app/evalgen.py"
click node_eval_runner "https://github.com/pranavatla/gate/blob/main/app/evalrun.py"
click node_eval_scoring "https://github.com/pranavatla/gate/blob/main/app/scorers.py"
click node_eval_compare "https://github.com/pranavatla/gate/blob/main/app/evalcheck.py"
click node_audit "https://github.com/pranavatla/gate/blob/main/app/audit.py"
click node_statistics "https://github.com/pranavatla/gate/blob/main/app/stats.py"
click node_quality_statistics "https://github.com/pranavatla/gate/blob/main/app/stats_quality.py"
click node_admin "https://github.com/pranavatla/gate/blob/main/app/admin.py"
click node_tracing "https://github.com/pranavatla/gate/blob/main/app/tracing.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tenant_auth,node_key_hash,node_policy,node_rate_limit,node_agent_governance,node_budget,node_admin toneBlue
class node_chat_api,node_prompt_registry,node_okf,node_feature_flags,node_semantic_cache toneAmber
class node_failover,node_provider_router,node_anthropic_adapter,node_openai_adapter,node_gemini_adapter,node_bedrock_adapter,node_provider_http toneMint
class node_eval_cases,node_eval_generation,node_eval_runner,node_eval_scoring,node_eval_compare,node_audit,node_statistics,node_quality_statistics,node_tracing toneRose
class node_postgres,node_redis,node_tenant_app,node_provider_services toneIndigo
```

## Components

### Tenant Governance

- [Tenant Authentication](https://github.com/pranavatla/gate/blob/main/app/auth.py): `app/auth.py`
- [Key Hashing](https://github.com/pranavatla/gate/blob/main/app/keys.py): `app/keys.py`
- [Request Policy](https://github.com/pranavatla/gate/blob/main/app/policy.py): `app/policy.py`
- [Rate Limits](https://github.com/pranavatla/gate/blob/main/app/ratelimit.py): `app/ratelimit.py`
- [Agent Governance](https://github.com/pranavatla/gate/blob/main/app/agents.py): `app/agents.py`
- [Budget Controls](https://github.com/pranavatla/gate/blob/main/app/budget.py): `app/budget.py`
- [Tenant Administration](https://github.com/pranavatla/gate/blob/main/app/admin.py): `app/admin.py`

### Chat Request

- [Chat API](https://github.com/pranavatla/gate/blob/main/app/main.py): `app/main.py`
- [Prompt Registry](https://github.com/pranavatla/gate/blob/main/app/prompts.py): `app/prompts.py`
- [OKF Bundles](https://github.com/pranavatla/gate/blob/main/app/okf.py): `app/okf.py`
- [Feature Flags](https://github.com/pranavatla/gate/blob/main/app/flags.py): `app/flags.py`
- [Semantic Cache](https://github.com/pranavatla/gate/blob/main/app/cache.py): `app/cache.py`

### Provider Execution

- [Failover and Breakers](https://github.com/pranavatla/gate/blob/main/app/failover.py): `app/failover.py`
- [Provider Router](https://github.com/pranavatla/gate/blob/main/app/router.py): `app/router.py`
- [Anthropic Adapter](https://github.com/pranavatla/gate/blob/main/app/providers/anthropic_adapter.py): `app/providers/anthropic_adapter.py`
- [OpenAI Adapter](https://github.com/pranavatla/gate/blob/main/app/providers/openai_adapter.py): `app/providers/openai_adapter.py`
- [Gemini Adapter](https://github.com/pranavatla/gate/blob/main/app/providers/gemini_adapter.py): `app/providers/gemini_adapter.py`
- [Bedrock Adapter](https://github.com/pranavatla/gate/blob/main/app/providers/bedrock_adapter.py): `app/providers/bedrock_adapter.py`
- [Provider HTTP](https://github.com/pranavatla/gate/blob/main/app/providers/http.py): `app/providers/http.py`

### Quality and Insight

- [Eval Case Import and Generation](https://github.com/pranavatla/gate/blob/main/app/evalimport.py): `app/evalimport.py`
- [OKF Eval Generation](https://github.com/pranavatla/gate/blob/main/app/evalgen.py): `app/evalgen.py`
- [Eval Runner](https://github.com/pranavatla/gate/blob/main/app/evalrun.py): `app/evalrun.py`
- [Eval Scoring](https://github.com/pranavatla/gate/blob/main/app/scorers.py): `app/scorers.py`
- [Eval Comparison](https://github.com/pranavatla/gate/blob/main/app/evalcheck.py): `app/evalcheck.py`
- [Audit Trail](https://github.com/pranavatla/gate/blob/main/app/audit.py): `app/audit.py`
- [Gateway Statistics](https://github.com/pranavatla/gate/blob/main/app/stats.py): `app/stats.py`
- [Eval Statistics](https://github.com/pranavatla/gate/blob/main/app/stats_quality.py): `app/stats_quality.py`
- [Optional Tracing](https://github.com/pranavatla/gate/blob/main/app/tracing.py): `app/tracing.py`

### Runtime State

- **PostgreSQL and pgvector**
- **Redis**

### Other components

- **Tenant App**
- **Model Providers**

## Connections

- Tenant App → Chat API: sends chat (evidence: `README.md`)
- Chat API → Tenant Authentication: authenticates (evidence: `README.md`)
- Tenant Authentication → Key Hashing: hashes key (evidence: `app/auth.py`)
- Tenant Authentication → PostgreSQL and pgvector: reads tenant (evidence: `app/auth.py`)
- Chat API → Rate Limits: checks limits (evidence: `README.md`)
- Rate Limits → Redis: uses token bucket (evidence: `README.md`)
- Chat API → Request Policy: applies policy (evidence: `app/main.py`)
- Chat API → OKF Bundles: loads context (evidence: `app/main.py`)
- Chat API → Prompt Registry: resolves prompt (evidence: `app/main.py`)
- Chat API → Feature Flags: checks flags (evidence: `app/main.py`)
- Chat API → Agent Governance: checks agent (evidence: `app/main.py`)
- Chat API → Semantic Cache: checks cache (evidence: `app/main.py`)
- Semantic Cache → PostgreSQL and pgvector: reads and writes (evidence: `README.md`)
- Chat API → Budget Controls: decides budget (evidence: `app/main.py`)
- Chat API → Failover and Breakers: requests completion (evidence: `app/main.py`)
- Failover and Breakers → Provider Router: routes model (evidence: `app/failover.py`)
- Provider Router → Anthropic Adapter: dispatches (evidence: `app/router.py`)
- Provider Router → OpenAI Adapter: dispatches (evidence: `app/router.py`)
- Provider Router → Gemini Adapter: dispatches (evidence: `app/router.py`)
- Provider Router → Bedrock Adapter: dispatches (evidence: `app/router.py`)
- Anthropic Adapter → Provider HTTP: posts request (evidence: `app/providers/anthropic_adapter.py`)
- OpenAI Adapter → Provider HTTP: posts request (evidence: `app/providers/openai_adapter.py`)
- Gemini Adapter → Provider HTTP: posts request (evidence: `app/providers/gemini_adapter.py`)
- Bedrock Adapter → Provider HTTP: posts request (evidence: `app/providers/bedrock_adapter.py`)
- Provider HTTP → Model Providers: calls provider (evidence: `app/providers/http.py`)
- Chat API → Audit Trail: records outcome (evidence: `app/main.py`)
- Audit Trail → PostgreSQL and pgvector: appends event (evidence: `README.md`)
- Eval Case Import and Generation → PostgreSQL and pgvector: stores cases
- OKF Eval Generation → Eval Case Import and Generation: generates cases
- Eval Runner → Chat API: sends eval requests (evidence: `README.md`)
- Eval Runner → Eval Scoring: scores answers (evidence: `app/evalrun.py`)
- Eval Scoring → Eval Comparison: provides scores (evidence: `README.md`)
- Eval Comparison → PostgreSQL and pgvector: uses run history
- Chat API → Gateway Statistics: serves stats (evidence: `README.md`)
- Chat API → Eval Statistics: serves eval stats (evidence: `README.md`)
- Gateway Statistics → PostgreSQL and pgvector: reads metrics (evidence: `app/stats.py`)
- Eval Statistics → PostgreSQL and pgvector: reads eval data (evidence: `app/stats_quality.py`)
- Tenant Administration → PostgreSQL and pgvector: manages tenants
- Chat API → Optional Tracing: annotates requests (evidence: `app/main.py`)