How pranavatla/gate works
gate.atla.in is a multi-tenant LLM gateway: tenant applications send a unified chat request using a virtual key, and the gateway applies governance and operational controls before returning a provider response. The chat handler owns the request workflow, including prompt/OKF enrichment, policy checks, cache and budget decisions, provider failover, and audit recording. Separate capabilities cover agent governance, administration, evaluations, and operational statistics. The README documents the broader product flow; some subsystem links below are therefore documented rather than verified against unsampled implementation.
Main entry points: main.py Chat API · router.py Provider Router · http.py Provider HTTP · auth.py Tenant Authentication
Components 32
Tenant Governance
- Tenant Authentication
app/auth.py - Key Hashing
app/keys.py - Request Policy
app/policy.py - Rate Limits
app/ratelimit.py - Agent Governance
app/agents.py - Budget Controls
app/budget.py - Tenant Administration
app/admin.py
Chat Request
- Chat API
app/main.py - Prompt Registry
app/prompts.py - OKF Bundles
app/okf.py - Feature Flags
app/flags.py - Semantic Cache
app/cache.py
Provider Execution
- Failover and Breakers
app/failover.py - Provider Router
app/router.py - Anthropic Adapter
app/providers/anthropic_adapter.py - OpenAI Adapter
app/providers/openai_adapter.py - Gemini Adapter
app/providers/gemini_adapter.py - Bedrock Adapter
app/providers/bedrock_adapter.py - Provider HTTP
app/providers/http.py
Quality and Insight
- Eval Case Import and Generation
app/evalimport.py - OKF Eval Generation
app/evalgen.py - Eval Runner
app/evalrun.py - Eval Scoring
app/scorers.py - Eval Comparison
app/evalcheck.py - Audit Trail
app/audit.py - Gateway Statistics
app/stats.py - Eval Statistics
app/stats_quality.py - Optional Tracing
app/tracing.py
Runtime State
- PostgreSQL and pgvector
- Redis
Other components
- Tenant App
- Model Providers
Connections 39
- Tenant App → Chat API: sends chat
README.md - Chat API → Tenant Authentication: authenticates
README.md - Tenant Authentication → Key Hashing: hashes key
app/auth.py - Tenant Authentication → PostgreSQL and pgvector: reads tenant
app/auth.py - Chat API → Rate Limits: checks limits
README.md - Rate Limits → Redis: uses token bucket
README.md - Chat API → Request Policy: applies policy
app/main.py - Chat API → OKF Bundles: loads context
app/main.py - Chat API → Prompt Registry: resolves prompt
app/main.py - Chat API → Feature Flags: checks flags
app/main.py - Chat API → Agent Governance: checks agent
app/main.py - Chat API → Semantic Cache: checks cache
app/main.py - Semantic Cache → PostgreSQL and pgvector: reads and writes
README.md - Chat API → Budget Controls: decides budget
app/main.py - Chat API → Failover and Breakers: requests completion
app/main.py - Failover and Breakers → Provider Router: routes model
app/failover.py - Provider Router → Anthropic Adapter: dispatches
app/router.py - Provider Router → OpenAI Adapter: dispatches
app/router.py - Provider Router → Gemini Adapter: dispatches
app/router.py - Provider Router → Bedrock Adapter: dispatches
app/router.py - Anthropic Adapter → Provider HTTP: posts request
app/providers/anthropic_adapter.py - OpenAI Adapter → Provider HTTP: posts request
app/providers/openai_adapter.py - Gemini Adapter → Provider HTTP: posts request
app/providers/gemini_adapter.py - Bedrock Adapter → Provider HTTP: posts request
app/providers/bedrock_adapter.py - Provider HTTP → Model Providers: calls provider
app/providers/http.py - Chat API → Audit Trail: records outcome
app/main.py - Audit Trail → PostgreSQL and pgvector: appends event
README.md - Eval Case Import and Generation → PostgreSQL and pgvector: stores cases
- OKF Eval Generation → Eval Case Import and Generation: generates cases
- Eval Runner → Chat API: sends eval requests
README.md - Eval Runner → Eval Scoring: scores answers
app/evalrun.py - Eval Scoring → Eval Comparison: provides scores
README.md - Eval Comparison → PostgreSQL and pgvector: uses run history
- Chat API → Gateway Statistics: serves stats
README.md - Chat API → Eval Statistics: serves eval stats
README.md - Gateway Statistics → PostgreSQL and pgvector: reads metrics
app/stats.py - Eval Statistics → PostgreSQL and pgvector: reads eval data
app/stats_quality.py - Tenant Administration → PostgreSQL and pgvector: manages tenants
- Chat API → Optional Tracing: annotates requests
app/main.py