# mirrowel/llm-api-key-proxy architecture

> Architecture diagram of the GitHub repository mirrowel/llm-api-key-proxy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mirrowel/llm-api-key-proxy
- Repository: https://github.com/mirrowel/llm-api-key-proxy

## Overview

The product combines a self-hosted OpenAI/Anthropic-compatible proxy with a reusable provider-rotation and resilience library. The main workflow is client request → proxy endpoint → rotating client → credential selection and limit checks → provider API → compatible response. Usage tracking, model metadata, and quota controls support that workflow. Some wiring is inferred from documented responsibilities and the tree; unsampled implementation details and uncertain links are omitted.

Main entry points: [`src/rotator_library/client/rotating_client.py`](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/rotating_client.py) (Rotating Client), [`src/rotator_library/usage/manager.py`](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/manager.py) (Usage Manager), [`src/proxy_app/main.py`](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/proxy_app/main.py) (Proxy Server), [`src/rotator_library/client/executor.py`](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/executor.py) (Request Executor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Proxy API"]
  node_proxy["Proxy Server<br/>[main.py]"]
  node_openai_endpoint["OpenAI Endpoint<br/>[main.py]"]
  node_anthropic_endpoint["Anthropic Endpoints<br/>[main.py]"]
  node_proxy_tools["Proxy Utilities"]
end

subgraph group_client["Request Resilience"]
  node_anthropic_compat["Anthropic Translation<br/>[translator.py]"]
  node_rotating_client["Rotating Client<br/>[rotating_client.py]"]
  node_executor["Request Executor<br/>[executor.py]"]
  node_retry["Stream Retry"]
  node_transforms["Request Transforms<br/>[transforms.py]"]
end

subgraph group_providers["Provider Integrations"]
  node_provider_factory["Provider Factory"]
  node_provider_plugins["Provider Plugins"]
  node_litellm["LiteLLM Providers"]
  node_provider_api(("LLM Provider APIs"))
end

subgraph group_usage["Usage Governance"]
  node_usage_manager["Usage Manager<br/>[manager.py]"]
  node_selection["Credential Selection<br/>[engine.py]"]
  node_limits["Limit Checks<br/>[engine.py]"]
  node_tracking["Usage Tracking<br/>[engine.py]"]
  node_usage_storage[("Usage State<br/>[storage.py]")]
end

subgraph group_models["Model Intelligence"]
  node_model_info["Model Registry"]
  node_catalogs(("Model Catalogs"))
  node_model_definitions["Model Definitions"]
end

node_consumer(("API Client"))

node_consumer -->|"sends chat"| node_openai_endpoint
node_consumer -->|"sends messages"| node_anthropic_endpoint
node_openai_endpoint -->|"handled by"| node_proxy
node_anthropic_endpoint -->|"handled by"| node_proxy
node_proxy -->|"dispatches requests"| node_rotating_client
node_proxy -->|"queries metadata"| node_model_info
node_anthropic_endpoint -->|"translates format"| node_anthropic_compat
node_rotating_client -->|"delegates requests"| node_executor
node_executor -->|"applies retries"| node_retry
node_executor -->|"transforms requests"| node_transforms
node_rotating_client -->|"resolves provider"| node_provider_factory
node_provider_factory -->|"creates provider"| node_provider_plugins
node_provider_plugins -->|"calls provider"| node_provider_api
node_litellm -.->|"supports provider calls"| node_provider_api
node_rotating_client -->|"manages credentials"| node_usage_manager
node_usage_manager -->|"selects credential"| node_selection
node_selection -->|"checks availability"| node_limits
node_usage_manager -->|"records usage"| node_tracking
node_usage_manager -->|"loads and saves"| node_usage_storage
node_model_info -.->|"fetches metadata"| node_catalogs
node_model_info -->|"uses definitions"| node_model_definitions

click node_proxy "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/proxy_app/main.py"
click node_openai_endpoint "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/proxy_app/main.py"
click node_anthropic_endpoint "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/proxy_app/main.py"
click node_anthropic_compat "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/anthropic_compat/translator.py"
click node_rotating_client "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/rotating_client.py"
click node_executor "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/executor.py"
click node_retry "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/stream_retry_policy.py"
click node_transforms "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/transforms.py"
click node_provider_factory "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/provider_factory.py"
click node_provider_plugins "https://github.com/mirrowel/llm-api-key-proxy/tree/main/src/rotator_library/providers"
click node_litellm "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/litellm_providers.py"
click node_usage_manager "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/manager.py"
click node_selection "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/selection/engine.py"
click node_limits "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/limits/engine.py"
click node_tracking "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/tracking/engine.py"
click node_usage_storage "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/persistence/storage.py"
click node_model_info "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/model_info_service.py"
click node_model_definitions "https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/model_definitions.py"
click node_proxy_tools "https://github.com/mirrowel/llm-api-key-proxy/tree/main/src/proxy_app"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_proxy,node_openai_endpoint,node_anthropic_endpoint,node_proxy_tools,node_consumer toneBlue
class node_anthropic_compat,node_rotating_client,node_executor,node_retry,node_transforms toneAmber
class node_provider_factory,node_provider_plugins,node_litellm,node_provider_api toneMint
class node_usage_manager,node_selection,node_limits,node_tracking,node_usage_storage toneRose
class node_model_info,node_catalogs,node_model_definitions toneIndigo
```

## Components

### Proxy API

- [Proxy Server](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/proxy_app/main.py): `src/proxy_app/main.py`
- [OpenAI Endpoint](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/proxy_app/main.py): `src/proxy_app/main.py`
- [Anthropic Endpoints](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/proxy_app/main.py): `src/proxy_app/main.py`
- [Proxy Utilities](https://github.com/mirrowel/llm-api-key-proxy/tree/main/src/proxy_app): `src/proxy_app`

### Request Resilience

- [Anthropic Translation](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/anthropic_compat/translator.py): `src/rotator_library/anthropic_compat/translator.py`
- [Rotating Client](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/rotating_client.py): `src/rotator_library/client/rotating_client.py`
- [Request Executor](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/executor.py): `src/rotator_library/client/executor.py`
- [Stream Retry](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/stream_retry_policy.py): `src/rotator_library/client/stream_retry_policy.py`
- [Request Transforms](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/client/transforms.py): `src/rotator_library/client/transforms.py`

### Provider Integrations

- [Provider Factory](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/provider_factory.py): `src/rotator_library/provider_factory.py`
- [Provider Plugins](https://github.com/mirrowel/llm-api-key-proxy/tree/main/src/rotator_library/providers): `src/rotator_library/providers`
- [LiteLLM Providers](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/litellm_providers.py): `src/rotator_library/litellm_providers.py`
- **LLM Provider APIs**

### Usage Governance

- [Usage Manager](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/manager.py): `src/rotator_library/usage/manager.py`
- [Credential Selection](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/selection/engine.py): `src/rotator_library/usage/selection/engine.py`
- [Limit Checks](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/limits/engine.py): `src/rotator_library/usage/limits/engine.py`
- [Usage Tracking](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/tracking/engine.py): `src/rotator_library/usage/tracking/engine.py`
- [Usage State](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/usage/persistence/storage.py): `src/rotator_library/usage/persistence/storage.py`

### Model Intelligence

- [Model Registry](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/model_info_service.py): `src/rotator_library/model_info_service.py`
- **Model Catalogs**
- [Model Definitions](https://github.com/mirrowel/llm-api-key-proxy/blob/main/src/rotator_library/model_definitions.py): `src/rotator_library/model_definitions.py`

### Other components

- **API Client**

## Connections

- API Client → OpenAI Endpoint: sends chat
- API Client → Anthropic Endpoints: sends messages
- OpenAI Endpoint → Proxy Server: handled by
- Anthropic Endpoints → Proxy Server: handled by
- Proxy Server → Rotating Client: dispatches requests
- Proxy Server → Model Registry: queries metadata
- Anthropic Endpoints → Anthropic Translation: translates format
- Rotating Client → Request Executor: delegates requests
- Request Executor → Stream Retry: applies retries
- Request Executor → Request Transforms: transforms requests
- Rotating Client → Provider Factory: resolves provider
- Provider Factory → Provider Plugins: creates provider
- Provider Plugins → LLM Provider APIs: calls provider
- LiteLLM Providers → LLM Provider APIs: supports provider calls
- Rotating Client → Usage Manager: manages credentials
- Usage Manager → Credential Selection: selects credential
- Credential Selection → Limit Checks: checks availability
- Usage Manager → Usage Tracking: records usage
- Usage Manager → Usage State: loads and saves
- Model Registry → Model Catalogs: fetches metadata
- Model Registry → Model Definitions: uses definitions