# bjhargrave/litellm architecture

> Architecture diagram of the GitHub repository bjhargrave/litellm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bjhargrave/litellm
- Repository: https://github.com/bjhargrave/litellm

## Overview

##### Project Type & Purpose
- LiteLLM is a full-stack LLM Gateway solution:
– A Python SDK/library (`litellm/…`) for unified LLM calls (completion, embedding, streaming, fine-tuning) across dozens of providers.
– A Proxy Server (FastAPI/Click CLI) that exposes an OpenAI‐compatible REST API to clients, adding auth, budgeting, rate limiting, routing, caching, and observability.
– Several web front-ends (Next.js admin UI, Docusaurus docs site, enterprise UI) and DevOps tooling (Docker, Helm charts, Azure templates, GitHub Actions, CircleCI).

##### File-Structure Analysis
- Core backend: `litellm/` (translation adapters, caching, router, proxy endpoints, secret managers, scheduler)
- Proxy CLI & server: `litellm/main.py`, `litellm/proxy/…`, `schema.prisma`, DB migrations
- Web UIs:
– Admin/UI Dashboard: `ui/litellm-dashboard/…` (Next.js + Tailwind)
– Enterprise UI: `enterprise/enterprise_ui/…`
– Docs site: `docs/my-website/…` (Docusaurus)
- Deployments:
– Dockerfiles (`docker/`, root `Dockerfile`, `docker-compose.yml`)
– Kubernetes/Helm (`deploy/charts/litellm-helm/…`, `deploy/kubernetes/`)
– Azure Resource Manager & Bicep (`deploy/azure_resource_manager/…`)
- CI/CD & Quality: `.circleci/`, `.github/workflows/`, `pre-commit`, `mypy`, `ruff`, `pytest`

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client Tier"
        CS1["Python SDK"]:::frontend
        CS2["Proxy CLI"]:::frontend
        CS3["OpenAI-Compatible SDK / curl"]:::frontend
        CS4["LangChain Integration"]:::frontend
    end

    subgraph "Presentation Tier"
        UI1["Admin Dashboard"]:::frontend
        UI2["Enterprise UI"]:::frontend
        UI3["Documentation Site"]:::frontend
    end

    subgraph "API Tier"
        PS["Proxy Server"]:::backend
        Auth["Auth & License Module"]:::backend
        Budget["Budget & Rate-Limit Manager"]:::backend
        Router["Routing Engine"]:::backend
        Cache["Caching Layer"]:::cache
    end

    PS --> Auth --> Budget --> Router --> Cache

    CS1 --> PS
    CS2 --> PS
    CS3 --> PS
    CS4 --> PS
    UI1 --> PS
    UI2 --> PS

    subgraph "Core Engine"
        Adapters["LLM Provider Adapters"]:::backend
        Secret["Secret Managers"]:::backend
    end

    Cache -->|cache miss| Adapters -->|calls| ExternalLLM
    Router --> Adapters
    Budget -->|checks| Secret

    subgraph "Data Tier"
        DB["PostgreSQL"]:::db
        REDIS["Redis"]:::cache
        FS["Disk/File Cache"]:::cache
        S3["S3 / Qdrant"]:::cache
    end

    PS --> DB
    Cache --> REDIS
    Cache --> FS
    Cache --> S3
    PS --> Prometheus
    PS --> OTEL

    subgraph "Observability"
        Prometheus["Prometheus"]:::cloud
        OTEL["OpenTelemetry Collector"]:::cloud
        Logs["Logging SaaS"]:::cloud
    end

    PS --> Logs
    Prometheus --> Logs

    ExternalLLM["External LLM Providers"]:::cloud

    subgraph "DevOps / CI-CD"
        CI["CircleCI"]:::infra
        GH["GitHub Actions"]:::infra
        DockerReg["Docker Registry"]:::infra
        K8s["Kubernetes Cluster"]:::infra
        Helm["Helm Charts"]:::infra
        ARM["Azure ARM/Bicep"]:::infra
        GS["Tests"]:::infra
    end

    GS --> CI
    GS --> GH
    CI --> DockerReg
    GH --> DockerReg
    DockerReg --> K8s
    Helm --> K8s
    ARM --> K8s

    click CS1 "https://github.com/bjhargrave/litellm/tree/main/litellm/"
    click PS "https://github.com/bjhargrave/litellm/blob/main/litellm/proxy/proxy_server.py"
    click Auth "https://github.com/bjhargrave/litellm/tree/main/litellm/proxy/"
    click CS2 "https://github.com/bjhargrave/litellm/blob/main/litellm/proxy/proxy_cli.py"
    click DB "https://github.com/bjhargrave/litellm/blob/main/litellm/proxy/schema.prisma"
    click Router "https://github.com/bjhargrave/litellm/tree/main/litellm/router_strategy/"
    click Adapters "https://github.com/bjhargrave/litellm/tree/main/litellm/llms/"
    click Cache "https://github.com/bjhargrave/litellm/tree/main/litellm/caching/"
    click Secret "https://github.com/bjhargrave/litellm/tree/main/litellm/secret_managers/"
    click Prometheus "https://github.com/bjhargrave/litellm/blob/main/litellm/integrations/prometheus.py"
    click OTEL "https://github.com/bjhargrave/litellm/blob/main/litellm/integrations/opentelemetry.py"
    click UI1 "https://github.com/bjhargrave/litellm/tree/main/ui/litellm-dashboard/"
    click UI2 "https://github.com/bjhargrave/litellm/tree/main/enterprise/enterprise_ui/"
    click UI3 "https://github.com/bjhargrave/litellm/tree/main/docs/my-website/"
    click DockerReg "https://github.com/bjhargrave/litellm/tree/main/docker/"
    click Helm "https://github.com/bjhargrave/litellm/tree/main/deploy/charts/litellm-helm/"
    click K8s "https://github.com/bjhargrave/litellm/tree/main/deploy/kubernetes/"
    click ARM "https://github.com/bjhargrave/litellm/tree/main/deploy/azure_resource_manager/"
    click CI "https://github.com/bjhargrave/litellm/tree/main/.circleci/"
    click GH "https://github.com/bjhargrave/litellm/tree/main/.github/workflows/"
    click GS "https://github.com/bjhargrave/litellm/tree/main/tests/"

    classDef frontend fill:#f9f,stroke:#333,stroke-width:1px
    classDef backend fill:#bbf,stroke:#333,stroke-width:1px
    classDef db fill:#bfb,stroke:#333,stroke-width:1px
    classDef cache fill:#ffb,stroke:#333,stroke-width:1px
    classDef cloud fill:#fcf,stroke:#333,stroke-width:1px
    classDef infra fill:#ffd,stroke:#333,stroke-width:1px
```
