How bjhargrave/litellm works
Project Type & Purpose
• LiteLLM is a full-stack LLM Gateway solution:
– A Python SDK/library (litellm/…) for unified LLM calls (completion, embedding, streaming, fine-tuning) across dozens of providers.
– A Proxy Server (FastAPI/Click CLI) that exposes an OpenAI‐compatible REST API to clients, adding auth, budgeting, rate limiting, routing, caching, and observability.
– Several web front-ends (Next.js admin UI, Docusaurus docs site, enterprise UI) and DevOps tooling (Docker, Helm charts, Azure templates, GitHub Actions, CircleCI).
File-Structure Analysis
• Core backend: litellm/ (translation adapters, caching, router, proxy endpoints, secret managers, scheduler)
• Proxy CLI & server: litellm/main.py, litellm/proxy/…, schema.prisma, DB migrations
• Web UIs:
– Admin/UI Dashboard: ui/litellm-dashboard/… (Next.js + Tailwind)
– Enterprise UI: enterprise/enterprise_ui/…
– Docs site: docs/my-website/… (Docusaurus)
• Deployments:
– Dockerfiles (docker/, root Dockerfile, docker-compose.yml)
– Kubernetes/Helm (deploy/charts/litellm-helm/…, deploy/kubernetes/)
– Azure Resource Manager & Bicep (deploy/azure_resource_manager/…)
• CI/CD & Quality: .circleci/, .github/workflows/, pre-commit, mypy, ruff, pytest