# llamastack/llama-stack architecture

> Architecture diagram of the GitHub repository llamastack/llama-stack, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/llamastack/llama-stack
- Repository: https://github.com/llamastack/llama-stack

## Overview

OGX is a model-agnostic, OpenAI-compatible agentic API server. A client request enters the server and API routes, then inference or Responses handling can use configured providers; the product also exposes document/vector search, batches, skills, and native Anthropic and Google APIs. The graph includes the server’s stack and provider layer, plus the CLI and web UI as distinct product surfaces. Unexcerpted subsystems are included where the README documents them; their internal call paths are not assumed.

Main entry points: [`src/ogx/core/server/server.py`](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/server/server.py) (HTTP server), [`src/ogx/core/stack.py`](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/stack.py) (Stack assembly), [`src/ogx_api/responses/fastapi_routes.py`](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/responses/fastapi_routes.py) (Responses API), [`src/ogx_api/inference/fastapi_routes.py`](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/inference/fastapi_routes.py) (API routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Interfaces"]
  node_server["HTTP server<br/>[server.py]"]
  node_cli["OGX CLI<br/>[ogx.py]"]
  node_webui["Web UI<br/>[page.tsx]"]
  node_corelibrary["Python client library<br/>[library_client.py]"]
end

subgraph group_api["API surface"]
  node_routes["API routes<br/>[fastapi_routes.py]"]
  node_responses["Responses API<br/>[fastapi_routes.py]"]
  node_messages["Messages API<br/>[fastapi_routes.py]"]
  node_vectorapi["Vector Stores API<br/>[fastapi_routes.py]"]
  node_files["Files and processors<br/>[fastapi_routes.py]"]
  node_batches["Batch API<br/>[fastapi_routes.py]"]
  node_skills["Skills API<br/>[fastapi_routes.py]"]
  node_conversations["Conversations API<br/>[fastapi_routes.py]"]
  node_schemas["API schemas<br/>[models.py]"]
  node_commonresponses["Response schemas<br/>[responses.py]"]
  node_openairesponses["OpenAI response logic"]
  node_routerutils["Route utilities<br/>[router_utils.py]"]
  node_version["API versions<br/>[version.py]"]
end

subgraph group_runtime["Runtime core"]
  node_stack["Stack assembly<br/>[stack.py]"]
  node_resolver["Provider resolver<br/>[resolver.py]"]
  node_distribution["Distribution config<br/>[distribution.py]"]
  node_inspect["Health and inspection<br/>[inspect.py]"]
  node_requestheaders["Request context<br/>[request_headers.py]"]
  node_routerregistry["Router registry"]
  node_datatypes["Core data types<br/>[datatypes.py]"]
  node_logging["Logging<br/>[log.py]"]
end

subgraph group_providers["Providers and data"]
  node_providerimpls["Provider implementations<br/>[openai.py]"]
  node_inlineproviders["Inline providers<br/>[impl.py]"]
  node_vectorproviders["Vector backends<br/>[chroma.py]"]
  node_fileproviders["File processors<br/>[docling.py]"]
end

subgraph group_state["State and jobs"]
  node_jobs["Job processing<br/>[worker.py]"]
  node_storage[("Storage backends<br/>[kvstore.py]")]
end

node_client(("API clients"))

node_client -->|"sends API requests"| node_server
node_server -->|"serves API routes"| node_routes
node_server -->|"serves Responses API"| node_responses
node_server -->|"serves Messages API"| node_messages
node_server -->|"serves vector APIs"| node_vectorapi
node_server -->|"serves file APIs"| node_files
node_server -->|"serves batch API"| node_batches
node_server -->|"serves skills API"| node_skills
node_server -.->|"serves conversations"| node_conversations
node_routes -.->|"dispatches inference"| node_providerimpls
node_responses -->|"uses response logic"| node_openairesponses
node_responses -->|"returns schemas"| node_commonresponses
node_vectorapi -.->|"searches vector stores"| node_vectorproviders
node_files -.->|"processes files"| node_fileproviders
node_messages -.->|"dispatches messages"| node_providerimpls
node_routes -->|"accepts and returns"| node_schemas
node_routes -->|"uses route utilities"| node_routerutils
node_responses -->|"uses route utilities"| node_routerutils
node_vectorapi -->|"uses route utilities"| node_routerutils
node_messages -->|"uses route utilities"| node_routerutils
node_server -->|"configures registry"| node_routerregistry
node_stack -->|"assembles configuration"| node_distribution
node_stack -->|"resolves providers"| node_resolver
node_stack -->|"initializes inspection"| node_inspect
node_inspect -->|"lists registered routes"| node_routerregistry
node_server -->|"loads runtime stack"| node_stack
node_stack -->|"uses core types"| node_datatypes
node_server -->|"handles request context"| node_requestheaders
node_server -->|"logs server activity"| node_logging
node_stack -->|"initializes job runtime"| node_jobs
node_responses -.->|"supports asynchronous work"| node_jobs
node_vectorapi -.->|"persists vector data"| node_storage
node_providerimpls -.->|"calls model services"| node_client
node_inlineproviders -.->|"supports local inference"| node_providerimpls
node_webui -.->|"calls API"| node_server
node_cli -.->|"connects to endpoint"| node_server
node_corelibrary -.->|"connects to server"| node_server

click node_server "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/server/server.py"
click node_cli "https://github.com/llamastack/llama-stack/blob/main/src/ogx/cli/ogx.py"
click node_webui "https://github.com/llamastack/llama-stack/blob/main/src/ogx_ui/app/chat-playground/page.tsx"
click node_routes "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/inference/fastapi_routes.py"
click node_responses "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/responses/fastapi_routes.py"
click node_messages "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/messages/fastapi_routes.py"
click node_vectorapi "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/vector_io/fastapi_routes.py"
click node_files "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/file_processors/fastapi_routes.py"
click node_batches "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/batches/fastapi_routes.py"
click node_skills "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/skills/fastapi_routes.py"
click node_conversations "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/conversations/fastapi_routes.py"
click node_schemas "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/inference/models.py"
click node_commonresponses "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/common/responses.py"
click node_openairesponses "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/openai_responses.py"
click node_routerutils "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/router_utils.py"
click node_version "https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/version.py"
click node_stack "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/stack.py"
click node_resolver "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/resolver.py"
click node_distribution "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/distribution.py"
click node_inspect "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/inspect.py"
click node_requestheaders "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/request_headers.py"
click node_routerregistry "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/server/fastapi_router_registry.py"
click node_datatypes "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/datatypes.py"
click node_logging "https://github.com/llamastack/llama-stack/blob/main/src/ogx/log.py"
click node_providerimpls "https://github.com/llamastack/llama-stack/blob/main/src/ogx/providers/remote/inference/openai/openai.py"
click node_inlineproviders "https://github.com/llamastack/llama-stack/blob/main/src/ogx/providers/inline/responses/builtin/impl.py"
click node_vectorproviders "https://github.com/llamastack/llama-stack/blob/main/src/ogx/providers/remote/vector_io/chroma/chroma.py"
click node_fileproviders "https://github.com/llamastack/llama-stack/blob/main/src/ogx/providers/inline/file_processor/docling/docling.py"
click node_jobs "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/jobs/worker.py"
click node_storage "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/storage/kvstore/kvstore.py"
click node_corelibrary "https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/library_client.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_server,node_cli,node_webui,node_corelibrary,node_client toneBlue
class node_routes,node_responses,node_messages,node_vectorapi,node_files,node_batches,node_skills,node_conversations,node_schemas,node_commonresponses,node_openairesponses,node_routerutils,node_version toneAmber
class node_stack,node_resolver,node_distribution,node_inspect,node_requestheaders,node_routerregistry,node_datatypes,node_logging toneMint
class node_providerimpls,node_inlineproviders,node_vectorproviders,node_fileproviders toneRose
class node_jobs,node_storage toneIndigo
```

## Components

### Interfaces

- [HTTP server](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/server/server.py): `src/ogx/core/server/server.py`
- [OGX CLI](https://github.com/llamastack/llama-stack/blob/main/src/ogx/cli/ogx.py): `src/ogx/cli/ogx.py`
- [Web UI](https://github.com/llamastack/llama-stack/blob/main/src/ogx_ui/app/chat-playground/page.tsx): `src/ogx_ui/app/chat-playground/page.tsx`
- [Python client library](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/library_client.py): `src/ogx/core/library_client.py`

### API surface

- [API routes](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/inference/fastapi_routes.py): `src/ogx_api/inference/fastapi_routes.py`
- [Responses API](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/responses/fastapi_routes.py): `src/ogx_api/responses/fastapi_routes.py`
- [Messages API](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/messages/fastapi_routes.py): `src/ogx_api/messages/fastapi_routes.py`
- [Vector Stores API](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/vector_io/fastapi_routes.py): `src/ogx_api/vector_io/fastapi_routes.py`
- [Files and processors](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/file_processors/fastapi_routes.py): `src/ogx_api/file_processors/fastapi_routes.py`
- [Batch API](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/batches/fastapi_routes.py): `src/ogx_api/batches/fastapi_routes.py`
- [Skills API](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/skills/fastapi_routes.py): `src/ogx_api/skills/fastapi_routes.py`
- [Conversations API](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/conversations/fastapi_routes.py): `src/ogx_api/conversations/fastapi_routes.py`
- [API schemas](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/inference/models.py): `src/ogx_api/inference/models.py`
- [Response schemas](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/common/responses.py): `src/ogx_api/common/responses.py`
- [OpenAI response logic](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/openai_responses.py): `src/ogx_api/openai_responses.py`
- [Route utilities](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/router_utils.py): `src/ogx_api/router_utils.py`
- [API versions](https://github.com/llamastack/llama-stack/blob/main/src/ogx_api/version.py): `src/ogx_api/version.py`

### Runtime core

- [Stack assembly](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/stack.py): `src/ogx/core/stack.py`
- [Provider resolver](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/resolver.py): `src/ogx/core/resolver.py`
- [Distribution config](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/distribution.py): `src/ogx/core/distribution.py`
- [Health and inspection](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/inspect.py): `src/ogx/core/inspect.py`
- [Request context](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/request_headers.py): `src/ogx/core/request_headers.py`
- [Router registry](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/server/fastapi_router_registry.py): `src/ogx/core/server/fastapi_router_registry.py`
- [Core data types](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/datatypes.py): `src/ogx/core/datatypes.py`
- [Logging](https://github.com/llamastack/llama-stack/blob/main/src/ogx/log.py): `src/ogx/log.py`

### Providers and data

- [Provider implementations](https://github.com/llamastack/llama-stack/blob/main/src/ogx/providers/remote/inference/openai/openai.py): `src/ogx/providers/remote/inference/openai/openai.py`
- [Inline providers](https://github.com/llamastack/llama-stack/blob/main/src/ogx/providers/inline/responses/builtin/impl.py): `src/ogx/providers/inline/responses/builtin/impl.py`
- [Vector backends](https://github.com/llamastack/llama-stack/blob/main/src/ogx/providers/remote/vector_io/chroma/chroma.py): `src/ogx/providers/remote/vector_io/chroma/chroma.py`
- [File processors](https://github.com/llamastack/llama-stack/blob/main/src/ogx/providers/inline/file_processor/docling/docling.py): `src/ogx/providers/inline/file_processor/docling/docling.py`

### State and jobs

- [Job processing](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/jobs/worker.py): `src/ogx/core/jobs/worker.py`
- [Storage backends](https://github.com/llamastack/llama-stack/blob/main/src/ogx/core/storage/kvstore/kvstore.py): `src/ogx/core/storage/kvstore/kvstore.py`

### Other components

- **API clients**

## Connections

- API clients → HTTP server: sends API requests (evidence: `README.md`)
- HTTP server → API routes: serves API routes (evidence: `README.md`)
- HTTP server → Responses API: serves Responses API (evidence: `README.md`)
- HTTP server → Messages API: serves Messages API (evidence: `README.md`)
- HTTP server → Vector Stores API: serves vector APIs (evidence: `README.md`)
- HTTP server → Files and processors: serves file APIs (evidence: `README.md`)
- HTTP server → Batch API: serves batch API (evidence: `README.md`)
- HTTP server → Skills API: serves skills API (evidence: `README.md`)
- HTTP server → Conversations API: serves conversations
- API routes → Provider implementations: dispatches inference
- Responses API → OpenAI response logic: uses response logic (evidence: `src/ogx_api/responses/fastapi_routes.py`)
- Responses API → Response schemas: returns schemas (evidence: `src/ogx_api/responses/fastapi_routes.py`)
- Vector Stores API → Vector backends: searches vector stores
- Files and processors → File processors: processes files
- Messages API → Provider implementations: dispatches messages
- API routes → API schemas: accepts and returns (evidence: `src/ogx_api/inference/fastapi_routes.py`)
- API routes → Route utilities: uses route utilities (evidence: `src/ogx_api/inference/fastapi_routes.py`)
- Responses API → Route utilities: uses route utilities (evidence: `src/ogx_api/responses/fastapi_routes.py`)
- Vector Stores API → Route utilities: uses route utilities (evidence: `src/ogx_api/vector_io/fastapi_routes.py`)
- Messages API → Route utilities: uses route utilities (evidence: `src/ogx_api/messages/fastapi_routes.py`)
- HTTP server → Router registry: configures registry (evidence: `src/ogx/core/server/server.py`)
- Stack assembly → Distribution config: assembles configuration (evidence: `src/ogx/core/stack.py`)
- Stack assembly → Provider resolver: resolves providers (evidence: `src/ogx/core/stack.py`)
- Stack assembly → Health and inspection: initializes inspection (evidence: `src/ogx/core/stack.py`)
- Health and inspection → Router registry: lists registered routes (evidence: `src/ogx/core/inspect.py`)
- HTTP server → Stack assembly: loads runtime stack (evidence: `src/ogx/core/server/server.py`)
- Stack assembly → Core data types: uses core types (evidence: `src/ogx/core/stack.py`)
- HTTP server → Request context: handles request context (evidence: `src/ogx/core/server/server.py`)
- HTTP server → Logging: logs server activity (evidence: `src/ogx/core/server/server.py`)
- Stack assembly → Job processing: initializes job runtime (evidence: `src/ogx/core/stack.py`)
- Responses API → Job processing: supports asynchronous work
- Vector Stores API → Storage backends: persists vector data
- Provider implementations → API clients: calls model services
- Inline providers → Provider implementations: supports local inference
- Web UI → HTTP server: calls API
- OGX CLI → HTTP server: connects to endpoint
- Python client library → HTTP server: connects to server
