How llamastack/llama-stack works
OGX is a model-agnostic, OpenAI-compatible agentic API server. A client request enters the server and API routes, then inference or Responses handling can use configured providers; the product also exposes document/vector search, batches, skills, and native Anthropic and Google APIs. The graph includes the server’s stack and provider layer, plus the CLI and web UI as distinct product surfaces. Unexcerpted subsystems are included where the README documents them; their internal call paths are not assumed.
Main entry points: server.py HTTP server · stack.py Stack assembly · responses/fastapi_routes.py Responses API · inference/fastapi_routes.py API routes
Components 32
Interfaces
- HTTP server
src/ogx/core/server/server.py - OGX CLI
src/ogx/cli/ogx.py - Web UI
src/ogx_ui/app/chat-playground/page.tsx - Python client library
src/ogx/core/library_client.py
API surface
- API routes
src/ogx_api/inference/fastapi_routes.py - Responses API
src/ogx_api/responses/fastapi_routes.py - Messages API
src/ogx_api/messages/fastapi_routes.py - Vector Stores API
src/ogx_api/vector_io/fastapi_routes.py - Files and processors
src/ogx_api/file_processors/fastapi_routes.py - Batch API
src/ogx_api/batches/fastapi_routes.py - Skills API
src/ogx_api/skills/fastapi_routes.py - Conversations API
src/ogx_api/conversations/fastapi_routes.py - API schemas
src/ogx_api/inference/models.py - Response schemas
src/ogx_api/common/responses.py - OpenAI response logic
src/ogx_api/openai_responses.py - Route utilities
src/ogx_api/router_utils.py - API versions
src/ogx_api/version.py
Runtime core
- Stack assembly
src/ogx/core/stack.py - Provider resolver
src/ogx/core/resolver.py - Distribution config
src/ogx/core/distribution.py - Health and inspection
src/ogx/core/inspect.py - Request context
src/ogx/core/request_headers.py - Router registry
src/ogx/core/server/fastapi_router_registry.py - Core data types
src/ogx/core/datatypes.py - Logging
src/ogx/log.py
Providers and data
- Provider implementations
src/ogx/providers/remote/inference/openai/openai.py - Inline providers
src/ogx/providers/inline/responses/builtin/impl.py - Vector backends
src/ogx/providers/remote/vector_io/chroma/chroma.py - File processors
src/ogx/providers/inline/file_processor/docling/docling.py
State and jobs
- Job processing
src/ogx/core/jobs/worker.py - Storage backends
src/ogx/core/storage/kvstore/kvstore.py
Other components
- API clients
Connections 37
- API clients → HTTP server: sends API requests
README.md - HTTP server → API routes: serves API routes
README.md - HTTP server → Responses API: serves Responses API
README.md - HTTP server → Messages API: serves Messages API
README.md - HTTP server → Vector Stores API: serves vector APIs
README.md - HTTP server → Files and processors: serves file APIs
README.md - HTTP server → Batch API: serves batch API
README.md - HTTP server → Skills API: serves skills API
README.md - HTTP server → Conversations API: serves conversations
- API routes → Provider implementations: dispatches inference
- Responses API → OpenAI response logic: uses response logic
src/ogx_api/responses/fastapi_routes.py - Responses API → Response schemas: returns schemas
src/ogx_api/responses/fastapi_routes.py - Vector Stores API → Vector backends: searches vector stores
- Files and processors → File processors: processes files
- Messages API → Provider implementations: dispatches messages
- API routes → API schemas: accepts and returns
src/ogx_api/inference/fastapi_routes.py - API routes → Route utilities: uses route utilities
src/ogx_api/inference/fastapi_routes.py - Responses API → Route utilities: uses route utilities
src/ogx_api/responses/fastapi_routes.py - Vector Stores API → Route utilities: uses route utilities
src/ogx_api/vector_io/fastapi_routes.py - Messages API → Route utilities: uses route utilities
src/ogx_api/messages/fastapi_routes.py - HTTP server → Router registry: configures registry
src/ogx/core/server/server.py - Stack assembly → Distribution config: assembles configuration
src/ogx/core/stack.py - Stack assembly → Provider resolver: resolves providers
src/ogx/core/stack.py - Stack assembly → Health and inspection: initializes inspection
src/ogx/core/stack.py - Health and inspection → Router registry: lists registered routes
src/ogx/core/inspect.py - HTTP server → Stack assembly: loads runtime stack
src/ogx/core/server/server.py - Stack assembly → Core data types: uses core types
src/ogx/core/stack.py - HTTP server → Request context: handles request context
src/ogx/core/server/server.py - HTTP server → Logging: logs server activity
src/ogx/core/server/server.py - Stack assembly → Job processing: initializes job runtime
src/ogx/core/stack.py - Responses API → Job processing: supports asynchronous work
- Vector Stores API → Storage backends: persists vector data
- Provider implementations → API clients: calls model services
- Inline providers → Provider implementations: supports local inference
- Web UI → HTTP server: calls API
- OGX CLI → HTTP server: connects to endpoint
- Python client library → HTTP server: connects to server