# mudler/localai architecture

> Architecture diagram of the GitHub repository mudler/localai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mudler/localai
- Repository: https://github.com/mudler/localai

## Overview

LocalAI is a self-hosted AI engine that serves model capabilities through compatible APIs and supports optional add-ons. The principal workflow is a user request entering the server API, being handled by the application and dispatched to model backends, then returned as a result. The README also documents model and modality breadth, hardware support, multi-user controls, chats and agents, and installable add-ons. The graph combines those documented capabilities with sampled/indexed module references; several internal dispatch details are not shown in the excerpts.

Main entry points: [`core/application/application.go`](https://github.com/mudler/localai/blob/master/core/application/application.go) (Application), [`core/http/endpoints/openai/realtime_tts_pipeline.go`](https://github.com/mudler/localai/blob/master/core/http/endpoints/openai/realtime_tts_pipeline.go) (OpenAI endpoints), [`backend/go/strata/engine.go`](https://github.com/mudler/localai/blob/master/backend/go/strata/engine.go) (Backend engines), [`core/http/react-ui/src/utils/api.js`](https://github.com/mudler/localai/blob/master/core/http/react-ui/src/utils/api.js) (Web interface)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry and API"]
  node_cli["CLI entry<br/>[main.go]"]
  node_application["Application<br/>[application.go]"]
  node_openai_api["OpenAI endpoints"]
  node_grpc["Backend gRPC<br/>[server.go]"]
end

subgraph group_runtime["Model runtime"]
  node_model_load["Model loading<br/>[application.go]"]
  node_backend_engine["Backend engines<br/>[engine.go]"]
  node_vllm_worker["Distributed vLLM<br/>[worker_vllm.go]"]
  node_backend_capabilities["Multimodal backends"]
end

subgraph group_governance["Identity and chat"]
  node_auth["User auth and quotas<br/>[application.go]"]
  node_chat_agents["Chats and agents<br/>[application.go]"]
  node_mcp_client["MCP client<br/>[useMCPClient.js]"]
end

subgraph group_addons["Add-on platform"]
  node_addon_manager["Add-on manager<br/>[application.go]"]
  node_addon_hostops["Add-on host operations<br/>[engine_jobs.go]"]
  node_addon_workspace["Add-on workspace"]
  node_addon_catalog["Signed add-on catalog<br/>[application.go]"]
end

subgraph group_experience["User interfaces"]
  node_model_storage[("Model storage report<br/>[storage.go]")]
  node_web_ui["Web interface<br/>[api.js]"]
  node_addon_ui["Add-on interfaces<br/>[ConnectorPage.jsx]"]
end

node_user(("API user"))

node_user -->|"sends requests"| node_openai_api
node_cli -.->|"starts server"| node_application
node_application -->|"creates loader"| node_model_load
node_openai_api -.->|"dispatches inference"| node_backend_engine
node_backend_engine -.->|"serves modalities"| node_backend_capabilities
node_vllm_worker -->|"runs distributed"| node_backend_engine
node_application -.->|"uses backend services"| node_grpc
node_application -->|"configures access"| node_auth
node_application -->|"initializes assistant"| node_chat_agents
node_chat_agents -.->|"uses MCP tools"| node_mcp_client
node_application -->|"initializes add-ons"| node_addon_manager
node_application -->|"sets host operations"| node_addon_hostops
node_addon_hostops -->|"uses workspace"| node_addon_workspace
node_application -->|"supports catalog"| node_addon_catalog
node_web_ui -->|"requests storage"| node_model_storage
node_web_ui -.->|"calls server APIs"| node_openai_api
node_addon_ui -.->|"configures add-ons"| node_addon_manager

click node_cli "https://github.com/mudler/localai/blob/master/cmd/local-ai/main.go"
click node_application "https://github.com/mudler/localai/blob/master/core/application/application.go"
click node_openai_api "https://github.com/mudler/localai/blob/master/core/http/endpoints/openai/realtime_tts_pipeline.go"
click node_grpc "https://github.com/mudler/localai/blob/master/pkg/grpc/server.go"
click node_model_load "https://github.com/mudler/localai/blob/master/core/application/application.go"
click node_backend_engine "https://github.com/mudler/localai/blob/master/backend/go/strata/engine.go"
click node_vllm_worker "https://github.com/mudler/localai/blob/master/core/cli/worker/worker_vllm.go"
click node_backend_capabilities "https://github.com/mudler/localai/blob/master/backend/cpp/audio-cpp/capability_routing.cpp"
click node_auth "https://github.com/mudler/localai/blob/master/core/application/application.go"
click node_chat_agents "https://github.com/mudler/localai/blob/master/core/application/application.go"
click node_mcp_client "https://github.com/mudler/localai/blob/master/core/http/react-ui/src/hooks/useMCPClient.js"
click node_addon_manager "https://github.com/mudler/localai/blob/master/core/application/application.go"
click node_addon_hostops "https://github.com/mudler/localai/blob/master/core/services/addons/hostops/engine_jobs.go"
click node_addon_workspace "https://github.com/mudler/localai/blob/master/core/services/addons/hostops/engine_workspace.go"
click node_addon_catalog "https://github.com/mudler/localai/blob/master/core/application/application.go"
click node_model_storage "https://github.com/mudler/localai/blob/master/core/gallery/storage.go"
click node_web_ui "https://github.com/mudler/localai/blob/master/core/http/react-ui/src/utils/api.js"
click node_addon_ui "https://github.com/mudler/localai/blob/master/addons/internal/connector-ui/src/ConnectorPage.jsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_application,node_openai_api,node_grpc,node_user toneBlue
class node_model_load,node_backend_engine,node_vllm_worker,node_backend_capabilities toneAmber
class node_auth,node_chat_agents,node_mcp_client toneMint
class node_addon_manager,node_addon_hostops,node_addon_workspace,node_addon_catalog toneRose
class node_model_storage,node_web_ui,node_addon_ui toneIndigo
```

## Components

### Entry and API

- [CLI entry](https://github.com/mudler/localai/blob/master/cmd/local-ai/main.go): `cmd/local-ai/main.go`
- [Application](https://github.com/mudler/localai/blob/master/core/application/application.go): `core/application/application.go`
- [OpenAI endpoints](https://github.com/mudler/localai/blob/master/core/http/endpoints/openai/realtime_tts_pipeline.go): `core/http/endpoints/openai/realtime_tts_pipeline.go`
- [Backend gRPC](https://github.com/mudler/localai/blob/master/pkg/grpc/server.go): `pkg/grpc/server.go`

### Model runtime

- [Model loading](https://github.com/mudler/localai/blob/master/core/application/application.go): `core/application/application.go`
- [Backend engines](https://github.com/mudler/localai/blob/master/backend/go/strata/engine.go): `backend/go/strata/engine.go`
- [Distributed vLLM](https://github.com/mudler/localai/blob/master/core/cli/worker/worker_vllm.go): `core/cli/worker/worker_vllm.go`
- [Multimodal backends](https://github.com/mudler/localai/blob/master/backend/cpp/audio-cpp/capability_routing.cpp): `backend/cpp/audio-cpp/capability_routing.cpp`

### Identity and chat

- [User auth and quotas](https://github.com/mudler/localai/blob/master/core/application/application.go): `core/application/application.go`
- [Chats and agents](https://github.com/mudler/localai/blob/master/core/application/application.go): `core/application/application.go`
- [MCP client](https://github.com/mudler/localai/blob/master/core/http/react-ui/src/hooks/useMCPClient.js): `core/http/react-ui/src/hooks/useMCPClient.js`

### Add-on platform

- [Add-on manager](https://github.com/mudler/localai/blob/master/core/application/application.go): `core/application/application.go`
- [Add-on host operations](https://github.com/mudler/localai/blob/master/core/services/addons/hostops/engine_jobs.go): `core/services/addons/hostops/engine_jobs.go`
- [Add-on workspace](https://github.com/mudler/localai/blob/master/core/services/addons/hostops/engine_workspace.go): `core/services/addons/hostops/engine_workspace.go`
- [Signed add-on catalog](https://github.com/mudler/localai/blob/master/core/application/application.go): `core/application/application.go`

### User interfaces

- [Model storage report](https://github.com/mudler/localai/blob/master/core/gallery/storage.go): `core/gallery/storage.go`
- [Web interface](https://github.com/mudler/localai/blob/master/core/http/react-ui/src/utils/api.js): `core/http/react-ui/src/utils/api.js`
- [Add-on interfaces](https://github.com/mudler/localai/blob/master/addons/internal/connector-ui/src/ConnectorPage.jsx): `addons/internal/connector-ui/src/ConnectorPage.jsx`

### Other components

- **API user**

## Connections

- API user → OpenAI endpoints: sends requests
- CLI entry → Application: starts server
- Application → Model loading: creates loader (evidence: `core/application/application.go`)
- OpenAI endpoints → Backend engines: dispatches inference
- Backend engines → Multimodal backends: serves modalities
- Distributed vLLM → Backend engines: runs distributed
- Application → Backend gRPC: uses backend services
- Application → User auth and quotas: configures access (evidence: `core/application/application.go`)
- Application → Chats and agents: initializes assistant (evidence: `core/application/application.go`)
- Chats and agents → MCP client: uses MCP tools
- Application → Add-on manager: initializes add-ons (evidence: `core/application/application.go`)
- Application → Add-on host operations: sets host operations (evidence: `core/application/application.go`)
- Add-on host operations → Add-on workspace: uses workspace (evidence: `core/services/addons/hostops/engine_workspace.go`)
- Application → Signed add-on catalog: supports catalog (evidence: `core/application/application.go`)
- Web interface → Model storage report: requests storage
- Web interface → OpenAI endpoints: calls server APIs
- Add-on interfaces → Add-on manager: configures add-ons
