How mudler/localai works
LocalAI is a self-hosted AI engine that serves model capabilities through compatible APIs and supports optional add-ons. The principal workflow is a user request entering the server API, being handled by the application and dispatched to model backends, then returned as a result. The README also documents model and modality breadth, hardware support, multi-user controls, chats and agents, and installable add-ons. The graph combines those documented capabilities with sampled/indexed module references; several internal dispatch details are not shown in the excerpts.
Main entry points: application.go Application · realtime_tts_pipeline.go OpenAI endpoints · engine.go Backend engines · api.js Web interface
Components 19
Entry and API
- CLI entry
cmd/local-ai/main.go - Application
core/application/application.go - OpenAI endpoints
core/http/endpoints/openai/realtime_tts_pipeline.go - Backend gRPC
pkg/grpc/server.go
Model runtime
- Model loading
core/application/application.go - Backend engines
backend/go/strata/engine.go - Distributed vLLM
core/cli/worker/worker_vllm.go - Multimodal backends
backend/cpp/audio-cpp/capability_routing.cpp
Identity and chat
- User auth and quotas
core/application/application.go - Chats and agents
core/application/application.go - MCP client
core/http/react-ui/src/hooks/useMCPClient.js
Add-on platform
- Add-on manager
core/application/application.go - Add-on host operations
core/services/addons/hostops/engine_jobs.go - Add-on workspace
core/services/addons/hostops/engine_workspace.go - Signed add-on catalog
core/application/application.go
User interfaces
- Model storage report
core/gallery/storage.go - Web interface
core/http/react-ui/src/utils/api.js - Add-on interfaces
addons/internal/connector-ui/src/ConnectorPage.jsx
Other components
- API user
Connections 17
- API user → OpenAI endpoints: sends requests
- CLI entry → Application: starts server
- Application → Model loading: creates loader
core/application/application.go - OpenAI endpoints → Backend engines: dispatches inference
- Backend engines → Multimodal backends: serves modalities
- Distributed vLLM → Backend engines: runs distributed
- Application → Backend gRPC: uses backend services
- Application → User auth and quotas: configures access
core/application/application.go - Application → Chats and agents: initializes assistant
core/application/application.go - Chats and agents → MCP client: uses MCP tools
- Application → Add-on manager: initializes add-ons
core/application/application.go - Application → Add-on host operations: sets host operations
core/application/application.go - Add-on host operations → Add-on workspace: uses workspace
core/services/addons/hostops/engine_workspace.go - Application → Signed add-on catalog: supports catalog
core/application/application.go - Web interface → Model storage report: requests storage
- Web interface → OpenAI endpoints: calls server APIs
- Add-on interfaces → Add-on manager: configures add-ons