How lemonade-sdk/lemonade works
Lemonade is a local AI inference server with a desktop management and experimentation app, plus CLI and standard API clients. The principal workflow is selecting or downloading a model, preparing/loading it, sending a chat or multimodal request, and receiving generated output. The graph preserves model/backend management, custom collections and routing, and inference modalities. Runtime ownership is only partially visible in the supplied excerpts; unverified server-internal links are dashed rather than inferred as calls.
Main entry points: App.tsx Desktop app · server.cpp Lemonade server · ModelManager.tsx Model manager · router.cpp Inference router
Components 24
Clients and desktop
- Desktop app
src/app/src/renderer/App.tsx - Chat interface
src/app/src/renderer/components/panels/LLMChatPanel.tsx - Model manager
src/app/src/renderer/ModelManager.tsx - Backend manager
src/app/src/renderer/BackendManager.tsx - Download manager
src/app/src/renderer/DownloadManager.tsx - CLI
src/cpp/cli/main.cpp - Server connection
src/app/src/renderer/utils/serverConfig.ts
Models and backends
- Model catalog data
src/app/src/renderer/utils/modelData.ts - Model and system hooks
src/app/src/renderer/hooks/useModels.tsx - Model preparation
src/app/src/renderer/utils/backendInstaller.ts - Download tracking
src/app/src/renderer/utils/downloadTracker.ts
Inference and routing
- Collections and rules
src/app/src/renderer/utils/customCollections.ts - Router rule model
src/app/src/renderer/utils/routerTree.ts - Router editor
src/app/src/renderer/components/RouterCollectionPanel.tsx - Multimodal panels
src/app/src/renderer/components/panels
Server runtime
- Lemonade server
src/cpp/server/server.cpp - Model registry
src/cpp/server/model_manager.cpp - Backend orchestration
src/cpp/server/backend_manager.cpp - Inference router
src/cpp/server/router.cpp - VRAM eviction
src/cpp/server/eviction_engine.cpp
External interfaces
- API-compatible apps
- Local accelerators
- Cloud providers
Other components
- User
Connections 26
- User → Desktop app: uses
README.md - User → CLI: runs commands
src/cpp/cli/main.cpp - Desktop app → Chat interface: opens
- Desktop app → Model manager: manages models
- Desktop app → Backend manager: manages backends
- Desktop app → Download manager: tracks downloads
- Desktop app → Router editor: configures routing
src/app/src/renderer/components/RouterCollectionPanel.tsx - Desktop app → Multimodal panels: offers generation
README.md - Model manager → Model catalog data: reads catalog
src/app/src/renderer/ModelManager.tsx - Model manager → Model preparation: prepares models
src/app/src/renderer/ModelManager.tsx - Backend manager → Model and system hooks: reads system state
src/app/src/renderer/BackendManager.tsx - Download manager → Download tracking: tracks progress
src/app/src/renderer/DownloadManager.tsx - Model and system hooks → Model catalog data: loads catalog
src/app/src/renderer/hooks/useModels.tsx - Router editor → Collections and rules: edits collections
src/app/src/renderer/components/RouterCollectionPanel.tsx - Router editor → Router rule model: edits rules
src/app/src/renderer/components/RouterCollectionPanel.tsx - Chat interface → Server connection: sends requests
src/app/src/renderer/components/panels/LLMChatPanel.tsx - Model manager → Server connection: queries server
src/app/src/renderer/ModelManager.tsx - Server connection → Lemonade server: connects API
- CLI → Lemonade server: calls HTTP API
src/cpp/cli/main.cpp - API-compatible apps → Lemonade server: uses compatible APIs
README.md - Lemonade server → Model registry: manages models
- Lemonade server → Backend orchestration: coordinates backends
- Lemonade server → Inference router: dispatches inference
- Inference router → Local accelerators: runs models
- Inference router → Cloud providers: offloads inference
- VRAM eviction → Inference router: evicts models
src/cpp/server/eviction_engine.cpp