How ld-reborn/ngino works
Ngino exposes local AI inference services through an outbound client tunnel: callers authenticate at the server, proxy requests are dispatched to connected clients, and clients forward them to a local inference upstream and stream responses back. The server also provides administration of keys, groups, models and connected clients, plus optional llama.cpp container management and embedding caching. The map uses README capabilities and sampled source; some server routing and admin details are documented but their implementations were not sampled, so uncertain internal links are omitted.
Main entry points: ReverseProxyEndpoint.cs HTTP proxy · TunnelClient.cs Tunnel client · AdminEndpoints.cs Admin endpoints · TunnelHub.cs Tunnel hub
Components 19
Public API
- HTTP proxy
src/Ngino.Server/ReverseProxyEndpoint.cs - Token auth
src/Ngino.Server/TokenAuthentication.cs - Embedding cache
src/Ngino.Server/EmbeddingCache.cs
Tunnel Runtime
- Tunnel hub
src/Ngino.Server/TunnelHub.cs - Tunnel connection
src/Ngino.Server/TunnelConnection.cs - Message transport
src/Ngino.Protocol/WebSocketMessageTransport.cs - Tunnel client
src/Ngino.Client/TunnelClient.cs - Client worker
src/Ngino.Client/TunnelWorker.cs
Inference Backend
- Upstream request
src/Ngino.Client/UpstreamRequest.cs - Local inference
- llama.cpp manager
src/Ngino.Client/LlamaCppManager.cs
Administration
- Admin UI
src/Ngino.Server/wwwroot/admin/app.js - Admin endpoints
src/Ngino.Server/AdminEndpoints.cs - Model keepalive
src/Ngino.Server/KeepaliveService.cs
Persistence
- Management store
src/Ngino.Server/ManagementStore.cs - Admin identity
src/Ngino.Server/Data/ApplicationDbContext.cs - Embedding SQLite
src/Ngino.Server/EmbeddingCache.cs
Other components
- API caller
- Admin operator
Connections 19
- API caller → HTTP proxy: sends HTTP
- HTTP proxy → Token auth: authorizes
- Token auth → Management store: checks keys
- HTTP proxy → Tunnel hub: dispatches requests
- HTTP proxy → Embedding cache: checks embeddings
- Tunnel hub → Tunnel connection: selects client
- Tunnel connection → Message transport: transfers messages
- Message transport → Tunnel client: transfers messages
- Client worker → Tunnel client: runs
- Tunnel client → Upstream request: forwards HTTP
- Upstream request → Local inference: sends request
- Tunnel client → llama.cpp manager: manages models
- Admin operator → Admin UI: manages service
- Admin UI → Admin endpoints: calls API
- Admin endpoints → Management store: updates management data
- Admin endpoints → Admin identity: manages accounts
- Model keepalive → Management store: reads group clients
- Model keepalive → Tunnel hub: sends model commands
- Embedding cache → Embedding SQLite: persists vectors