# ld-reborn/ngino architecture

> Architecture diagram of the GitHub repository ld-reborn/ngino, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ld-reborn/ngino
- Repository: https://github.com/ld-reborn/ngino

## Overview

Ngino exposes local AI inference services through an outbound client tunnel: callers authenticate at the server, proxy requests are dispatched to connected clients, and clients forward them to a local inference upstream and stream responses back. The server also provides administration of keys, groups, models and connected clients, plus optional llama.cpp container management and embedding caching. The map uses README capabilities and sampled source; some server routing and admin details are documented but their implementations were not sampled, so uncertain internal links are omitted.

Main entry points: [`src/Ngino.Server/ReverseProxyEndpoint.cs`](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/ReverseProxyEndpoint.cs) (HTTP proxy), [`src/Ngino.Client/TunnelClient.cs`](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Client/TunnelClient.cs) (Tunnel client), [`src/Ngino.Server/AdminEndpoints.cs`](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/AdminEndpoints.cs) (Admin endpoints), [`src/Ngino.Server/TunnelHub.cs`](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/TunnelHub.cs) (Tunnel hub)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_public["Public API"]
  node_proxy["HTTP proxy"]
  node_auth["Token auth"]
  node_cache["Embedding cache<br/>[EmbeddingCache.cs]"]
end

subgraph group_tunnel["Tunnel Runtime"]
  node_hub["Tunnel hub<br/>[TunnelHub.cs]"]
  node_connection["Tunnel connection"]
  node_protocol["Message transport"]
  node_client["Tunnel client<br/>[TunnelClient.cs]"]
  node_worker["Client worker<br/>[TunnelWorker.cs]"]
end

subgraph group_inference["Inference Backend"]
  node_request["Upstream request<br/>[UpstreamRequest.cs]"]
  node_upstream["Local inference"]
  node_llamacpp["llama.cpp manager<br/>[LlamaCppManager.cs]"]
end

subgraph group_admin["Administration"]
  node_ui["Admin UI<br/>[app.js]"]
  node_adminapi["Admin endpoints<br/>[AdminEndpoints.cs]"]
  node_keepalive["Model keepalive"]
end

subgraph group_state["Persistence"]
  node_management[("Management store<br/>[ManagementStore.cs]")]
  node_identity[("Admin identity")]
  node_embeddingdb[("Embedding SQLite<br/>[EmbeddingCache.cs]")]
end

node_caller(("API caller"))
node_admin(("Admin operator"))

node_caller -->|"sends HTTP"| node_proxy
node_proxy -->|"authorizes"| node_auth
node_auth -->|"checks keys"| node_management
node_proxy -->|"dispatches requests"| node_hub
node_proxy -->|"checks embeddings"| node_cache
node_hub -->|"selects client"| node_connection
node_connection -->|"transfers messages"| node_protocol
node_protocol -->|"transfers messages"| node_client
node_worker -->|"runs"| node_client
node_client -->|"forwards HTTP"| node_request
node_request -->|"sends request"| node_upstream
node_client -.->|"manages models"| node_llamacpp
node_admin -->|"manages service"| node_ui
node_ui -->|"calls API"| node_adminapi
node_adminapi -->|"updates management data"| node_management
node_adminapi -->|"manages accounts"| node_identity
node_keepalive -->|"reads group clients"| node_management
node_keepalive -->|"sends model commands"| node_hub
node_cache -->|"persists vectors"| node_embeddingdb

click node_proxy "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/ReverseProxyEndpoint.cs"
click node_auth "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/TokenAuthentication.cs"
click node_cache "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/EmbeddingCache.cs"
click node_hub "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/TunnelHub.cs"
click node_connection "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/TunnelConnection.cs"
click node_protocol "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Protocol/WebSocketMessageTransport.cs"
click node_client "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Client/TunnelClient.cs"
click node_worker "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Client/TunnelWorker.cs"
click node_request "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Client/UpstreamRequest.cs"
click node_llamacpp "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Client/LlamaCppManager.cs"
click node_ui "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/wwwroot/admin/app.js"
click node_adminapi "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/AdminEndpoints.cs"
click node_keepalive "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/KeepaliveService.cs"
click node_management "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/ManagementStore.cs"
click node_identity "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/Data/ApplicationDbContext.cs"
click node_embeddingdb "https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/EmbeddingCache.cs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_proxy,node_auth,node_cache toneBlue
class node_hub,node_connection,node_protocol,node_client,node_worker toneAmber
class node_request,node_upstream,node_llamacpp,node_caller toneMint
class node_ui,node_adminapi,node_keepalive toneRose
class node_management,node_identity,node_embeddingdb,node_admin toneIndigo
```

## Components

### Public API

- [HTTP proxy](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/ReverseProxyEndpoint.cs): `src/Ngino.Server/ReverseProxyEndpoint.cs`
- [Token auth](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/TokenAuthentication.cs): `src/Ngino.Server/TokenAuthentication.cs`
- [Embedding cache](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/EmbeddingCache.cs): `src/Ngino.Server/EmbeddingCache.cs`

### Tunnel Runtime

- [Tunnel hub](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/TunnelHub.cs): `src/Ngino.Server/TunnelHub.cs`
- [Tunnel connection](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/TunnelConnection.cs): `src/Ngino.Server/TunnelConnection.cs`
- [Message transport](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Protocol/WebSocketMessageTransport.cs): `src/Ngino.Protocol/WebSocketMessageTransport.cs`
- [Tunnel client](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Client/TunnelClient.cs): `src/Ngino.Client/TunnelClient.cs`
- [Client worker](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Client/TunnelWorker.cs): `src/Ngino.Client/TunnelWorker.cs`

### Inference Backend

- [Upstream request](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Client/UpstreamRequest.cs): `src/Ngino.Client/UpstreamRequest.cs`
- **Local inference**
- [llama.cpp manager](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Client/LlamaCppManager.cs): `src/Ngino.Client/LlamaCppManager.cs`

### Administration

- [Admin UI](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/wwwroot/admin/app.js): `src/Ngino.Server/wwwroot/admin/app.js`
- [Admin endpoints](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/AdminEndpoints.cs): `src/Ngino.Server/AdminEndpoints.cs`
- [Model keepalive](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/KeepaliveService.cs): `src/Ngino.Server/KeepaliveService.cs`

### Persistence

- [Management store](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/ManagementStore.cs): `src/Ngino.Server/ManagementStore.cs`
- [Admin identity](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/Data/ApplicationDbContext.cs): `src/Ngino.Server/Data/ApplicationDbContext.cs`
- [Embedding SQLite](https://github.com/ld-reborn/ngino/blob/main/src/Ngino.Server/EmbeddingCache.cs): `src/Ngino.Server/EmbeddingCache.cs`

### Other components

- **API caller**
- **Admin operator**

## Connections

- API caller → HTTP proxy: sends HTTP
- HTTP proxy → Token auth: authorizes
- Token auth → Management store: checks keys
- HTTP proxy → Tunnel hub: dispatches requests
- HTTP proxy → Embedding cache: checks embeddings
- Tunnel hub → Tunnel connection: selects client
- Tunnel connection → Message transport: transfers messages
- Message transport → Tunnel client: transfers messages
- Client worker → Tunnel client: runs
- Tunnel client → Upstream request: forwards HTTP
- Upstream request → Local inference: sends request
- Tunnel client → llama.cpp manager: manages models
- Admin operator → Admin UI: manages service
- Admin UI → Admin endpoints: calls API
- Admin endpoints → Management store: updates management data
- Admin endpoints → Admin identity: manages accounts
- Model keepalive → Management store: reads group clients
- Model keepalive → Tunnel hub: sends model commands
- Embedding cache → Embedding SQLite: persists vectors