# zhongkaifu/tensorsharp architecture

> Architecture diagram of the GitHub repository zhongkaifu/tensorsharp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zhongkaifu/tensorsharp
- Repository: https://github.com/zhongkaifu/tensorsharp
- Video tour (about a minute): https://gitdiagram.com/zhongkaifu/tensorsharp/video

## Overview

TensorSharp is a local .NET inference engine and serving product: a user submits text or multimodal input through a CLI, browser UI, or compatible HTTP API; chat generation renders and tokenizes the request, then the inference runtime schedules model execution and returns generated output. Separate product branches cover image/video generation, embeddings, model acquisition, and optional agent workflows. The README documents these capabilities, but the sampled source is incomplete, so links to unsampled subsystems are shown only where documentation supports them.

Main entry points: [`TensorSharp.Models/Models`](https://github.com/zhongkaifu/tensorsharp/tree/main/TensorSharp.Models/Models) (Model families), [`TensorAgent/src/TensorAgent.Core/Hosting/AgentAppHost.cs`](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/src/TensorAgent.Core/Hosting/AgentAppHost.cs) (Agent host), [`TensorSharp.Runtime/Scheduling/InferenceEngine.cs`](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Runtime/Scheduling/InferenceEngine.cs) (Inference engine), [`TensorSharp.Chat/ChatGenerationPipeline.cs`](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Chat/ChatGenerationPipeline.cs) (Chat generation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_serving["Serving and chat"]
  node_interfaces["CLI, UI, APIs"]
  node_chat["Chat generation"]
  node_embeddings["Embedding APIs"]
  node_catalog["Model catalog<br/>[ModelCatalog.cs]"]
  node_modelstore[("Model storage<br/>[ModelStore.cs]")]
  node_downloads["Model downloads"]
end

subgraph group_runtime["Inference runtime"]
  node_engine["Inference engine<br/>[InferenceEngine.cs]"]
  node_scheduler["Request scheduling<br/>[Scheduling]"]
  node_kv[("KV cache<br/>[KvCacheStorage.cs]")]
  node_models["Model families<br/>[Models]"]
  node_ggml["GGML backend<br/>[GgmlStorage.cs]"]
  node_cuda["CUDA backend<br/>[Dsv4CudaEngine.cs]"]
  node_mlx["MLX backend<br/>[MlxStorage.cs]"]
end

subgraph group_media["Media generation"]
  node_minimax["MiniMax video"]
  node_wan["Wan video"]
  node_qwenimage["Qwen image<br/>[Models]"]
end

subgraph group_agent["Agent workflows"]
  node_agenthost["Agent host<br/>[AgentAppHost.cs]"]
  node_skills["Skills and tools"]
  node_toolwork["Sandboxed work"]
end

node_user(("User"))

node_user -->|"submits input"| node_interfaces
node_interfaces -->|"requests generation"| node_chat
node_interfaces -->|"requests embeddings"| node_embeddings
node_chat -->|"submits inference"| node_engine
node_engine -->|"schedules requests"| node_scheduler
node_engine -->|"executes model"| node_models
node_models -->|"uses cache"| node_kv
node_models -.->|"runs on backend"| node_ggml
node_models -.->|"runs on backend"| node_cuda
node_models -.->|"runs on backend"| node_mlx
node_interfaces -->|"requests video"| node_minimax
node_interfaces -->|"requests video"| node_wan
node_interfaces -->|"requests image"| node_qwenimage
node_user -->|"selects model"| node_catalog
node_catalog -->|"uses catalog store"| node_modelstore
node_downloads -->|"stores downloads"| node_modelstore
node_user -.->|"starts agent work"| node_agenthost
node_agenthost -.->|"requests model output"| node_chat
node_agenthost -.->|"loads skills"| node_skills
node_agenthost -.->|"dispatches tools"| node_toolwork

click node_chat "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Chat/ChatGenerationPipeline.cs"
click node_catalog "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/src/TensorAgent.Core/Catalog/ModelCatalog.cs"
click node_modelstore "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/src/TensorAgent.Core/Catalog/ModelStore.cs"
click node_downloads "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/src/TensorAgent.Core/Downloads/ModelDownloadManager.cs"
click node_engine "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Runtime/Scheduling/InferenceEngine.cs"
click node_scheduler "https://github.com/zhongkaifu/tensorsharp/tree/main/TensorSharp.Runtime/Scheduling"
click node_kv "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Models/KvCacheStorage.cs"
click node_models "https://github.com/zhongkaifu/tensorsharp/tree/main/TensorSharp.Models/Models"
click node_ggml "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Backends.GGML/GgmlStorage.cs"
click node_cuda "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Backends.Cuda/Dsv4/Dsv4CudaEngine.cs"
click node_mlx "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Backends.MLX/MlxStorage.cs"
click node_minimax "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Models/Models/MiniMaxH3/MiniMaxH3Pipeline.cs"
click node_wan "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Models/Models/WanVideo/WanVideoPipeline.cs"
click node_qwenimage "https://github.com/zhongkaifu/tensorsharp/tree/main/TensorSharp.Models/Models"
click node_agenthost "https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/src/TensorAgent.Core/Hosting/AgentAppHost.cs"
click node_skills "https://github.com/zhongkaifu/tensorsharp/tree/main/TensorAgent/skills"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_interfaces,node_chat,node_embeddings,node_catalog,node_modelstore,node_downloads,node_user toneBlue
class node_engine,node_scheduler,node_kv,node_models,node_ggml,node_cuda,node_mlx toneAmber
class node_minimax,node_wan,node_qwenimage toneMint
class node_agenthost,node_skills,node_toolwork toneRose
```

## Components

### Serving and chat

- **CLI, UI, APIs**
- [Chat generation](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Chat/ChatGenerationPipeline.cs): `TensorSharp.Chat/ChatGenerationPipeline.cs`
- **Embedding APIs**
- [Model catalog](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/src/TensorAgent.Core/Catalog/ModelCatalog.cs): `TensorAgent/src/TensorAgent.Core/Catalog/ModelCatalog.cs`
- [Model storage](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/src/TensorAgent.Core/Catalog/ModelStore.cs): `TensorAgent/src/TensorAgent.Core/Catalog/ModelStore.cs`
- [Model downloads](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/src/TensorAgent.Core/Downloads/ModelDownloadManager.cs): `TensorAgent/src/TensorAgent.Core/Downloads/ModelDownloadManager.cs`

### Inference runtime

- [Inference engine](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Runtime/Scheduling/InferenceEngine.cs): `TensorSharp.Runtime/Scheduling/InferenceEngine.cs`
- [Request scheduling](https://github.com/zhongkaifu/tensorsharp/tree/main/TensorSharp.Runtime/Scheduling): `TensorSharp.Runtime/Scheduling`
- [KV cache](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Models/KvCacheStorage.cs): `TensorSharp.Models/KvCacheStorage.cs`
- [Model families](https://github.com/zhongkaifu/tensorsharp/tree/main/TensorSharp.Models/Models): `TensorSharp.Models/Models`
- [GGML backend](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Backends.GGML/GgmlStorage.cs): `TensorSharp.Backends.GGML/GgmlStorage.cs`
- [CUDA backend](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Backends.Cuda/Dsv4/Dsv4CudaEngine.cs): `TensorSharp.Backends.Cuda/Dsv4/Dsv4CudaEngine.cs`
- [MLX backend](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Backends.MLX/MlxStorage.cs): `TensorSharp.Backends.MLX/MlxStorage.cs`

### Media generation

- [MiniMax video](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Models/Models/MiniMaxH3/MiniMaxH3Pipeline.cs): `TensorSharp.Models/Models/MiniMaxH3/MiniMaxH3Pipeline.cs`
- [Wan video](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorSharp.Models/Models/WanVideo/WanVideoPipeline.cs): `TensorSharp.Models/Models/WanVideo/WanVideoPipeline.cs`
- [Qwen image](https://github.com/zhongkaifu/tensorsharp/tree/main/TensorSharp.Models/Models): `TensorSharp.Models/Models`

### Agent workflows

- [Agent host](https://github.com/zhongkaifu/tensorsharp/blob/main/TensorAgent/src/TensorAgent.Core/Hosting/AgentAppHost.cs): `TensorAgent/src/TensorAgent.Core/Hosting/AgentAppHost.cs`
- [Skills and tools](https://github.com/zhongkaifu/tensorsharp/tree/main/TensorAgent/skills): `TensorAgent/skills`
- **Sandboxed work**

### Other components

- **User**

## Connections

- User → CLI, UI, APIs: submits input
- CLI, UI, APIs → Chat generation: requests generation
- CLI, UI, APIs → Embedding APIs: requests embeddings
- Chat generation → Inference engine: submits inference
- Inference engine → Request scheduling: schedules requests
- Inference engine → Model families: executes model
- Model families → KV cache: uses cache
- Model families → GGML backend: runs on backend
- Model families → CUDA backend: runs on backend
- Model families → MLX backend: runs on backend
- CLI, UI, APIs → MiniMax video: requests video
- CLI, UI, APIs → Wan video: requests video
- CLI, UI, APIs → Qwen image: requests image
- User → Model catalog: selects model
- Model catalog → Model storage: uses catalog store
- Model downloads → Model storage: stores downloads
- User → Agent host: starts agent work
- Agent host → Chat generation: requests model output
- Agent host → Skills and tools: loads skills
- Agent host → Sandboxed work: dispatches tools