How zhongkaifu/tensorsharp works
TensorSharp is a local .NET inference engine and serving product: a user submits text or multimodal input through a CLI, browser UI, or compatible HTTP API; chat generation renders and tokenizes the request, then the inference runtime schedules model execution and returns generated output. Separate product branches cover image/video generation, embeddings, model acquisition, and optional agent workflows. The README documents these capabilities, but the sampled source is incomplete, so links to unsampled subsystems are shown only where documentation supports them.
Main entry points: Models Model families · AgentAppHost.cs Agent host · InferenceEngine.cs Inference engine · ChatGenerationPipeline.cs Chat generation
Components 20
Serving and chat
- CLI, UI, APIs
- Chat generation
TensorSharp.Chat/ChatGenerationPipeline.cs - Embedding APIs
- Model catalog
TensorAgent/src/TensorAgent.Core/Catalog/ModelCatalog.cs - Model storage
TensorAgent/src/TensorAgent.Core/Catalog/ModelStore.cs - Model downloads
TensorAgent/src/TensorAgent.Core/Downloads/ModelDownloadManager.cs
Inference runtime
- Inference engine
TensorSharp.Runtime/Scheduling/InferenceEngine.cs - Request scheduling
TensorSharp.Runtime/Scheduling - KV cache
TensorSharp.Models/KvCacheStorage.cs - Model families
TensorSharp.Models/Models - GGML backend
TensorSharp.Backends.GGML/GgmlStorage.cs - CUDA backend
TensorSharp.Backends.Cuda/Dsv4/Dsv4CudaEngine.cs - MLX backend
TensorSharp.Backends.MLX/MlxStorage.cs
Media generation
- MiniMax video
TensorSharp.Models/Models/MiniMaxH3/MiniMaxH3Pipeline.cs - Wan video
TensorSharp.Models/Models/WanVideo/WanVideoPipeline.cs - Qwen image
TensorSharp.Models/Models
Agent workflows
- Agent host
TensorAgent/src/TensorAgent.Core/Hosting/AgentAppHost.cs - Skills and tools
TensorAgent/skills - Sandboxed work
Other components
- User
Connections 20
- User → CLI, UI, APIs: submits input
- CLI, UI, APIs → Chat generation: requests generation
- CLI, UI, APIs → Embedding APIs: requests embeddings
- Chat generation → Inference engine: submits inference
- Inference engine → Request scheduling: schedules requests
- Inference engine → Model families: executes model
- Model families → KV cache: uses cache
- Model families → GGML backend: runs on backend
- Model families → CUDA backend: runs on backend
- Model families → MLX backend: runs on backend
- CLI, UI, APIs → MiniMax video: requests video
- CLI, UI, APIs → Wan video: requests video
- CLI, UI, APIs → Qwen image: requests image
- User → Model catalog: selects model
- Model catalog → Model storage: uses catalog store
- Model downloads → Model storage: stores downloads
- User → Agent host: starts agent work
- Agent host → Chat generation: requests model output
- Agent host → Skills and tools: loads skills
- Agent host → Sandboxed work: dispatches tools