# saddam213/tensorstack architecture

> Architecture diagram of the GitHub repository saddam213/tensorstack, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/saddam213/tensorstack
- Repository: https://github.com/saddam213/tensorstack

## Overview

1. **What this repository is**
   - This is a modular .NET solution that packages a reusable AI SDK plus a reference desktop app, **Amuse**, built on top of it.
   - The architecture is centered on **local inference** and **media workflows** rather than a traditional client/server web stack.

2. **Top-level layering**
   - The solution splits into:
     - **UI/app shell**: `Amuse.App`
     - **Shared application contracts**: `Amuse.Common`
     - **Execution hosts**: `Amuse.Host.Onnx`, `Amuse.Host.PyTorch`
     - **Core SDK packages**: `TensorStack.*`
     - **Reusable UI framework**: `TensorStack.WPF`
     - **Media adapters**: `TensorStack.Audio*`, `TensorStack.Video*`, `TensorStack.Image*`
   - This keeps the product-specific UI separate from inference engines and media primitives.

3. **Reference application boundary**
   - `Amuse.App` is the main desktop UI, built with **WPF**.
   - Its structure is typical MVVM-style app layering:
     - `Views` for task-specific screens
     - `Controls` for reusable input widgets
     - `Dialogs` for model/environment/download management
     - `Services` for orchestration
     - `Common` for domain models and option objects
   - The app covers many workflows: text-to-image, image edit/inpaint/extract/upscale, video interpolation/compose/extract, audio transcription, text-to-video, and settings.

4. **Application service layer**
   - The services in `Amuse.App/Services` are the main orchestration boundary between UI and execution:
     - `DiffusionService`, `UpscaleService`, `ExtractService`, `InterpolationService`
     - `ModelDownloadService`, `EnvironmentService`, `HardwareService`
     - `HistoryService`, `AutomationManager`, `MigrationService`, `PreviewService`
   - These services manage task lifecycle, model selection, environment setup, download state, and result/history tracking.

5. **Shared domain models and state**
   - `Amuse.App/Common` contains the app’s domain objects and options:
     - model descriptors, device/environment models, download queue items
     - workflow-specific inputs and history records
     - memory profiles, scheduler options, template settings, statistics
   - This is the boundary where UI-friendly state is shaped before being sent into the runtime layers.

6. **Core execution and pipeline boundary**
   - `TensorStack.Common` is the runtime foundation used across the SDK:
     - device/runtime abstractions
     - model/session/config types
     - file/queue/metadata utilities
     - pipeline interfaces and run-progress contracts
   - It also defines the **client/server process boundary** through `PipelineClient`, `PipelineServer`, and `ProcessHandler`, which is important for understanding how the app talks to isolated inference processes.

7. **Inference providers and runtime backends**
   - The repository supports multiple execution providers through separate packages:
     - `TensorStack.Providers.CPU`
     - `TensorStack.Providers.CUDA`
     - `TensorStack.Providers.DML`
   - `Amuse.App/Backend` shows the app can target at least two diffusion runtimes:
     - `OnnxDiffusion`
     - `PyTorchDiffusion`
   - `IDiffusionRuntime` is the abstraction that lets the app swap backend implementations without changing the UI/service layer.

8. **Model-specific AI packages**
   - The SDK is organized by problem domain:
     - `TensorStack.StableDiffusion` for image/video diffusion pipelines, schedulers, prompts, and model configs
     - `TensorStack.TextGeneration` for LLM/tokenizer/sampler/cache-based generation flows
     - `TensorStack.Upscaler` for image enhancement pipelines
     - `TensorStack.Extractors` for background/pose/feature extraction pipelines
   - These packages contain the actual model wrappers, pipeline classes, schedulers, tokenizers, samplers, and option/result types used by higher-level apps.

9. **Python interoperability path**
   - `TensorStack.Python` is a second major execution path, bridging .NET to Python-based pipelines.
   - It includes:
     - process/config/serializer management
     - scheduler options
     - Python-side library code under `Python/lib/tensorstack`
     - many pipeline definitions and model templates under `Pipelines/`
   - Architecturally, this is the bridge for models that are implemented or orchestrated in Python rather than pure .NET/ONNX.

10. **Media processing subsystems**
   - Audio, video, and image are split into shared abstractions plus platform-specific implementations:
     - `TensorStack.Audio` and `TensorStack.Audio.Windows`
     - `TensorStack.Video` and `TensorStack.Video.Windows`
     - `TensorStack.Image` plus WPF/Bitmap/ImageSharp adapters
   - The Windows-specific packages wrap native tools/libraries such as **FFmpeg** and **OpenCvSharp**, which matters because media ingestion/export is part of the execution pipeline, not just a UI concern.

11. **Reusable WPF UI toolkit**
   - `TensorStack.WPF` is a shared presentation framework used by the app:
     - custom controls, dialogs, converters, behaviors
     - navigation/dialog/download/component services
   - This separates generic desktop UI building blocks from Amuse-specific screens and keeps the app code focused on AI workflow composition.

12. **Model assets, previews, and packaging**
   - The repo includes many on-disk model/config templates under `TensorStack.Python/Pipelines/Templates` and preview ONNX models under `Amuse.App/Plugins/Preview`.
   - These are runtime assets, not just source code; they form part of the deployment diagram because pipelines depend on model files, scheduler configs, tokenizer configs, and preview artifacts.

13. **Download and external model access**
   - The presence of `HuggingFace.cs`, access-token models, gated-model dialogs, and download services indicates the app can retrieve models from external registries, especially Hugging Face.
   - That creates a distinct boundary between local orchestration and remote model acquisition/authentication.

14. **End-to-end flow to diagram**
   - A typical flow is:
     - user interacts with a WPF view/control
     - the view delegates to an app service
     - the service loads settings/model metadata and builds a pipeline request
     - the request crosses into a runtime host or provider-backed SDK package
     - inference runs locally, possibly through a separate ONNX/PyTorch/Python process
     - output media is returned, previewed, stored in history, or exported via audio/video/image adapters
   - That’s the main architecture story: **UI orchestration on top of modular local AI runtimes with strong process and media boundaries**.

Main entry points: [`TensorStack.Common`](https://github.com/saddam213/tensorstack/blob/master/TensorStack.Common) (TensorStack Common), [`Apps/Amuse.App/Services`](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Services) (Services), [`TensorStack.StableDiffusion`](https://github.com/saddam213/tensorstack/blob/master/TensorStack.StableDiffusion) (Stable Diffusion), [`Apps/Amuse.App`](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App) (Amuse App)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_amuse["Amuse App"]
  node_amuse_app["Amuse App<br/>WPF desktop<br/>[Amuse.App]"]
  node_amuse_views["Views<br/>workflow screens<br/>[Views]"]
  node_amuse_services["Services<br/>orchestration<br/>[Services]"]
  node_amuse_common["App Common<br/>domain models<br/>[Common]"]
  node_amuse_backend["Backends<br/>runtime adapters<br/>[Backend]"]
  node_amuse_hf["Model Access<br/>registry access<br/>[HuggingFace.cs]"]
  node_preview_assets["Preview Models<br/>runtime assets<br/>[Preview]"]
end

subgraph group_group_bridge["Process Bridge"]
  node_pipeline_bridge["Pipeline Bridge<br/>client/server<br/>[Amuse.Common]"]
  node_pipeline_client["Client<br/>request side<br/>[PipelineClient.cs]"]
  node_pipeline_server["Server<br/>host side<br/>[PipelineServer.cs]"]
  node_process_handler["Process Host<br/>process control<br/>[ProcessHandler.cs]"]
end

subgraph group_group_sdk["TensorStack SDK"]
  node_tensor_common["TensorStack Common<br/>runtime core<br/>[TensorStack.Common]"]
  node_sd_sdk["Stable Diffusion<br/>image/video diffusion"]
  node_text_sdk["Text Generation<br/>llm runtime"]
  node_upscaler_sdk["Upscaler<br/>enhancement pipeline"]
  node_extractor_sdk["Extractors<br/>vision extraction"]
  node_python_sdk["Python Bridge<br/>python runtime<br/>[TensorStack.Python]"]
  node_providers["Providers"]
end

subgraph group_group_media["Media Stack"]
  node_audio_media["Audio<br/>[TensorStack.Audio]"]
  node_video_media["Video<br/>[TensorStack.Video]"]
  node_image_media["Image<br/>[TensorStack.Image]"]
end

subgraph group_group_ui["WPF Toolkit"]
  node_wpf_toolkit["WPF Toolkit<br/>ui framework<br/>[TensorStack.WPF]"]
end

node_amuse_app -->|"renders"| node_amuse_views
node_amuse_app -->|"uses"| node_amuse_services
node_amuse_app -->|"binds"| node_amuse_common
node_amuse_views -->|"compose"| node_wpf_toolkit
node_amuse_services -->|"selects"| node_amuse_backend
node_amuse_services -->|"dispatches"| node_pipeline_client
node_amuse_services -->|"downloads"| node_amuse_hf
node_amuse_services -->|"loads"| node_preview_assets
node_amuse_backend -->|"invokes"| node_sd_sdk
node_amuse_backend -->|"targets"| node_providers
node_pipeline_client -->|"sends"| node_pipeline_bridge
node_pipeline_bridge -->|"routes"| node_pipeline_server
node_process_handler -->|"launches"| node_pipeline_server
node_pipeline_server -->|"runs on"| node_tensor_common
node_tensor_common -->|"supports"| node_sd_sdk
node_tensor_common -->|"supports"| node_text_sdk
node_tensor_common -->|"supports"| node_upscaler_sdk
node_tensor_common -->|"supports"| node_extractor_sdk
node_python_sdk -->|"bridges"| node_tensor_common
node_python_sdk -->|"hosts"| node_sd_sdk
node_audio_media -->|"uses"| node_tensor_common
node_video_media -->|"uses"| node_tensor_common
node_image_media -->|"uses"| node_tensor_common
node_sd_sdk -.->|"produces"| node_audio_media
node_sd_sdk -.->|"produces"| node_video_media
node_amuse_common -->|"maps to"| node_tensor_common
node_wpf_toolkit -->|"enables"| node_amuse_app

click node_amuse_app "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App"
click node_amuse_views "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Views"
click node_amuse_services "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Services"
click node_amuse_common "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Common"
click node_amuse_backend "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Backend"
click node_amuse_hf "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/HuggingFace.cs"
click node_pipeline_bridge "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.Common"
click node_pipeline_client "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.Common/PipelineClient.cs"
click node_pipeline_server "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.Common/PipelineServer.cs"
click node_process_handler "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.Common/ProcessHandler.cs"
click node_tensor_common "https://github.com/saddam213/tensorstack/blob/master/TensorStack.Common"
click node_sd_sdk "https://github.com/saddam213/tensorstack/blob/master/TensorStack.StableDiffusion"
click node_text_sdk "https://github.com/saddam213/tensorstack/blob/master/TensorStack.TextGeneration"
click node_upscaler_sdk "https://github.com/saddam213/tensorstack/blob/master/TensorStack.Upscaler"
click node_extractor_sdk "https://github.com/saddam213/tensorstack/blob/master/TensorStack.Extractors"
click node_python_sdk "https://github.com/saddam213/tensorstack/blob/master/TensorStack.Python"
click node_providers "https://github.com/saddam213/tensorstack/blob/master/TensorStack.Providers.CPU"
click node_audio_media "https://github.com/saddam213/tensorstack/blob/master/TensorStack.Audio"
click node_video_media "https://github.com/saddam213/tensorstack/blob/master/TensorStack.Video"
click node_image_media "https://github.com/saddam213/tensorstack/blob/master/TensorStack.Image"
click node_wpf_toolkit "https://github.com/saddam213/tensorstack/blob/master/TensorStack.WPF"
click node_preview_assets "https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Plugins/Preview"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_amuse_app,node_amuse_views,node_amuse_services,node_amuse_common,node_amuse_backend,node_amuse_hf,node_preview_assets toneBlue
class node_pipeline_bridge,node_pipeline_client,node_pipeline_server,node_process_handler toneAmber
class node_tensor_common,node_sd_sdk,node_text_sdk,node_upscaler_sdk,node_extractor_sdk,node_python_sdk,node_providers toneMint
class node_audio_media,node_video_media,node_image_media toneRose
class node_wpf_toolkit toneIndigo
```

## Components

### Amuse App

Desktop product shell and workflow orchestration

- [Amuse App](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App): WPF desktop · Reference desktop application for AI workflows and settings · `Apps/Amuse.App`
- [Views](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Views): workflow screens · Task-specific desktop screens for generation, editing, and settings · `Apps/Amuse.App/Views`
- [Services](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Services): orchestration · Application services that coordinate models, history, downloads, and execution · `Apps/Amuse.App/Services`
- [App Common](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Common): domain models · UI-friendly domain objects, option models, history records, and state · `Apps/Amuse.App/Common`
- [Backends](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Backend): runtime adapters · Diffusion runtime implementations for ONNX and PyTorch · `Apps/Amuse.App/Backend`
- [Model Access](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/HuggingFace.cs): registry access · External model discovery, gated access, and download integration · `Apps/Amuse.App/HuggingFace.cs`
- [Preview Models](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.App/Plugins/Preview): runtime assets · Packaged ONNX preview models used for model selection and previewing · `Apps/Amuse.App/Plugins/Preview`

### Process Bridge

Shared client/server boundary for isolated inference hosts

- [Pipeline Bridge](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.Common): client/server · Process-bound message layer used to talk to isolated inference hosts · `Apps/Amuse.Common`
- [Client](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.Common/PipelineClient.cs): request side · Sends pipeline and command requests from the app side · `Apps/Amuse.Common/PipelineClient.cs`
- [Server](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.Common/PipelineServer.cs): host side · Receives requests and coordinates host-side pipeline execution · `Apps/Amuse.Common/PipelineServer.cs`
- [Process Host](https://github.com/saddam213/tensorstack/blob/master/Apps/Amuse.Common/ProcessHandler.cs): process control · Spawns and supervises the isolated inference process · `Apps/Amuse.Common/ProcessHandler.cs`

### TensorStack SDK

Core runtime, model, and pipeline packages

- [TensorStack Common](https://github.com/saddam213/tensorstack/blob/master/TensorStack.Common): runtime core · Shared device, model, tensor, and pipeline abstractions · `TensorStack.Common`
- [Stable Diffusion](https://github.com/saddam213/tensorstack/blob/master/TensorStack.StableDiffusion): image/video diffusion · Diffusion models, schedulers, and pipeline families · `TensorStack.StableDiffusion`
- [Text Generation](https://github.com/saddam213/tensorstack/blob/master/TensorStack.TextGeneration): llm runtime · Tokenizer, cache, sampler, and generation pipelines · `TensorStack.TextGeneration`
- [Upscaler](https://github.com/saddam213/tensorstack/blob/master/TensorStack.Upscaler): enhancement pipeline · Image upscaling models and pipeline orchestration · `TensorStack.Upscaler`
- [Extractors](https://github.com/saddam213/tensorstack/blob/master/TensorStack.Extractors): vision extraction · Background and pose extraction pipelines · `TensorStack.Extractors`
- [Python Bridge](https://github.com/saddam213/tensorstack/blob/master/TensorStack.Python): python runtime · .NET-to-Python process bridge, serializers, schedulers, and pipeline templates · `TensorStack.Python`
- [Providers](https://github.com/saddam213/tensorstack/blob/master/TensorStack.Providers.CPU): execution providers · CPU, CUDA, and DML execution provider packages · `TensorStack.Providers.CPU`

### Media Stack

Audio, video, and image processing primitives

- [Audio](https://github.com/saddam213/tensorstack/blob/master/TensorStack.Audio): audio primitives · Audio input and timeline abstractions plus Windows capture helpers · `TensorStack.Audio`
- [Video](https://github.com/saddam213/tensorstack/blob/master/TensorStack.Video): video primitives · Video input and interpolation pipeline plus Windows capture helpers · `TensorStack.Video`
- [Image](https://github.com/saddam213/tensorstack/blob/master/TensorStack.Image): image input · Image input abstractions and adapters for bitmap, ImageSharp, and WPF · `TensorStack.Image`

### WPF Toolkit

Reusable desktop UI components

- [WPF Toolkit](https://github.com/saddam213/tensorstack/blob/master/TensorStack.WPF): ui framework · Reusable WPF controls, dialogs, navigation, and window base classes · `TensorStack.WPF`

## Connections

- Amuse App → Views: renders · Main window hosts the workflow screens
- Amuse App → Services: uses · App shell delegates orchestration to services
- Amuse App → App Common: binds · UI state is shaped through shared app models
- Views → WPF Toolkit: compose · Views are built from shared WPF controls and dialogs
- Services → Backends: selects · Services choose ONNX or PyTorch diffusion runtimes
- Services → Client: dispatches · Service calls cross the process boundary through the client
- Services → Model Access: downloads · Services manage remote model acquisition and access gating
- Services → Preview Models: loads · Preview and model browsing use packaged ONNX assets
- Backends → Stable Diffusion: invokes · Diffusion backends run the model pipelines
- Backends → Providers: targets · Backends rely on execution providers for device acceleration
- Client → Pipeline Bridge: sends · Client serializes requests into pipeline messages
- Pipeline Bridge → Server: routes · Messages are handled by the host-side server
- Process Host → Server: launches · Process handler starts and supervises the host process
- Server → TensorStack Common: runs on · Host execution uses the shared runtime foundation
- TensorStack Common → Stable Diffusion: supports · Stable diffusion builds on shared device/model abstractions
- TensorStack Common → Text Generation: supports · Text generation uses the same runtime foundation
- TensorStack Common → Upscaler: supports · Upscaling pipelines share the common runtime layer
- TensorStack Common → Extractors: supports · Extraction pipelines share the common runtime layer
- Python Bridge → TensorStack Common: bridges · Python execution is integrated through the common process and model abstractions
- Python Bridge → Stable Diffusion: hosts · Python pipelines mirror diffusion and media workflows
- Audio → TensorStack Common: uses · Audio primitives consume shared tensor and queue types
- Video → TensorStack Common: uses · Video primitives consume shared tensor and queue types
- Image → TensorStack Common: uses · Image adapters build on the shared tensor and image abstractions
- Stable Diffusion → Audio: produces · Some diffusion workflows emit audio-associated outputs or inputs
- Stable Diffusion → Video: produces · Video pipelines and interpolation workflows integrate with video processing
- App Common → TensorStack Common: maps to · App models are converted into runtime options and requests
- WPF Toolkit → Amuse App: enables · Reusable UI infrastructure supports the desktop app shell
