How saddam213/tensorstack works
1. What this repository is
- This is a modular .NET solution that packages a reusable AI SDK plus a reference desktop app, Amuse, built on top of it.
- The architecture is centered on local inference and media workflows rather than a traditional client/server web stack.
2. Top-level layering
- The solution splits into:
- UI/app shell: Amuse.App
- Shared application contracts: Amuse.Common
- Execution hosts: Amuse.Host.Onnx, Amuse.Host.PyTorch
- Core SDK packages: TensorStack.*
- Reusable UI framework: TensorStack.WPF
- Media adapters: TensorStack.Audio*, TensorStack.Video*, TensorStack.Image*
- This keeps the product-specific UI separate from inference engines and media primitives.
3. Reference application boundary
- Amuse.App is the main desktop UI, built with WPF.
- Its structure is typical MVVM-style app layering:
- Views for task-specific screens
- Controls for reusable input widgets
- Dialogs for model/environment/download management
- Services for orchestration
- Common for domain models and option objects
- The app covers many workflows: text-to-image, image edit/inpaint/extract/upscale, video interpolation/compose/extract, audio transcription, text-to-video, and settings.
4. Application service layer
- The services in Amuse.App/Services are the main orchestration boundary between UI and execution:
- DiffusionService, UpscaleService, ExtractService, InterpolationService
- ModelDownloadService, EnvironmentService, HardwareService
- HistoryService, AutomationManager, MigrationService, PreviewService
- These services manage task lifecycle, model selection, environment setup, download state, and result/history tracking.
5. Shared domain models and state
- Amuse.App/Common contains the app’s domain objects and options:
- model descriptors, device/environment models, download queue items
- workflow-specific inputs and history records
- memory profiles, scheduler options, template settings, statistics
- This is the boundary where UI-friendly state is shaped before being sent into the runtime layers.
6. Core execution and pipeline boundary
- TensorStack.Common is the runtime foundation used across the SDK:
- device/runtime abstractions
- model/session/config types
- file/queue/metadata utilities
- pipeline interfaces and run-progress contracts
- It also defines the client/server process boundary through PipelineClient, PipelineServer, and ProcessHandler, which is important for understanding how the app talks to isolated inference processes.
7. Inference providers and runtime backends
- The repository supports multiple execution providers through separate packages:
- TensorStack.Providers.CPU
- TensorStack.Providers.CUDA
- TensorStack.Providers.DML
- Amuse.App/Backend shows the app can target at least two diffusion runtimes:
- OnnxDiffusion
- PyTorchDiffusion
- IDiffusionRuntime is the abstraction that lets the app swap backend implementations without changing the UI/service layer.
8. Model-specific AI packages
- The SDK is organized by problem domain:
- TensorStack.StableDiffusion for image/video diffusion pipelines, schedulers, prompts, and model configs
- TensorStack.TextGeneration for LLM/tokenizer/sampler/cache-based generation flows
- TensorStack.Upscaler for image enhancement pipelines
- TensorStack.Extractors for background/pose/feature extraction pipelines
- These packages contain the actual model wrappers, pipeline classes, schedulers, tokenizers, samplers, and option/result types used by higher-level apps.
9. Python interoperability path
- TensorStack.Python is a second major execution path, bridging .NET to Python-based pipelines.
- It includes:
- process/config/serializer management
- scheduler options
- Python-side library code under Python/lib/tensorstack
- many pipeline definitions and model templates under Pipelines/
- Architecturally, this is the bridge for models that are implemented or orchestrated in Python rather than pure .NET/ONNX.
10. Media processing subsystems
- Audio, video, and image are split into shared abstractions plus platform-specific implementations:
- TensorStack.Audio and TensorStack.Audio.Windows
- TensorStack.Video and TensorStack.Video.Windows
- TensorStack.Image plus WPF/Bitmap/ImageSharp adapters
- The Windows-specific packages wrap native tools/libraries such as FFmpeg and OpenCvSharp, which matters because media ingestion/export is part of the execution pipeline, not just a UI concern.
11. Reusable WPF UI toolkit
- TensorStack.WPF is a shared presentation framework used by the app:
- custom controls, dialogs, converters, behaviors
- navigation/dialog/download/component services
- This separates generic desktop UI building blocks from Amuse-specific screens and keeps the app code focused on AI workflow composition.
12. Model assets, previews, and packaging
- The repo includes many on-disk model/config templates under TensorStack.Python/Pipelines/Templates and preview ONNX models under Amuse.App/Plugins/Preview.
- These are runtime assets, not just source code; they form part of the deployment diagram because pipelines depend on model files, scheduler configs, tokenizer configs, and preview artifacts.
13. Download and external model access
- The presence of HuggingFace.cs, access-token models, gated-model dialogs, and download services indicates the app can retrieve models from external registries, especially Hugging Face.
- That creates a distinct boundary between local orchestration and remote model acquisition/authentication.
14. End-to-end flow to diagram
- A typical flow is:
- user interacts with a WPF view/control
- the view delegates to an app service
- the service loads settings/model metadata and builds a pipeline request
- the request crosses into a runtime host or provider-backed SDK package
- inference runs locally, possibly through a separate ONNX/PyTorch/Python process
- output media is returned, previewed, stored in history, or exported via audio/video/image adapters
- That’s the main architecture story: UI orchestration on top of modular local AI runtimes with strong process and media boundaries.
Main entry points: TensorStack.Common TensorStack Common · Services Services · TensorStack.StableDiffusion Stable Diffusion · Amuse.App Amuse App
Components 22
Amuse App
Desktop product shell and workflow orchestration
- Amuse App · Reference desktop application for AI workflows and settings
Apps/Amuse.App - Views · Task-specific desktop screens for generation, editing, and settings
Apps/Amuse.App/Views - Services · Application services that coordinate models, history, downloads, and execution
Apps/Amuse.App/Services - App Common · UI-friendly domain objects, option models, history records, and state
Apps/Amuse.App/Common - Backends · Diffusion runtime implementations for ONNX and PyTorch
Apps/Amuse.App/Backend - Model Access · External model discovery, gated access, and download integration
Apps/Amuse.App/HuggingFace.cs - Preview Models · Packaged ONNX preview models used for model selection and previewing
Apps/Amuse.App/Plugins/Preview
Process Bridge
Shared client/server boundary for isolated inference hosts
- Pipeline Bridge · Process-bound message layer used to talk to isolated inference hosts
Apps/Amuse.Common - Client · Sends pipeline and command requests from the app side
Apps/Amuse.Common/PipelineClient.cs - Server · Receives requests and coordinates host-side pipeline execution
Apps/Amuse.Common/PipelineServer.cs - Process Host · Spawns and supervises the isolated inference process
Apps/Amuse.Common/ProcessHandler.cs
TensorStack SDK
Core runtime, model, and pipeline packages
- TensorStack Common · Shared device, model, tensor, and pipeline abstractions
TensorStack.Common - Stable Diffusion · Diffusion models, schedulers, and pipeline families
TensorStack.StableDiffusion - Text Generation · Tokenizer, cache, sampler, and generation pipelines
TensorStack.TextGeneration - Upscaler · Image upscaling models and pipeline orchestration
TensorStack.Upscaler - Extractors · Background and pose extraction pipelines
TensorStack.Extractors - Python Bridge · .NET-to-Python process bridge, serializers, schedulers, and pipeline templates
TensorStack.Python - Providers · CPU, CUDA, and DML execution provider packages
TensorStack.Providers.CPU
Media Stack
Audio, video, and image processing primitives
WPF Toolkit
Reusable desktop UI components
- WPF Toolkit · Reusable WPF controls, dialogs, navigation, and window base classes
TensorStack.WPF
Connections 27
- Amuse App → Views: renders · Main window hosts the workflow screens
- Amuse App → Services: uses · App shell delegates orchestration to services
- Amuse App → App Common: binds · UI state is shaped through shared app models
- Views → WPF Toolkit: compose · Views are built from shared WPF controls and dialogs
- Services → Backends: selects · Services choose ONNX or PyTorch diffusion runtimes
- Services → Client: dispatches · Service calls cross the process boundary through the client
- Services → Model Access: downloads · Services manage remote model acquisition and access gating
- Services → Preview Models: loads · Preview and model browsing use packaged ONNX assets
- Backends → Stable Diffusion: invokes · Diffusion backends run the model pipelines
- Backends → Providers: targets · Backends rely on execution providers for device acceleration
- Client → Pipeline Bridge: sends · Client serializes requests into pipeline messages
- Pipeline Bridge → Server: routes · Messages are handled by the host-side server
- Process Host → Server: launches · Process handler starts and supervises the host process
- Server → TensorStack Common: runs on · Host execution uses the shared runtime foundation
- TensorStack Common → Stable Diffusion: supports · Stable diffusion builds on shared device/model abstractions
- TensorStack Common → Text Generation: supports · Text generation uses the same runtime foundation
- TensorStack Common → Upscaler: supports · Upscaling pipelines share the common runtime layer
- TensorStack Common → Extractors: supports · Extraction pipelines share the common runtime layer
- Python Bridge → TensorStack Common: bridges · Python execution is integrated through the common process and model abstractions
- Python Bridge → Stable Diffusion: hosts · Python pipelines mirror diffusion and media workflows
- Audio → TensorStack Common: uses · Audio primitives consume shared tensor and queue types
- Video → TensorStack Common: uses · Video primitives consume shared tensor and queue types
- Image → TensorStack Common: uses · Image adapters build on the shared tensor and image abstractions
- Stable Diffusion → Audio: produces · Some diffusion workflows emit audio-associated outputs or inputs
- Stable Diffusion → Video: produces · Video pipelines and interpolation workflows integrate with video processing
- App Common → TensorStack Common: maps to · App models are converted into runtime options and requests
- WPF Toolkit → Amuse App: enables · Reusable UI infrastructure supports the desktop app shell