# osaurus-ai/vmlx-swift architecture

> Architecture diagram of the GitHub repository osaurus-ai/vmlx-swift, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/osaurus-ai/vmlx-swift
- Repository: https://github.com/osaurus-ai/vmlx-swift

## Overview

vmlx-swift is a Swift MLX inference stack for local language, vision-language, multimodal, embedding, and image-generation workloads. The main workflow is an application submitting text or chat input, loading a model and tokenizer, then receiving generated stream events; chat templates, generation controls, cache behavior, and tool/reasoning parsing shape that runtime. Major branches include media processing, distributed execution, and Flux image generation. The README describes the facade’s intended coverage; sampled implementation is limited, so edges for unsampled subsystems reflect documented responsibilities rather than verified call sites.

Main entry points: [`Libraries/MLXLLM/LLMModel.swift`](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLLM/LLMModel.swift) (LLM Interface), [`Libraries/MLXLMCommon/BatchEngine/BatchEngine.swift`](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/BatchEngine/BatchEngine.swift) (Batch Generation), [`Libraries/MLXVLM/VLMModel.swift`](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXVLM/VLMModel.swift) (VLM Runtime), [`Libraries/MLXDistributedCore/DistributedModePlanner.swift`](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXDistributedCore/DistributedModePlanner.swift) (Cluster Planning)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Application API"]
  node_facade["VMLX Facade<br/>[VMLX.swift]"]
  node_chat["Chat Sessions<br/>[ChatSession.swift]"]
  node_text_result["Text And Events"]
end

subgraph group_language["Language Runtime"]
  node_llm_api["LLM Interface<br/>[LLMModel.swift]"]
  node_model_load["Model Loading<br/>[Load.swift]"]
  node_model_factory["Model Factory<br/>[ModelFactory.swift]"]
  node_llm_models["Language Families<br/>[Llama.swift]"]
  node_tokenizer["Tokenizer<br/>[Tokenizer.swift]"]
  node_templates["Chat Templates"]
  node_batch_engine["Batch Generation<br/>[BatchEngine.swift]"]
  node_cache[("Generation Caches")]
  node_reasoning_tools["Reasoning And Tools"]
  node_generation_stream["Generation Stream"]
end

subgraph group_media["Media Models"]
  node_vlm_api["VLM Runtime<br/>[VLMModel.swift]"]
  node_media_processing["Media Processing"]
  node_vlm_models["Vision And Omni Models<br/>[Qwen35.swift]"]
  node_embeddings["Embedding Pipeline"]
end

subgraph group_distributed["Distributed Runtime"]
  node_distributed_core["Cluster Planning"]
  node_collectives["Distributed Collectives<br/>[Collectives.swift]"]
  node_stage_transport["Pipeline Transport"]
end

subgraph group_image["Image Generation"]
  node_flux_engine["Flux Engine<br/>[FluxEngine.swift]"]
  node_flux_models["Flux Model Families<br/>[Exports.swift]"]
  node_image_result["Image Events<br/>[Requests.swift]"]
end

node_application(("Embedding Application"))

node_application -->|"imports API"| node_facade
node_facade -->|"exposes chat"| node_chat
node_chat -->|"submits turns"| node_llm_api
node_llm_api -->|"loads model"| node_model_load
node_model_load -->|"constructs"| node_model_factory
node_model_factory -->|"selects family"| node_llm_models
node_llm_api -->|"encodes input"| node_tokenizer
node_llm_api -->|"resolves template"| node_templates
node_llm_api -->|"generates"| node_batch_engine
node_batch_engine -->|"reads and writes"| node_cache
node_batch_engine -->|"routes output"| node_reasoning_tools
node_batch_engine -->|"emits events"| node_generation_stream
node_generation_stream -->|"yields events"| node_text_result
node_application -->|"submits media input"| node_vlm_api
node_vlm_api -->|"prepares media"| node_media_processing
node_vlm_api -->|"runs model"| node_vlm_models
node_application -->|"requests vectors"| node_embeddings
node_application -->|"plans cluster"| node_distributed_core
node_distributed_core -->|"configures execution"| node_collectives
node_distributed_core -->|"coordinates stages"| node_stage_transport
node_application -->|"requests image"| node_flux_engine
node_flux_engine -->|"dispatches model"| node_flux_models
node_flux_engine -->|"streams events"| node_image_result

click node_facade "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/VMLX/VMLX.swift"
click node_chat "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/ChatSession.swift"
click node_llm_api "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLLM/LLMModel.swift"
click node_model_load "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/Load.swift"
click node_model_factory "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/ModelFactory.swift"
click node_llm_models "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLLM/Models/Llama.swift"
click node_tokenizer "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/Tokenizer.swift"
click node_templates "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/ChatTemplateResolution.swift"
click node_batch_engine "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/BatchEngine/BatchEngine.swift"
click node_cache "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/Cache/CacheCoordinator.swift"
click node_reasoning_tools "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/Tool/ToolCallProcessor.swift"
click node_generation_stream "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/GenerationStreamRouting.swift"
click node_text_result "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/GenerationActivity.swift"
click node_vlm_api "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXVLM/VLMModel.swift"
click node_media_processing "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXVLM/MediaProcessing.swift"
click node_vlm_models "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXVLM/Models/Qwen35.swift"
click node_embeddings "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXEmbedders/Model2VecStaticEmbeddingPipeline.swift"
click node_distributed_core "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXDistributedCore/DistributedModePlanner.swift"
click node_collectives "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXDistributedTP/Collectives.swift"
click node_stage_transport "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXDistributedTransport/PipelineStageClient.swift"
click node_flux_engine "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/vMLXFlux/FluxEngine.swift"
click node_flux_models "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/vMLXFluxModels/Exports.swift"
click node_image_result "https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/vMLXFluxKit/Requests.swift"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_facade,node_chat,node_text_result toneBlue
class node_llm_api,node_model_load,node_model_factory,node_llm_models,node_tokenizer,node_templates,node_batch_engine,node_cache,node_reasoning_tools,node_generation_stream toneAmber
class node_vlm_api,node_media_processing,node_vlm_models,node_embeddings toneMint
class node_distributed_core,node_collectives,node_stage_transport toneRose
class node_flux_engine,node_flux_models,node_image_result,node_application toneIndigo
```

## Components

### Application API

- [VMLX Facade](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/VMLX/VMLX.swift): `Libraries/VMLX/VMLX.swift`
- [Chat Sessions](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/ChatSession.swift): `Libraries/MLXLMCommon/ChatSession.swift`
- [Text And Events](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/GenerationActivity.swift): `Libraries/MLXLMCommon/GenerationActivity.swift`

### Language Runtime

- [LLM Interface](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLLM/LLMModel.swift): `Libraries/MLXLLM/LLMModel.swift`
- [Model Loading](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/Load.swift): `Libraries/MLXLMCommon/Load.swift`
- [Model Factory](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/ModelFactory.swift): `Libraries/MLXLMCommon/ModelFactory.swift`
- [Language Families](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLLM/Models/Llama.swift): `Libraries/MLXLLM/Models/Llama.swift`
- [Tokenizer](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/Tokenizer.swift): `Libraries/MLXLMCommon/Tokenizer.swift`
- [Chat Templates](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/ChatTemplateResolution.swift): `Libraries/MLXLMCommon/ChatTemplateResolution.swift`
- [Batch Generation](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/BatchEngine/BatchEngine.swift): `Libraries/MLXLMCommon/BatchEngine/BatchEngine.swift`
- [Generation Caches](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/Cache/CacheCoordinator.swift): `Libraries/MLXLMCommon/Cache/CacheCoordinator.swift`
- [Reasoning And Tools](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/Tool/ToolCallProcessor.swift): `Libraries/MLXLMCommon/Tool/ToolCallProcessor.swift`
- [Generation Stream](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXLMCommon/GenerationStreamRouting.swift): `Libraries/MLXLMCommon/GenerationStreamRouting.swift`

### Media Models

- [VLM Runtime](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXVLM/VLMModel.swift): `Libraries/MLXVLM/VLMModel.swift`
- [Media Processing](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXVLM/MediaProcessing.swift): `Libraries/MLXVLM/MediaProcessing.swift`
- [Vision And Omni Models](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXVLM/Models/Qwen35.swift): `Libraries/MLXVLM/Models/Qwen35.swift`
- [Embedding Pipeline](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXEmbedders/Model2VecStaticEmbeddingPipeline.swift): `Libraries/MLXEmbedders/Model2VecStaticEmbeddingPipeline.swift`

### Distributed Runtime

- [Cluster Planning](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXDistributedCore/DistributedModePlanner.swift): `Libraries/MLXDistributedCore/DistributedModePlanner.swift`
- [Distributed Collectives](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXDistributedTP/Collectives.swift): `Libraries/MLXDistributedTP/Collectives.swift`
- [Pipeline Transport](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/MLXDistributedTransport/PipelineStageClient.swift): `Libraries/MLXDistributedTransport/PipelineStageClient.swift`

### Image Generation

- [Flux Engine](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/vMLXFlux/FluxEngine.swift): `Libraries/vMLXFlux/FluxEngine.swift`
- [Flux Model Families](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/vMLXFluxModels/Exports.swift): `Libraries/vMLXFluxModels/Exports.swift`
- [Image Events](https://github.com/osaurus-ai/vmlx-swift/blob/main/Libraries/vMLXFluxKit/Requests.swift): `Libraries/vMLXFluxKit/Requests.swift`

### Other components

- **Embedding Application**

## Connections

- Embedding Application → VMLX Facade: imports API
- VMLX Facade → Chat Sessions: exposes chat
- Chat Sessions → LLM Interface: submits turns
- LLM Interface → Model Loading: loads model
- Model Loading → Model Factory: constructs
- Model Factory → Language Families: selects family
- LLM Interface → Tokenizer: encodes input
- LLM Interface → Chat Templates: resolves template
- LLM Interface → Batch Generation: generates
- Batch Generation → Generation Caches: reads and writes
- Batch Generation → Reasoning And Tools: routes output
- Batch Generation → Generation Stream: emits events
- Generation Stream → Text And Events: yields events
- Embedding Application → VLM Runtime: submits media input
- VLM Runtime → Media Processing: prepares media
- VLM Runtime → Vision And Omni Models: runs model
- Embedding Application → Embedding Pipeline: requests vectors
- Embedding Application → Cluster Planning: plans cluster
- Cluster Planning → Distributed Collectives: configures execution
- Cluster Planning → Pipeline Transport: coordinates stages
- Embedding Application → Flux Engine: requests image
- Flux Engine → Flux Model Families: dispatches model
- Flux Engine → Image Events: streams events