# exo-explore/exo architecture

> Architecture diagram of the GitHub repository exo-explore/exo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/exo-explore/exo
- Repository: https://github.com/exo-explore/exo

## Overview

1. **Overall shape**
   exo is a multi-process, multi-language local AI cluster system. The main runtime is a Python package (`src/exo`) that acts as the coordinator/control plane and also hosts worker logic. It is complemented by Rust networking/runtime helpers, a Swift macOS app wrapper, and a Svelte dashboard UI.

2. **Control plane vs. worker plane**
   The Python code is split around a clear boundary:
   - `src/exo/master` manages cluster state, placement, and instance lifecycle.
   - `src/exo/worker` runs model execution, loading, batching, and runner supervision.
   - `src/exo/api` exposes external compatibility APIs and command endpoints.
   This separation is important for diagrams: the master decides what should run, while workers actually execute inference.

3. **Cluster coordination and state**
   The master subsystem keeps the authoritative view of the cluster: nodes, instances, topology, and placement decisions. Files under `src/exo/shared/types`, `src/exo/shared/topology.py`, `src/exo/shared/election.py`, and `src/exo/master/placement.py` suggest that cluster state is modeled explicitly and serialized across processes/nodes. The dashboard’s cluster view and `/state`-style API endpoints are fed from this state.

4. **Networking and device discovery**
   Low-level discovery and swarm membership live in Rust (`rust/networking`) and are likely consumed through Python bindings in `rust/exo_rs`. This layer handles device discovery, heartbeats, and cluster connectivity, including the Thunderbolt/RDMA-oriented transport path mentioned in the README. The Python side uses these network events to build a realtime topology and coordinate workers.

5. **Inference execution path**
   Inference flows from API request to runner execution:
   - A request arrives through an API adapter (`chat_completions`, `claude`, `responses`, `ollama`).
   - It is mapped into internal task/command types in `src/exo/shared/types`.
   - The worker runner layer (`src/exo/worker/runner`) executes the request using a selected engine.
   - Streaming results are returned to the client, usually over SSE/streaming HTTP.
   The architecture is request-driven and asynchronous; instance creation and deletion are commands, while generation is a separate streaming path.

6. **Model execution engines**
   The worker supports at least two major engine families:
   - `src/exo/worker/engines/mlx` for text/LLM inference on Apple Silicon/macOS, including batching, cache, distributed execution, and patches.
   - `src/exo/worker/engines/image` for image generation models, with model-specific adapters for Flux and Qwen.
   The README’s macOS GPU support and Linux CPU-only note materially affect deployment diagrams: MLX is the primary backend, while Linux is currently a reduced-capability environment.

7. **Model catalog, downloading, and weight mapping**
   Model lifecycle is split into cataloging, download, and weight application:
   - `src/exo/shared/models/model_cards.py` and the `resources/*_model_cards` TOML files define known models and presets.
   - `src/exo/download/*` handles Hugging Face resolution, shard downloads, retries, verification, offline mode, and cancellation.
   - `src/exo/worker/engines/*/weights` and shared weight-mapping code convert model-specific checkpoints into runtime-ready tensors.
   This is a major architectural boundary: model metadata and storage are separate from the runtime execution graph.

8. **Placement, topology, and distributed planning**
   The cluster is topology-aware. `src/exo/master/placement.py`, `src/exo/master/placement_utils.py`, and `src/exo/shared/topology.py` imply that exo evaluates node memory, connectivity, and latency before deciding how to shard a model. The README’s “preview placements” endpoint reflects this design: placement is computed before instance creation, not hardcoded. Tensor parallelism and pipeline-style distribution are first-class concepts.

9. **Shared types and eventing**
   Most cross-cutting contracts are centralized in `src/exo/shared/types` and utility modules like `src/exo/utils/event_buffer.py`, `channels.py`, `reactive.py`, and `disk_event_log.py`. These provide the message/event vocabulary for commands, tasks, worker status, topology updates, and streamed tokens. This is the glue between master, worker, API, and UI.

10. **External API compatibility layer**
   `src/exo/api` exposes multiple front doors that all map onto the same internal execution model:
   - OpenAI Chat Completions
   - Anthropic Claude Messages
   - OpenAI Responses
   - Ollama
   This adapter layer is a key diagram boundary: external clients speak different schemas, but they converge on the same instance/task/streaming pipeline.

11. **User-facing surfaces**
   There are two primary user interfaces:
   - The Svelte dashboard in `dashboard/`, which provides cluster, model, download, trace, and chat views.
   - The macOS Swift app in `app/EXO/`, which manages launch, network setup, settings, and uninstall behavior.
   The dashboard is the interactive control surface; the app is the native system-integrated shell for macOS.

12. **Platform integration and packaging**
   The repo includes substantial platform-specific support:
   - Rust crates for networking/runtime helpers.
   - Swift app packaging and macOS network/location setup.
   - Nix, PyPI/uv, and PyInstaller/dmg packaging.
   - Typing stubs under `.typings/` for MLX, mlx-lm, safetensors, and internal modules.
   This indicates exo is not a single deployment artifact, but a platformed system with native, Python, and JS components.

13. **Benchmarks, tests, and support tooling**
   `bench/`, `tests/`, and the various `src/exo/*/tests` directories show that performance evaluation, cluster resilience, API compatibility, and distributed correctness are treated as first-class concerns. `tools/` contains an auxiliary client/harness package, while `tmp/` holds operational scripts and historical experiments.

14. **High-level data flow to diagram**
   The most accurate architecture diagram would show:
   external client or dashboard → Python API layer → master/placement/state → worker/runner/engine → model downloads/weights → streamed response back to client.
   In parallel, discovery/networking feeds the master with node presence and topology, and the Swift app / Rust layer handles system and transport concerns underneath the Python control plane.

Main entry points: [`src/exo/master/main.py`](https://github.com/exo-explore/exo/blob/main/src/exo/master/main.py) (Master), [`src/exo/worker/runner/runner.py`](https://github.com/exo-explore/exo/blob/main/src/exo/worker/runner/runner.py) (Runner), [`src/exo/api/main.py`](https://github.com/exo-explore/exo/blob/main/src/exo/api/main.py) (API server), [`src/exo/main.py`](https://github.com/exo-explore/exo/blob/main/src/exo/main.py) (exo entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_python["Python runtime"]
  node_src_exo_main(("exo entry<br/>python entrypoint<br/>[main.py]"))
  node_api_main["API server<br/>HTTP control plane<br/>[main.py]"]
  node_api_adapters["API adapters<br/>compat adapters"]
  node_master_main["Master<br/>cluster coordinator<br/>[main.py]"]
  node_placement{{"Placement<br/>topology planner<br/>[placement.py]"}}
  node_shared_types["Shared types<br/>contracts"]
  node_topology["Topology<br/>cluster model<br/>[topology.py]"]
  node_eventing["Event bus<br/>streaming utils<br/>[event_buffer.py]"]
  node_worker_main["Worker<br/>execution node<br/>[main.py]"]
  node_runner["Runner<br/>inference supervisor<br/>[runner.py]"]
  node_mlx_engine{{"MLX engine<br/>LLM backend"}}
  node_image_engine{{"Image engine<br/>image backend"}}
  node_download["Downloads<br/>model fetcher"]
  node_model_cards["Model catalog<br/>model metadata<br/>[model_cards.py]"]
  node_routing["Routing<br/>message router<br/>[router.py]"]
end

subgraph group_native["Native layers"]
  node_rust_net{{"Rust networking<br/>swarm transport"}}
  node_rust_bindings["Rust bindings<br/>python-native bridge<br/>[lib.rs]"]
end

subgraph group_ui["User surfaces"]
  node_dashboard["Dashboard<br/>Svelte UI"]
  node_swift_app["Mac app<br/>Swift shell"]
end

subgraph group_support["Support tooling"]
  node_bench_tools["Bench/tools<br/>eval tooling"]
end

node_src_exo_main -->|"starts"| node_api_main
node_src_exo_main -->|"starts"| node_master_main
node_src_exo_main -->|"starts"| node_worker_main
node_api_main -->|"maps requests"| node_api_adapters
node_api_adapters -->|"emits"| node_shared_types
node_api_main -->|"routes"| node_routing
node_master_main -->|"uses"| node_shared_types
node_master_main -->|"tracks"| node_topology
node_master_main -->|"plans"| node_placement
node_placement -->|"selects"| node_model_cards
node_worker_main -->|"supervises"| node_runner
node_runner -->|"executes"| node_mlx_engine
node_runner -->|"executes"| node_image_engine
node_runner -->|"loads models"| node_download
node_download -->|"resolves"| node_model_cards
node_rust_net -->|"exposed via"| node_rust_bindings
node_rust_bindings -->|"feeds"| node_topology
node_rust_net -->|"signals"| node_master_main
node_dashboard -->|"consumes"| node_api_main
node_swift_app -->|"uses"| node_rust_bindings
node_swift_app -.->|"opens"| node_dashboard
node_bench_tools -.->|"evaluates"| node_api_main
node_bench_tools -.->|"measures"| node_runner

click node_src_exo_main "https://github.com/exo-explore/exo/blob/main/src/exo/main.py"
click node_api_main "https://github.com/exo-explore/exo/blob/main/src/exo/api/main.py"
click node_api_adapters "https://github.com/exo-explore/exo/tree/main/src/exo/api/adapters"
click node_master_main "https://github.com/exo-explore/exo/blob/main/src/exo/master/main.py"
click node_placement "https://github.com/exo-explore/exo/blob/main/src/exo/master/placement.py"
click node_shared_types "https://github.com/exo-explore/exo/tree/main/src/exo/shared/types"
click node_topology "https://github.com/exo-explore/exo/blob/main/src/exo/shared/topology.py"
click node_eventing "https://github.com/exo-explore/exo/blob/main/src/exo/utils/event_buffer.py"
click node_worker_main "https://github.com/exo-explore/exo/blob/main/src/exo/worker/main.py"
click node_runner "https://github.com/exo-explore/exo/blob/main/src/exo/worker/runner/runner.py"
click node_mlx_engine "https://github.com/exo-explore/exo/tree/main/src/exo/worker/engines/mlx"
click node_image_engine "https://github.com/exo-explore/exo/tree/main/src/exo/worker/engines/image"
click node_download "https://github.com/exo-explore/exo/tree/main/src/exo/download"
click node_model_cards "https://github.com/exo-explore/exo/blob/main/src/exo/shared/models/model_cards.py"
click node_routing "https://github.com/exo-explore/exo/blob/main/src/exo/routing/router.py"
click node_rust_net "https://github.com/exo-explore/exo/tree/main/rust/networking/src"
click node_rust_bindings "https://github.com/exo-explore/exo/blob/main/rust/exo_rs/src/lib.rs"
click node_dashboard "https://github.com/exo-explore/exo/tree/main/dashboard/src/routes"
click node_swift_app "https://github.com/exo-explore/exo/tree/main/app/EXO/EXO"
click node_bench_tools "https://github.com/exo-explore/exo/tree/main/bench"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_src_exo_main,node_api_main,node_api_adapters,node_master_main,node_placement,node_shared_types,node_topology,node_eventing,node_worker_main,node_runner,node_mlx_engine,node_image_engine,node_download,node_model_cards,node_routing toneBlue
class node_rust_net,node_rust_bindings toneAmber
class node_dashboard,node_swift_app toneMint
class node_bench_tools toneRose
```

## Components

### Python runtime

Coordinator, APIs, worker logic, and shared contracts

- [exo entry](https://github.com/exo-explore/exo/blob/main/src/exo/main.py): python entrypoint · Top-level package launcher and process bootstrap · `src/exo/main.py`
- [API server](https://github.com/exo-explore/exo/blob/main/src/exo/api/main.py): HTTP control plane · External API front door and command/stream handling · `src/exo/api/main.py`
- [API adapters](https://github.com/exo-explore/exo/tree/main/src/exo/api/adapters): compat adapters · OpenAI, Anthropic, Ollama, and Responses request translation · `src/exo/api/adapters`
- [Master](https://github.com/exo-explore/exo/blob/main/src/exo/master/main.py): cluster coordinator · Authoritative cluster state, lifecycle, and orchestration · `src/exo/master/main.py`
- [Placement](https://github.com/exo-explore/exo/blob/main/src/exo/master/placement.py): topology planner · Topology-aware model placement and instance allocation · `src/exo/master/placement.py`
- [Shared types](https://github.com/exo-explore/exo/tree/main/src/exo/shared/types): contracts · Commands, tasks, state, topology, and worker-facing schemas · `src/exo/shared/types`
- [Topology](https://github.com/exo-explore/exo/blob/main/src/exo/shared/topology.py): cluster model · Node presence, connectivity, memory, and election state · `src/exo/shared/topology.py`
- [Event bus](https://github.com/exo-explore/exo/blob/main/src/exo/utils/event_buffer.py): streaming utils · Buffered events, channels, reactive state, and disk event log · `src/exo/utils/event_buffer.py`
- [Worker](https://github.com/exo-explore/exo/blob/main/src/exo/worker/main.py): execution node · Worker process that hosts model execution and supervision · `src/exo/worker/main.py`
- [Runner](https://github.com/exo-explore/exo/blob/main/src/exo/worker/runner/runner.py): inference supervisor · Task execution, lifecycle management, diagnostics, and streaming results · `src/exo/worker/runner/runner.py`
- [MLX engine](https://github.com/exo-explore/exo/tree/main/src/exo/worker/engines/mlx): LLM backend · Apple Silicon inference engine with batching, cache, and distributed support · `src/exo/worker/engines/mlx`
- [Image engine](https://github.com/exo-explore/exo/tree/main/src/exo/worker/engines/image): image backend · Flux and Qwen image generation pipeline and model adapters · `src/exo/worker/engines/image`
- [Downloads](https://github.com/exo-explore/exo/tree/main/src/exo/download): model fetcher · Hugging Face resolution, shard download, verification, offline mode, and cancellation · `src/exo/download`
- [Model catalog](https://github.com/exo-explore/exo/blob/main/src/exo/shared/models/model_cards.py): model metadata · Known models and presets used by the coordinator and worker · `src/exo/shared/models/model_cards.py`
- [Routing](https://github.com/exo-explore/exo/blob/main/src/exo/routing/router.py): message router · Topic routing and connection message handling between processes · `src/exo/routing/router.py`

### Native layers

Rust networking/runtime helpers and Swift macOS shell

- [Rust networking](https://github.com/exo-explore/exo/tree/main/rust/networking/src): swarm transport · Discovery, swarm membership, heartbeats, and low-level cluster connectivity · `rust/networking/src`
- [Rust bindings](https://github.com/exo-explore/exo/blob/main/rust/exo_rs/src/lib.rs): python-native bridge · Bindings and helpers exposed to Python for networking/runtime support · `rust/exo_rs/src/lib.rs`

### User surfaces

Dashboard and macOS app

- [Dashboard](https://github.com/exo-explore/exo/tree/main/dashboard/src/routes): Svelte UI · Cluster, download, trace, chat, and topology views · `dashboard/src/routes`
- [Mac app](https://github.com/exo-explore/exo/tree/main/app/EXO/EXO): Swift shell · Native macOS wrapper for launch, network setup, status, and settings · `app/EXO/EXO`

### Support tooling

Benchmarks, scripts, and packaging

- [Bench/tools](https://github.com/exo-explore/exo/tree/main/bench): eval tooling · Benchmark harnesses, scripts, and operational utilities · `bench`

## Connections

- exo entry → API server: starts · Launches the API/control-plane process
- exo entry → Master: starts · Launches cluster coordinator logic
- exo entry → Worker: starts · Launches worker execution process
- API server → API adapters: maps requests · Normalizes external API schemas into internal commands
- API adapters → Shared types: emits · Converts client payloads into shared commands and tasks
- API server → Routing: routes · Dispatches commands and streams events through internal routing
- Master → Shared types: uses · Reads and writes cluster state and command schemas
- Master → Topology: tracks · Maintains authoritative cluster topology and node state
- Master → Placement: plans · Computes placement before instance creation
- Placement → Model catalog: selects · Uses model metadata and presets to choose layout
- Worker → Runner: supervises · Hosts execution lifecycle and task forwarding
- Runner → MLX engine: executes · Runs text inference and generation on MLX
- Runner → Image engine: executes · Runs image generation workflows
- Runner → Downloads: loads models · Triggers model fetch and verification when needed
- Downloads → Model catalog: resolves · Uses catalog metadata to locate and validate weights
- Rust networking → Rust bindings: exposed via · Networking/runtime helpers are surfaced to Python
- Rust bindings → Topology: feeds · Provides discovery and membership signals to cluster state
- Rust networking → Master: signals · Discovery and heartbeats inform coordinator state
- Dashboard → API server: consumes · Reads state and sends control requests through the API
- Mac app → Rust bindings: uses · Native shell relies on Rust helpers for process and network setup
- Mac app → Dashboard: opens · Can launch or surface the dashboard experience
- Bench/tools → API server: evaluates · Exercises API, streaming, and cluster behavior
- Bench/tools → Runner: measures · Benchmarks worker throughput and latency
