How msr327/edge-swarm works
edge-swarm is a peer-to-peer runtime for distributing LLM inference across heterogeneous LAN devices. The documented workflow discovers peers and hardware, assigns contiguous model-layer slices, then runs token generation locally or streams activations to remote workers over gRPC; workers keep per-session KV caches and the final slice samples tokens. The CLI exposes discovery, planning, and worker startup, while the sampled runner provides the generation coordinator. README capabilities such as simulation and hardware support are documented, but their implementation paths are not mapped here. Edges are limited to sampled calls and explicit README relationships.
Main entry points: cli.py CLI Commands · grpc_server.py gRPC Server · pipeline_runner.py Local Pipeline Worker · grpc_client.py gRPC Client
Components 14
CLI and Coordination
- CLI Commands
swarm/cli.py - Pipeline Coordinator
swarm/engine/pipeline_runner.py
Peer Discovery and Scheduling
- Peer Discovery
swarm/discovery/peer_discovery.py - Layer Partitioner
swarm/scheduler/partitioner.py - Pipeline Plan
swarm/scheduler/partitioner.py
Model Execution
- Sliced Transformer
swarm/engine/model_loader.py - Local Pipeline Worker
swarm/engine/pipeline_runner.py - Session KV Cache
swarm/engine/pipeline_runner.py
Tensor Transport
- gRPC Server
swarm/network/grpc_server.py - gRPC Client
swarm/network/grpc_client.py - Tensor Serializer
swarm/network/serializer.py - RPC Messages
protos/swarm.proto
Other components
- Operator
- LAN Peers
Connections 20
- Operator → CLI Commands: invokes
README.md - CLI Commands → Peer Discovery: starts discovery
swarm/cli.py - CLI Commands → Layer Partitioner: calculates plan
swarm/cli.py - CLI Commands → Sliced Transformer: creates model
swarm/cli.py - CLI Commands → Local Pipeline Worker: creates worker
swarm/cli.py - CLI Commands → gRPC Server: starts server
swarm/cli.py - LAN Peers → Peer Discovery: advertises peers
README.md - Peer Discovery → Layer Partitioner: provides nodes
swarm/scheduler/partitioner.py - Layer Partitioner → Pipeline Plan: produces slices
swarm/scheduler/partitioner.py - Pipeline Coordinator → Pipeline Plan: iterates slices
swarm/engine/pipeline_runner.py - Pipeline Coordinator → Local Pipeline Worker: runs local slice
swarm/engine/pipeline_runner.py - Local Pipeline Worker → Sliced Transformer: executes forward
swarm/engine/pipeline_runner.py - Local Pipeline Worker → Session KV Cache: updates cache
swarm/engine/pipeline_runner.py - Pipeline Coordinator → gRPC Client: forwards remote slice
swarm/engine/pipeline_runner.py - gRPC Client → gRPC Server: streams activations
README.md - gRPC Server → Local Pipeline Worker: invokes callback
swarm/network/grpc_server.py - gRPC Client → Tensor Serializer: serializes tensors
swarm/network/grpc_client.py - gRPC Server → Tensor Serializer: encodes tensors
swarm/network/grpc_server.py - gRPC Client → RPC Messages: uses messages
protos/swarm.proto - gRPC Server → RPC Messages: implements service
protos/swarm.proto