# kaist-ina/specedge architecture

> Architecture diagram of the GitHub repository kaist-ina/specedge, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kaist-ina/specedge
- Repository: https://github.com/kaist-ina/specedge

## Overview

SpecEdge serves interactive LLM requests by drafting speculative token trees on edge clients and validating them on a target-model server; the server returns selections that the client uses to continue generation. The repository also provides proactive drafting, auto-batch, server-only, and layer-split experiment modes, plus metric calculators. Edges reflect README statements or the supplied source index and excerpts; several model and orchestration implementations are indexed but not excerpted, so their internal behavior is represented conservatively.

Main entry points: [`src/strategy/server_verify/specexec/grpc.py`](https://github.com/kaist-ina/specedge/blob/main/src/strategy/server_verify/specexec/grpc.py) (Server Validator), [`src/specedge/engine/graph.py`](https://github.com/kaist-ina/specedge/blob/main/src/specedge/engine/graph.py) (Graph Engine), [`src/specedge/client/specexec.py`](https://github.com/kaist-ina/specedge/blob/main/src/specedge/client/specexec.py) (Speculative Client), [`src/strategy/edge_draft/specexec.py`](https://github.com/kaist-ina/specedge/blob/main/src/strategy/edge_draft/specexec.py) (Edge Drafting)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_serving["SpecEdge Serving"]
  node_client_host["Edge Host Launcher<br/>[client_host.py]"]
  node_client["Edge Client<br/>[client.py]"]
  node_spec_client["Speculative Client<br/>[specexec.py]"]
  node_proactive["Proactive Drafter<br/>[proactive.py]"]
  node_rpc_client["gRPC Client<br/>[grpc.py]"]
  node_server["Batch Server<br/>[batch_server.py]"]
  node_server_validator["Server Validator<br/>[grpc.py]"]
  node_grpc_service["gRPC Service"]
end

subgraph group_execution["Inference Execution"]
  node_request_data["Prompt Datasets<br/>[util.py]"]
  node_config["Runtime Configuration<br/>[config.py]"]
  node_tree["Speculation Trees<br/>[tree.py]"]
  node_request_manager["Request Manager<br/>[request_manager.py]"]
  node_edge_draft["Edge Drafting<br/>[specexec.py]"]
  node_edge_verify["Edge Verification<br/>[specexec.py]"]
  node_inference_engine["Graph Engine<br/>[graph.py]"]
  node_kv_cache[("KV Cache<br/>[cache.py]")]
  node_model_adapters["Model Adapters"]
end

subgraph group_baselines["Serving Baselines"]
  node_auto_batch["Auto Batch<br/>[auto_batch.py]"]
  node_server_only["Server-Only Runner<br/>[server_only.py]"]
  node_server_only_verify["Server-Only Inference<br/>[server_only.py]"]
  node_layer_split_head["Layer-Split Head"]
  node_layer_split_tail["Layer-Split Tail"]
end

subgraph group_analysis["Metrics and Analysis"]
  node_metrics["Experiment Metrics"]
  node_results["Result Files"]
end

node_user(("Request User"))

node_user -->|"initiates run"| node_client_host
node_client_host -->|"launches clients"| node_client
node_client -->|"runs decoding"| node_spec_client
node_spec_client -.->|"uses optionally"| node_proactive
node_spec_client -.->|"updates tree"| node_tree
node_spec_client -->|"sends validation"| node_rpc_client
node_rpc_client -->|"calls RPC"| node_grpc_service
node_server -->|"registers service"| node_grpc_service
node_server -->|"starts validator"| node_server_validator
node_server_validator -->|"runs inference"| node_inference_engine
node_server_validator -->|"loads prompts"| node_request_data
node_server_validator -->|"reads settings"| node_config
node_server_validator -.->|"coordinates drafting"| node_edge_draft
node_server_validator -.->|"validates selections"| node_edge_verify
node_edge_draft -->|"forwards candidates"| node_inference_engine
node_edge_draft -->|"adds candidates"| node_tree
node_edge_draft -->|"tracks requests"| node_request_manager
node_edge_verify -->|"forwards candidates"| node_inference_engine
node_edge_verify -->|"updates selection"| node_tree
node_inference_engine -->|"uses cache"| node_kv_cache
node_model_adapters -->|"updates cache"| node_kv_cache
node_auto_batch -->|"runs batching"| node_inference_engine
node_server_only -->|"runs baseline"| node_server_only_verify
node_server_only_verify -->|"runs model"| node_inference_engine
node_layer_split_head -->|"uses split model"| node_model_adapters
node_layer_split_tail -->|"uses split model"| node_model_adapters
node_user -->|"runs analysis"| node_metrics
node_metrics -->|"reads JSONL"| node_results

click node_client_host "https://github.com/kaist-ina/specedge/blob/main/src/script/client_host.py"
click node_client "https://github.com/kaist-ina/specedge/blob/main/src/script/client.py"
click node_spec_client "https://github.com/kaist-ina/specedge/blob/main/src/specedge/client/specexec.py"
click node_proactive "https://github.com/kaist-ina/specedge/blob/main/src/specedge/client/proactive.py"
click node_rpc_client "https://github.com/kaist-ina/specedge/blob/main/src/specedge/network/grpc.py"
click node_server "https://github.com/kaist-ina/specedge/blob/main/src/script/batch_server.py"
click node_server_validator "https://github.com/kaist-ina/specedge/blob/main/src/strategy/server_verify/specexec/grpc.py"
click node_grpc_service "https://github.com/kaist-ina/specedge/blob/main/src/specedge_grpc/specedge_pb2_grpc.py"
click node_request_data "https://github.com/kaist-ina/specedge/blob/main/src/util.py"
click node_config "https://github.com/kaist-ina/specedge/blob/main/src/config.py"
click node_tree "https://github.com/kaist-ina/specedge/blob/main/src/specedge/tree.py"
click node_request_manager "https://github.com/kaist-ina/specedge/blob/main/src/strategy/request_manager.py"
click node_edge_draft "https://github.com/kaist-ina/specedge/blob/main/src/strategy/edge_draft/specexec.py"
click node_edge_verify "https://github.com/kaist-ina/specedge/blob/main/src/strategy/edge_verify/specexec.py"
click node_inference_engine "https://github.com/kaist-ina/specedge/blob/main/src/specedge/engine/graph.py"
click node_kv_cache "https://github.com/kaist-ina/specedge/blob/main/src/model/cache.py"
click node_model_adapters "https://github.com/kaist-ina/specedge/tree/main/src/model"
click node_auto_batch "https://github.com/kaist-ina/specedge/blob/main/src/script/auto_batch.py"
click node_server_only "https://github.com/kaist-ina/specedge/blob/main/src/script/server_only.py"
click node_server_only_verify "https://github.com/kaist-ina/specedge/blob/main/src/strategy/server_verify/specexec/server_only.py"
click node_layer_split_head "https://github.com/kaist-ina/specedge/blob/main/src/script/layer_split_head.py"
click node_layer_split_tail "https://github.com/kaist-ina/specedge/blob/main/src/script/layer_split_tail.py"
click node_metrics "https://github.com/kaist-ina/specedge/tree/main/src/metric"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_client_host,node_client,node_spec_client,node_proactive,node_rpc_client,node_server,node_server_validator,node_grpc_service,node_user toneBlue
class node_request_data,node_config,node_tree,node_request_manager,node_edge_draft,node_edge_verify,node_inference_engine,node_kv_cache,node_model_adapters toneAmber
class node_auto_batch,node_server_only,node_server_only_verify,node_layer_split_head,node_layer_split_tail toneMint
class node_metrics,node_results toneRose
```

## Components

### SpecEdge Serving

- [Edge Host Launcher](https://github.com/kaist-ina/specedge/blob/main/src/script/client_host.py): `src/script/client_host.py`
- [Edge Client](https://github.com/kaist-ina/specedge/blob/main/src/script/client.py): `src/script/client.py`
- [Speculative Client](https://github.com/kaist-ina/specedge/blob/main/src/specedge/client/specexec.py): `src/specedge/client/specexec.py`
- [Proactive Drafter](https://github.com/kaist-ina/specedge/blob/main/src/specedge/client/proactive.py): `src/specedge/client/proactive.py`
- [gRPC Client](https://github.com/kaist-ina/specedge/blob/main/src/specedge/network/grpc.py): `src/specedge/network/grpc.py`
- [Batch Server](https://github.com/kaist-ina/specedge/blob/main/src/script/batch_server.py): `src/script/batch_server.py`
- [Server Validator](https://github.com/kaist-ina/specedge/blob/main/src/strategy/server_verify/specexec/grpc.py): `src/strategy/server_verify/specexec/grpc.py`
- [gRPC Service](https://github.com/kaist-ina/specedge/blob/main/src/specedge_grpc/specedge_pb2_grpc.py): `src/specedge_grpc/specedge_pb2_grpc.py`

### Inference Execution

- [Prompt Datasets](https://github.com/kaist-ina/specedge/blob/main/src/util.py): `src/util.py`
- [Runtime Configuration](https://github.com/kaist-ina/specedge/blob/main/src/config.py): `src/config.py`
- [Speculation Trees](https://github.com/kaist-ina/specedge/blob/main/src/specedge/tree.py): `src/specedge/tree.py`
- [Request Manager](https://github.com/kaist-ina/specedge/blob/main/src/strategy/request_manager.py): `src/strategy/request_manager.py`
- [Edge Drafting](https://github.com/kaist-ina/specedge/blob/main/src/strategy/edge_draft/specexec.py): `src/strategy/edge_draft/specexec.py`
- [Edge Verification](https://github.com/kaist-ina/specedge/blob/main/src/strategy/edge_verify/specexec.py): `src/strategy/edge_verify/specexec.py`
- [Graph Engine](https://github.com/kaist-ina/specedge/blob/main/src/specedge/engine/graph.py): `src/specedge/engine/graph.py`
- [KV Cache](https://github.com/kaist-ina/specedge/blob/main/src/model/cache.py): `src/model/cache.py`
- [Model Adapters](https://github.com/kaist-ina/specedge/tree/main/src/model): `src/model`

### Serving Baselines

- [Auto Batch](https://github.com/kaist-ina/specedge/blob/main/src/script/auto_batch.py): `src/script/auto_batch.py`
- [Server-Only Runner](https://github.com/kaist-ina/specedge/blob/main/src/script/server_only.py): `src/script/server_only.py`
- [Server-Only Inference](https://github.com/kaist-ina/specedge/blob/main/src/strategy/server_verify/specexec/server_only.py): `src/strategy/server_verify/specexec/server_only.py`
- [Layer-Split Head](https://github.com/kaist-ina/specedge/blob/main/src/script/layer_split_head.py): `src/script/layer_split_head.py`
- [Layer-Split Tail](https://github.com/kaist-ina/specedge/blob/main/src/script/layer_split_tail.py): `src/script/layer_split_tail.py`

### Metrics and Analysis

- [Experiment Metrics](https://github.com/kaist-ina/specedge/tree/main/src/metric): `src/metric`
- **Result Files**

### Other components

- **Request User**

## Connections

- Request User → Edge Host Launcher: initiates run (evidence: `README.md`)
- Edge Host Launcher → Edge Client: launches clients (evidence: `README.md`)
- Edge Client → Speculative Client: runs decoding (evidence: `src/script/client.py`)
- Speculative Client → Proactive Drafter: uses optionally (evidence: `src/specedge/client/specexec.py`)
- Speculative Client → Speculation Trees: updates tree (evidence: `src/specedge/client/specexec.py`)
- Speculative Client → gRPC Client: sends validation (evidence: `src/specedge/client/specexec.py`)
- gRPC Client → gRPC Service: calls RPC (evidence: `src/specedge/network/grpc.py`)
- Batch Server → gRPC Service: registers service (evidence: `src/script/batch_server.py`)
- Batch Server → Server Validator: starts validator (evidence: `src/script/batch_server.py`)
- Server Validator → Graph Engine: runs inference (evidence: `src/strategy/server_verify/specexec/grpc.py`)
- Server Validator → Prompt Datasets: loads prompts (evidence: `src/strategy/server_verify/specexec/grpc.py`)
- Server Validator → Runtime Configuration: reads settings (evidence: `src/strategy/server_verify/specexec/grpc.py`)
- Server Validator → Edge Drafting: coordinates drafting
- Server Validator → Edge Verification: validates selections
- Edge Drafting → Graph Engine: forwards candidates (evidence: `src/strategy/edge_draft/specexec.py`)
- Edge Drafting → Speculation Trees: adds candidates (evidence: `src/strategy/edge_draft/specexec.py`)
- Edge Drafting → Request Manager: tracks requests (evidence: `src/strategy/edge_draft/specexec.py`)
- Edge Verification → Graph Engine: forwards candidates (evidence: `src/strategy/edge_verify/specexec.py`)
- Edge Verification → Speculation Trees: updates selection (evidence: `src/strategy/edge_verify/specexec.py`)
- Graph Engine → KV Cache: uses cache (evidence: `src/specedge/engine/graph.py`)
- Model Adapters → KV Cache: updates cache (evidence: `src/model/llama.py`)
- Auto Batch → Graph Engine: runs batching (evidence: `src/script/auto_batch.py`)
- Server-Only Runner → Server-Only Inference: runs baseline (evidence: `src/script/server_only.py`)
- Server-Only Inference → Graph Engine: runs model (evidence: `src/strategy/server_verify/specexec/server_only.py`)
- Layer-Split Head → Model Adapters: uses split model (evidence: `src/script/layer_split_head.py`)
- Layer-Split Tail → Model Adapters: uses split model (evidence: `src/script/layer_split_tail.py`)
- Request User → Experiment Metrics: runs analysis (evidence: `README.md`)
- Experiment Metrics → Result Files: reads JSONL (evidence: `README.md`)
