How kaist-ina/specedge works
SpecEdge serves interactive LLM requests by drafting speculative token trees on edge clients and validating them on a target-model server; the server returns selections that the client uses to continue generation. The repository also provides proactive drafting, auto-batch, server-only, and layer-split experiment modes, plus metric calculators. Edges reflect README statements or the supplied source index and excerpts; several model and orchestration implementations are indexed but not excerpted, so their internal behavior is represented conservatively.
Main entry points: grpc.py Server Validator · graph.py Graph Engine · client/specexec.py Speculative Client · edge_draft/specexec.py Edge Drafting
Components 25
SpecEdge Serving
- Edge Host Launcher
src/script/client_host.py - Edge Client
src/script/client.py - Speculative Client
src/specedge/client/specexec.py - Proactive Drafter
src/specedge/client/proactive.py - gRPC Client
src/specedge/network/grpc.py - Batch Server
src/script/batch_server.py - Server Validator
src/strategy/server_verify/specexec/grpc.py - gRPC Service
src/specedge_grpc/specedge_pb2_grpc.py
Inference Execution
- Prompt Datasets
src/util.py - Runtime Configuration
src/config.py - Speculation Trees
src/specedge/tree.py - Request Manager
src/strategy/request_manager.py - Edge Drafting
src/strategy/edge_draft/specexec.py - Edge Verification
src/strategy/edge_verify/specexec.py - Graph Engine
src/specedge/engine/graph.py - KV Cache
src/model/cache.py - Model Adapters
src/model
Serving Baselines
- Auto Batch
src/script/auto_batch.py - Server-Only Runner
src/script/server_only.py - Server-Only Inference
src/strategy/server_verify/specexec/server_only.py - Layer-Split Head
src/script/layer_split_head.py - Layer-Split Tail
src/script/layer_split_tail.py
Metrics and Analysis
- Experiment Metrics
src/metric - Result Files
Other components
- Request User
Connections 28
- Request User → Edge Host Launcher: initiates run
README.md - Edge Host Launcher → Edge Client: launches clients
README.md - Edge Client → Speculative Client: runs decoding
src/script/client.py - Speculative Client → Proactive Drafter: uses optionally
src/specedge/client/specexec.py - Speculative Client → Speculation Trees: updates tree
src/specedge/client/specexec.py - Speculative Client → gRPC Client: sends validation
src/specedge/client/specexec.py - gRPC Client → gRPC Service: calls RPC
src/specedge/network/grpc.py - Batch Server → gRPC Service: registers service
src/script/batch_server.py - Batch Server → Server Validator: starts validator
src/script/batch_server.py - Server Validator → Graph Engine: runs inference
src/strategy/server_verify/specexec/grpc.py - Server Validator → Prompt Datasets: loads prompts
src/strategy/server_verify/specexec/grpc.py - Server Validator → Runtime Configuration: reads settings
src/strategy/server_verify/specexec/grpc.py - Server Validator → Edge Drafting: coordinates drafting
- Server Validator → Edge Verification: validates selections
- Edge Drafting → Graph Engine: forwards candidates
src/strategy/edge_draft/specexec.py - Edge Drafting → Speculation Trees: adds candidates
src/strategy/edge_draft/specexec.py - Edge Drafting → Request Manager: tracks requests
src/strategy/edge_draft/specexec.py - Edge Verification → Graph Engine: forwards candidates
src/strategy/edge_verify/specexec.py - Edge Verification → Speculation Trees: updates selection
src/strategy/edge_verify/specexec.py - Graph Engine → KV Cache: uses cache
src/specedge/engine/graph.py - Model Adapters → KV Cache: updates cache
src/model/llama.py - Auto Batch → Graph Engine: runs batching
src/script/auto_batch.py - Server-Only Runner → Server-Only Inference: runs baseline
src/script/server_only.py - Server-Only Inference → Graph Engine: runs model
src/strategy/server_verify/specexec/server_only.py - Layer-Split Head → Model Adapters: uses split model
src/script/layer_split_head.py - Layer-Split Tail → Model Adapters: uses split model
src/script/layer_split_tail.py - Request User → Experiment Metrics: runs analysis
README.md - Experiment Metrics → Result Files: reads JSONL
README.md