How mateocabanal/logan works
Logan is a local-LLM compiler and inference runtime, with logand providing a persistent service interface. The main workflow is model selection or checkpoint input, optional hardware-aware compilation, loading into a model-specific runtime, then token generation returned through an API or chat client. Separate model paths serve sparse Qwen and dense Llama/MiniCPM workloads; shared runtime foundations cover storage, scheduling, state, and caching, while Metal provides Apple acceleration. The README documents additional format, model, and RouteScout capabilities. Some subsystem links rely on README descriptions because their implementation was not sampled.
Main entry points: src/lib.rs Qwen sparse runtime · main.rs Inference daemon · pipeline.rs Model compiler · src/lib.rs Llama dense runtime
Components 21
Service and clients
- Inference daemon
logand/src/main.rs - Interactive chat
logan-chat/src/engine.rs - Daemon settings and jobs
logand/src/main.rs
Compilation and formats
- Model compiler
logan-compiler/src/pipeline.rs - Compiler IR
logan-ir/src/graph.rs - Logan artifacts
logan-artifact/src/lib.rs - COLI compatibility
logan-format/src/package.rs
Inference runtimes
- Qwen sparse runtime
logan-qwen4/src/lib.rs - Llama dense runtime
logan-llama/src/lib.rs - Qwen reference runtime
logan-qwen/src/lib.rs - Spark runtime
logan-spark/src/lib.rs - ANE execution
logan-ane/src/lib.rs - RouteScout prediction
logan-qwen4/src/route_predictor.rs
Runtime foundations
- Runtime foundations
logan-core/src/lib.rs - Weight sources
logan-core/src/storage.rs - Scheduling and residency
logan-core/src/sched/runtime.rs - State and prefix cache
logan-core/src/state/mod.rs
Hardware acceleration
- Metal backend
logan-metal/src/lib.rs
Other components
- API or UI client
- Model source
- Apple hardware
Connections 24
- API or UI client → Inference daemon: sends API requests
- API or UI client → Interactive chat: enters chat
- Interactive chat → Qwen sparse runtime: routes inference
- Interactive chat → Llama dense runtime: routes inference
- Model source → Inference daemon: downloads models
- Inference daemon → Daemon settings and jobs: reads and updates
- Inference daemon → Qwen sparse runtime: routes Qwen
- Inference daemon → Llama dense runtime: routes Llama
- Inference daemon → Model compiler: quantizes models
- Model compiler → Model source: reads checkpoints
- Model compiler → Compiler IR: constructs representation
- Model compiler → Logan artifacts: emits packages
- Model compiler → COLI compatibility: supports legacy format
- Qwen sparse runtime → Weight sources: resolves weights
- Llama dense runtime → Weight sources: resolves weights
- Spark runtime → Weight sources: loads weights
- Qwen sparse runtime → Scheduling and residency: uses scheduling
- Qwen sparse runtime → State and prefix cache: uses prefix state
- Llama dense runtime → State and prefix cache: uses prefix state
- Qwen sparse runtime → Metal backend: submits kernels
- Llama dense runtime → Metal backend: uses dense projections
- ANE execution → Apple hardware: executes on ANE
- Metal backend → Apple hardware: executes on Metal
- RouteScout prediction → Qwen sparse runtime: advises prefetch