How dex-zha/juqe works
JUQE is an on-demand MoE expert-weight loader: the llama.cpp adapter captures routed expert IDs, prepares a runtime batch, and makes loaded weights available in engine-owned pools. Rust validates pack metadata, manages cache slots and worker loads, and performs buffered or Linux direct reads. The CLI separately inspects and verifies packs. Some runtime and adapter implementation files were not sampled, so those subsystem links are documented or omitted rather than inferred as detailed calls.
Main entry points: main.rs CLI commands · routing.cpp Route capture · runtime.rs Runtime lifecycle · pools.cpp Pinned pools
Components 26
Engine adapter
- Route capture
adapters/llama/routing.cpp - Adapter state
adapters/llama/state.h - Tensor layout
adapters/llama/layout.cpp - Pinned pools
adapters/llama/pools.cpp
C ABI and pools
- ABI boundary
crates/juqe-ffi/src/boundary.rs - Runtime creation
crates/juqe-ffi/src/runtime/create.rs - Batch API
crates/juqe-ffi/src/runtime/batch.rs - Pool memory bridge
crates/juqe-ffi/src/memory/region.rs - Pack metadata API
crates/juqe-ffi/src/pack.rs
Loading runtime
- Runtime lifecycle
crates/juqe-runtime/src/runtime.rs - Load dispatch
crates/juqe-runtime/src/dispatch.rs - Batch leases
crates/juqe-runtime/src/batch.rs - LRU cache state
crates/juqe-core/src/cache/state.rs - Load workers
crates/juqe-runtime/src/worker.rs - Expert storage
crates/juqe-runtime/src/storage.rs
Pack and I/O
- Index validation
crates/juqe-core/src/format/parser.rs - Expert reader
crates/juqe-io/src/reader.rs - Read source
crates/juqe-io/src/source.rs - Aligned scratch
crates/juqe-io/src/buffer.rs - Positional reads
crates/juqe-io/src/positional.rs - Linux scatter I/O
crates/juqe-io/src/linux/scatter.rs
Pack tools
- CLI commands
crates/juqe-cli/src/main.rs - Command parsing
crates/juqe-cli/src/args.rs - Pack verification
crates/juqe-cli/src/commands/verify.rs - Pack inspection
crates/juqe-cli/src/commands/inspect.rs
Other components
- llama.cpp engine
Connections 27
- llama.cpp engine → Route capture: invokes callbacks
- Route capture → Adapter state: uses state
adapters/llama/routing.cpp - Route capture → Batch API: prepares batches
adapters/llama/routing.cpp - Route capture → Pinned pools: binds routed slots
- Tensor layout → Adapter state: configures layout
adapters/llama/layout.cpp - Pinned pools → Adapter state: supplies pools
adapters/llama/pools.cpp - ABI boundary → Runtime creation: guards calls
- ABI boundary → Batch API: guards calls
crates/juqe-ffi/src/runtime/batch.rs - Runtime creation → Runtime lifecycle: creates runtime
- Batch API → Runtime lifecycle: prepares batches
- Pool memory bridge → Pinned pools: views memory
- Pack metadata API → Index validation: opens pack index
- Runtime lifecycle → Load dispatch: dispatches loads
- Runtime lifecycle → LRU cache state: manages cache
- Load dispatch → Load workers: queues jobs
- Load workers → Expert storage: loads expert
crates/juqe-runtime/src/worker.rs - Expert storage → Expert reader: reads matrices
- Expert storage → Pinned pools: writes destination slots
- Expert reader → Read source: reads from source
crates/juqe-io/src/reader.rs - Read source → Positional reads: stages buffered reads
crates/juqe-io/src/source.rs - Read source → Linux scatter I/O: attempts direct reads
crates/juqe-io/src/source.rs - Expert reader → Aligned scratch: uses scratch buffer
crates/juqe-io/src/reader.rs - CLI commands → Command parsing: parses arguments
crates/juqe-cli/src/main.rs - CLI commands → Pack inspection: dispatches commands
crates/juqe-cli/src/main.rs - CLI commands → Pack verification: dispatches commands
crates/juqe-cli/src/main.rs - Pack verification → Command parsing: uses verify options
crates/juqe-cli/src/commands/verify.rs - CLI commands → Index validation: inspects validated packs