# dex-zha/juqe architecture

> Architecture diagram of the GitHub repository dex-zha/juqe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dex-zha/juqe
- Repository: https://github.com/dex-zha/juqe

## Overview

JUQE is an on-demand MoE expert-weight loader: the llama.cpp adapter captures routed expert IDs, prepares a runtime batch, and makes loaded weights available in engine-owned pools. Rust validates pack metadata, manages cache slots and worker loads, and performs buffered or Linux direct reads. The CLI separately inspects and verifies packs. Some runtime and adapter implementation files were not sampled, so those subsystem links are documented or omitted rather than inferred as detailed calls.

Main entry points: [`crates/juqe-cli/src/main.rs`](https://github.com/dex-zha/juqe/blob/main/crates/juqe-cli/src/main.rs) (CLI commands), [`adapters/llama/routing.cpp`](https://github.com/dex-zha/juqe/blob/main/adapters/llama/routing.cpp) (Route capture), [`crates/juqe-runtime/src/runtime.rs`](https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/runtime.rs) (Runtime lifecycle), [`adapters/llama/pools.cpp`](https://github.com/dex-zha/juqe/blob/main/adapters/llama/pools.cpp) (Pinned pools)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_adapter["Engine adapter"]
  node_routing["Route capture<br/>[routing.cpp]"]
  node_adapter_state["Adapter state<br/>[state.h]"]
  node_tensor_layout["Tensor layout<br/>[layout.cpp]"]
  node_engine_pools[("Pinned pools<br/>[pools.cpp]")]
end

subgraph group_abi["C ABI and pools"]
  node_ffi_boundary["ABI boundary<br/>[boundary.rs]"]
  node_ffi_create["Runtime creation<br/>[create.rs]"]
  node_ffi_batch["Batch API<br/>[batch.rs]"]
  node_ffi_memory["Pool memory bridge<br/>[region.rs]"]
  node_pack_handle["Pack metadata API<br/>[pack.rs]"]
end

subgraph group_runtime["Loading runtime"]
  node_runtime_api["Runtime lifecycle<br/>[runtime.rs]"]
  node_dispatch["Load dispatch<br/>[dispatch.rs]"]
  node_batch_leases["Batch leases<br/>[batch.rs]"]
  node_cache_state["LRU cache state<br/>[state.rs]"]
  node_worker["Load workers<br/>[worker.rs]"]
  node_storage["Expert storage<br/>[storage.rs]"]
end

subgraph group_io["Pack and I/O"]
  node_format_parser["Index validation<br/>[parser.rs]"]
  node_reader["Expert reader<br/>[reader.rs]"]
  node_source["Read source<br/>[source.rs]"]
  node_aligned_buffer["Aligned scratch<br/>[buffer.rs]"]
  node_positional["Positional reads<br/>[positional.rs]"]
  node_scatter["Linux scatter I/O<br/>[scatter.rs]"]
end

subgraph group_tools["Pack tools"]
  node_cli["CLI commands<br/>[main.rs]"]
  node_cli_args["Command parsing<br/>[args.rs]"]
  node_verify["Pack verification<br/>[verify.rs]"]
  node_inspect["Pack inspection<br/>[inspect.rs]"]
end

node_llama["llama.cpp engine"]

node_llama -->|"invokes callbacks"| node_routing
node_routing -->|"uses state"| node_adapter_state
node_routing -->|"prepares batches"| node_ffi_batch
node_routing -.->|"binds routed slots"| node_engine_pools
node_tensor_layout -->|"configures layout"| node_adapter_state
node_engine_pools -->|"supplies pools"| node_adapter_state
node_ffi_boundary -->|"guards calls"| node_ffi_create
node_ffi_boundary -->|"guards calls"| node_ffi_batch
node_ffi_create -.->|"creates runtime"| node_runtime_api
node_ffi_batch -.->|"prepares batches"| node_runtime_api
node_ffi_memory -.->|"views memory"| node_engine_pools
node_pack_handle -.->|"opens pack index"| node_format_parser
node_runtime_api -.->|"dispatches loads"| node_dispatch
node_runtime_api -.->|"manages cache"| node_cache_state
node_dispatch -.->|"queues jobs"| node_worker
node_worker -->|"loads expert"| node_storage
node_storage -->|"reads matrices"| node_reader
node_storage -.->|"writes destination slots"| node_engine_pools
node_reader -->|"reads from source"| node_source
node_source -->|"stages buffered reads"| node_positional
node_source -->|"attempts direct reads"| node_scatter
node_reader -->|"uses scratch buffer"| node_aligned_buffer
node_cli -->|"parses arguments"| node_cli_args
node_cli -->|"dispatches commands"| node_inspect
node_cli -->|"dispatches commands"| node_verify
node_verify -->|"uses verify options"| node_cli_args
node_cli -.->|"inspects validated packs"| node_format_parser

click node_routing "https://github.com/dex-zha/juqe/blob/main/adapters/llama/routing.cpp"
click node_adapter_state "https://github.com/dex-zha/juqe/blob/main/adapters/llama/state.h"
click node_tensor_layout "https://github.com/dex-zha/juqe/blob/main/adapters/llama/layout.cpp"
click node_engine_pools "https://github.com/dex-zha/juqe/blob/main/adapters/llama/pools.cpp"
click node_ffi_boundary "https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/boundary.rs"
click node_ffi_create "https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/runtime/create.rs"
click node_ffi_batch "https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/runtime/batch.rs"
click node_ffi_memory "https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/memory/region.rs"
click node_pack_handle "https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/pack.rs"
click node_runtime_api "https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/runtime.rs"
click node_dispatch "https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/dispatch.rs"
click node_batch_leases "https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/batch.rs"
click node_cache_state "https://github.com/dex-zha/juqe/blob/main/crates/juqe-core/src/cache/state.rs"
click node_worker "https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/worker.rs"
click node_storage "https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/storage.rs"
click node_format_parser "https://github.com/dex-zha/juqe/blob/main/crates/juqe-core/src/format/parser.rs"
click node_reader "https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/reader.rs"
click node_source "https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/source.rs"
click node_aligned_buffer "https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/buffer.rs"
click node_positional "https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/positional.rs"
click node_scatter "https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/linux/scatter.rs"
click node_cli "https://github.com/dex-zha/juqe/blob/main/crates/juqe-cli/src/main.rs"
click node_cli_args "https://github.com/dex-zha/juqe/blob/main/crates/juqe-cli/src/args.rs"
click node_verify "https://github.com/dex-zha/juqe/blob/main/crates/juqe-cli/src/commands/verify.rs"
click node_inspect "https://github.com/dex-zha/juqe/blob/main/crates/juqe-cli/src/commands/inspect.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_routing,node_adapter_state,node_tensor_layout,node_engine_pools toneBlue
class node_ffi_boundary,node_ffi_create,node_ffi_batch,node_ffi_memory,node_pack_handle toneAmber
class node_runtime_api,node_dispatch,node_batch_leases,node_cache_state,node_worker,node_storage toneMint
class node_format_parser,node_reader,node_source,node_aligned_buffer,node_positional,node_scatter toneRose
class node_cli,node_cli_args,node_verify,node_inspect,node_llama toneIndigo
```

## Components

### Engine adapter

- [Route capture](https://github.com/dex-zha/juqe/blob/main/adapters/llama/routing.cpp): `adapters/llama/routing.cpp`
- [Adapter state](https://github.com/dex-zha/juqe/blob/main/adapters/llama/state.h): `adapters/llama/state.h`
- [Tensor layout](https://github.com/dex-zha/juqe/blob/main/adapters/llama/layout.cpp): `adapters/llama/layout.cpp`
- [Pinned pools](https://github.com/dex-zha/juqe/blob/main/adapters/llama/pools.cpp): `adapters/llama/pools.cpp`

### C ABI and pools

- [ABI boundary](https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/boundary.rs): `crates/juqe-ffi/src/boundary.rs`
- [Runtime creation](https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/runtime/create.rs): `crates/juqe-ffi/src/runtime/create.rs`
- [Batch API](https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/runtime/batch.rs): `crates/juqe-ffi/src/runtime/batch.rs`
- [Pool memory bridge](https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/memory/region.rs): `crates/juqe-ffi/src/memory/region.rs`
- [Pack metadata API](https://github.com/dex-zha/juqe/blob/main/crates/juqe-ffi/src/pack.rs): `crates/juqe-ffi/src/pack.rs`

### Loading runtime

- [Runtime lifecycle](https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/runtime.rs): `crates/juqe-runtime/src/runtime.rs`
- [Load dispatch](https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/dispatch.rs): `crates/juqe-runtime/src/dispatch.rs`
- [Batch leases](https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/batch.rs): `crates/juqe-runtime/src/batch.rs`
- [LRU cache state](https://github.com/dex-zha/juqe/blob/main/crates/juqe-core/src/cache/state.rs): `crates/juqe-core/src/cache/state.rs`
- [Load workers](https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/worker.rs): `crates/juqe-runtime/src/worker.rs`
- [Expert storage](https://github.com/dex-zha/juqe/blob/main/crates/juqe-runtime/src/storage.rs): `crates/juqe-runtime/src/storage.rs`

### Pack and I/O

- [Index validation](https://github.com/dex-zha/juqe/blob/main/crates/juqe-core/src/format/parser.rs): `crates/juqe-core/src/format/parser.rs`
- [Expert reader](https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/reader.rs): `crates/juqe-io/src/reader.rs`
- [Read source](https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/source.rs): `crates/juqe-io/src/source.rs`
- [Aligned scratch](https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/buffer.rs): `crates/juqe-io/src/buffer.rs`
- [Positional reads](https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/positional.rs): `crates/juqe-io/src/positional.rs`
- [Linux scatter I/O](https://github.com/dex-zha/juqe/blob/main/crates/juqe-io/src/linux/scatter.rs): `crates/juqe-io/src/linux/scatter.rs`

### Pack tools

- [CLI commands](https://github.com/dex-zha/juqe/blob/main/crates/juqe-cli/src/main.rs): `crates/juqe-cli/src/main.rs`
- [Command parsing](https://github.com/dex-zha/juqe/blob/main/crates/juqe-cli/src/args.rs): `crates/juqe-cli/src/args.rs`
- [Pack verification](https://github.com/dex-zha/juqe/blob/main/crates/juqe-cli/src/commands/verify.rs): `crates/juqe-cli/src/commands/verify.rs`
- [Pack inspection](https://github.com/dex-zha/juqe/blob/main/crates/juqe-cli/src/commands/inspect.rs): `crates/juqe-cli/src/commands/inspect.rs`

### Other components

- **llama.cpp engine**

## Connections

- llama.cpp engine → Route capture: invokes callbacks
- Route capture → Adapter state: uses state (evidence: `adapters/llama/routing.cpp`)
- Route capture → Batch API: prepares batches (evidence: `adapters/llama/routing.cpp`)
- Route capture → Pinned pools: binds routed slots
- Tensor layout → Adapter state: configures layout (evidence: `adapters/llama/layout.cpp`)
- Pinned pools → Adapter state: supplies pools (evidence: `adapters/llama/pools.cpp`)
- ABI boundary → Runtime creation: guards calls
- ABI boundary → Batch API: guards calls (evidence: `crates/juqe-ffi/src/runtime/batch.rs`)
- Runtime creation → Runtime lifecycle: creates runtime
- Batch API → Runtime lifecycle: prepares batches
- Pool memory bridge → Pinned pools: views memory
- Pack metadata API → Index validation: opens pack index
- Runtime lifecycle → Load dispatch: dispatches loads
- Runtime lifecycle → LRU cache state: manages cache
- Load dispatch → Load workers: queues jobs
- Load workers → Expert storage: loads expert (evidence: `crates/juqe-runtime/src/worker.rs`)
- Expert storage → Expert reader: reads matrices
- Expert storage → Pinned pools: writes destination slots
- Expert reader → Read source: reads from source (evidence: `crates/juqe-io/src/reader.rs`)
- Read source → Positional reads: stages buffered reads (evidence: `crates/juqe-io/src/source.rs`)
- Read source → Linux scatter I/O: attempts direct reads (evidence: `crates/juqe-io/src/source.rs`)
- Expert reader → Aligned scratch: uses scratch buffer (evidence: `crates/juqe-io/src/reader.rs`)
- CLI commands → Command parsing: parses arguments (evidence: `crates/juqe-cli/src/main.rs`)
- CLI commands → Pack inspection: dispatches commands (evidence: `crates/juqe-cli/src/main.rs`)
- CLI commands → Pack verification: dispatches commands (evidence: `crates/juqe-cli/src/main.rs`)
- Pack verification → Command parsing: uses verify options (evidence: `crates/juqe-cli/src/commands/verify.rs`)
- CLI commands → Index validation: inspects validated packs
