# ai-dynamo/aisimulate architecture

> Architecture diagram of the GitHub repository ai-dynamo/aisimulate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ai-dynamo/aisimulate
- Repository: https://github.com/ai-dynamo/aisimulate

## Overview

AISimulate predicts LLM-serving performance and searches deployment configurations offline. The main workflow accepts typed deployment or search configuration, evaluates candidate behavior with the compiled performance engine and runtime data, then emits reports or concrete recommendations. Separate capabilities include deterministic workload replay, legacy deployment-artifact generation, and collection/publication of empirical performance data. The graph keeps these product branches distinct; links among unsampled subsystems are omitted rather than inferred from directory names.

Main entry points: [`crates/core/src/replay/components/engine.rs`](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/components/engine.rs) (Replay engine component), [`python/aisimulate/src/aisimulate_core/sdk/engine.py`](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/engine.py) (Compiled engine), [`crates/core/src/replay/replayer.rs`](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/replayer.rs) (Replay orchestration), [`python/aisimulate/src/aisimulate/main.py`](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/main.py) (Prediction and search)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User workflows"]
  node_engine_config["Engine config<br/>[engine.py]"]
  node_cli["Prediction and search<br/>[main.py]"]
  node_recommend["Recommendation<br/>[recommend.py]"]
  node_output["Prediction reports<br/>[output.py]"]
end

subgraph group_model["Performance modeling"]
  node_compiled_engine["Compiled engine<br/>[engine.py]"]
  node_rust_step["Rust engine step"]
  node_op_models["Operator models<br/>[attention.py]"]
  node_model_families["Model families<br/>[deepseek_v41.py]"]
  node_perf_db[("Performance database<br/>[perf_database.py]")]
  node_perfmodel["Performance estimators<br/>[estimator.rs]"]
end

subgraph group_replay["Workload replay"]
  node_replay_api["Replay entry<br/>[engine.rs]"]
  node_replayer["Replay orchestration<br/>[replayer.rs]"]
  node_replay_engine["Replay engine component<br/>[engine.rs]"]
  node_round_robin["Round-robin core<br/>[round_robin.rs]"]
  node_disagg["Disaggregated replay<br/>[disagg.rs]"]
  node_replay_protocol["Replay protocol<br/>[protocol.rs]"]
  node_engine_contracts["Scheduler contracts<br/>[contracts.rs]"]
  node_belady["KV eviction policy<br/>[belady.rs]"]
  node_hashing["KV hashing<br/>[hashing.rs]"]
end

subgraph group_empirical["Empirical data"]
  node_collector["Performance collectors<br/>[collect.py]"]
  node_fpm_collector["Forward-pass collection<br/>[runner.py]"]
end

subgraph group_generation["Deployment generation"]
  node_generator["Artifact generator<br/>[engine.py]"]
  node_sweeper["Configuration sweeper<br/>[afd_engine.py]"]
  node_legacy_cli["Compatibility workflows<br/>[main.py]"]
end

node_user(("Operator"))

node_user -->|"runs"| node_cli
node_user -->|"configures"| node_engine_config
node_cli -->|"writes report"| node_output
node_cli -->|"searches candidates"| node_recommend
node_engine_config -.->|"configures"| node_compiled_engine
node_compiled_engine -->|"executes through"| node_rust_step
node_rust_step -->|"queries"| node_perf_db
node_compiled_engine -->|"uses"| node_op_models
node_compiled_engine -->|"uses"| node_model_families
node_user -->|"replays workload"| node_replay_api
node_replay_api -->|"runs"| node_replayer
node_replayer -->|"dispatches"| node_replay_engine
node_replayer -->|"uses"| node_round_robin
node_replayer -->|"supports topology"| node_disagg
node_replay_engine -->|"implements"| node_engine_contracts
node_replay_engine -->|"accounts requests"| node_replay_protocol
node_user -->|"generates artifacts"| node_generator
node_user -->|"runs workflows"| node_legacy_cli
node_legacy_cli -->|"generates configs"| node_generator
node_user -.->|"searches configurations"| node_sweeper
node_collector -.->|"supplies measurements"| node_perf_db
node_fpm_collector -.->|"provides observations"| node_perfmodel
node_replay_engine -.->|"uses eviction policy"| node_belady
node_replay_engine -.->|"uses cache keys"| node_hashing

click node_engine_config "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/config/engine.py"
click node_cli "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/main.py"
click node_recommend "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/recommend.py"
click node_output "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/output.py"
click node_compiled_engine "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/engine.py"
click node_rust_step "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/rust_engine_step.py"
click node_op_models "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/operations/attention.py"
click node_model_families "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/deepseek_v41.py"
click node_perf_db "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/perf_database.py"
click node_perfmodel "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/perfmodel/fpm/estimator.rs"
click node_replay_api "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/engine.rs"
click node_replayer "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/replayer.rs"
click node_replay_engine "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/components/engine.rs"
click node_round_robin "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/core/round_robin.rs"
click node_disagg "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/disagg.rs"
click node_replay_protocol "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/protocol.rs"
click node_engine_contracts "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/engine/generalized/contracts.rs"
click node_collector "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/collector/collect.py"
click node_fpm_collector "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/collector/fpm_forward/runner.py"
click node_generator "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/generator/rendering/engine.py"
click node_sweeper "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/sweeper/afd_engine.py"
click node_legacy_cli "https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/legacy_cli/main.py"
click node_belady "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/engine/belady.rs"
click node_hashing "https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/engine/common/hashing.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_engine_config,node_cli,node_recommend,node_output toneBlue
class node_compiled_engine,node_rust_step,node_op_models,node_model_families,node_perf_db,node_perfmodel toneAmber
class node_replay_api,node_replayer,node_replay_engine,node_round_robin,node_disagg,node_replay_protocol,node_engine_contracts,node_belady,node_hashing toneMint
class node_collector,node_fpm_collector toneRose
class node_generator,node_sweeper,node_legacy_cli,node_user toneIndigo
```

## Components

### User workflows

- [Engine config](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/config/engine.py): `python/aisimulate/src/aisimulate/config/engine.py`
- [Prediction and search](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/main.py): `python/aisimulate/src/aisimulate/main.py`
- [Recommendation](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/recommend.py): `python/aisimulate/src/aisimulate/recommend.py`
- [Prediction reports](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/output.py): `python/aisimulate/src/aisimulate/output.py`

### Performance modeling

- [Compiled engine](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/engine.py): `python/aisimulate/src/aisimulate_core/sdk/engine.py`
- [Rust engine step](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/rust_engine_step.py): `python/aisimulate/src/aisimulate_core/sdk/rust_engine_step.py`
- [Operator models](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/operations/attention.py): `python/aisimulate/src/aisimulate_core/sdk/operations/attention.py`
- [Model families](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/deepseek_v41.py): `python/aisimulate/src/aisimulate_core/sdk/deepseek_v41.py`
- [Performance database](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate_core/sdk/perf_database.py): `python/aisimulate/src/aisimulate_core/sdk/perf_database.py`
- [Performance estimators](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/perfmodel/fpm/estimator.rs): `crates/core/src/perfmodel/fpm/estimator.rs`

### Workload replay

- [Replay entry](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/engine.rs): `crates/core/src/replay/engine.rs`
- [Replay orchestration](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/replayer.rs): `crates/core/src/replay/replayer.rs`
- [Replay engine component](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/components/engine.rs): `crates/core/src/replay/components/engine.rs`
- [Round-robin core](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/core/round_robin.rs): `crates/core/src/replay/core/round_robin.rs`
- [Disaggregated replay](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/disagg.rs): `crates/core/src/replay/disagg.rs`
- [Replay protocol](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/replay/protocol.rs): `crates/core/src/replay/protocol.rs`
- [Scheduler contracts](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/engine/generalized/contracts.rs): `crates/core/src/engine/generalized/contracts.rs`
- [KV eviction policy](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/engine/belady.rs): `crates/core/src/engine/belady.rs`
- [KV hashing](https://github.com/ai-dynamo/aisimulate/blob/main/crates/core/src/engine/common/hashing.rs): `crates/core/src/engine/common/hashing.rs`

### Empirical data

- [Performance collectors](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/collector/collect.py): `python/aisimulate/collector/collect.py`
- [Forward-pass collection](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/collector/fpm_forward/runner.py): `python/aisimulate/collector/fpm_forward/runner.py`

### Deployment generation

- [Artifact generator](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/generator/rendering/engine.py): `python/aisimulate/src/aisimulate/generator/rendering/engine.py`
- [Configuration sweeper](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/sweeper/afd_engine.py): `python/aisimulate/src/aisimulate/sweeper/afd_engine.py`
- [Compatibility workflows](https://github.com/ai-dynamo/aisimulate/blob/main/python/aisimulate/src/aisimulate/legacy_cli/main.py): `python/aisimulate/src/aisimulate/legacy_cli/main.py`

### Other components

- **Operator**

## Connections

- Operator → Prediction and search: runs
- Operator → Engine config: configures
- Prediction and search → Prediction reports: writes report (evidence: `python/aisimulate/src/aisimulate/main.py`)
- Prediction and search → Recommendation: searches candidates (evidence: `python/aisimulate/src/aisimulate/main.py`)
- Engine config → Compiled engine: configures
- Compiled engine → Rust engine step: executes through (evidence: `python/aisimulate/src/aisimulate_core/sdk/engine.py`)
- Rust engine step → Performance database: queries (evidence: `python/aisimulate/src/aisimulate_core/sdk/rust_engine_step.py`)
- Compiled engine → Operator models: uses (evidence: `python/aisimulate/src/aisimulate_core/sdk/engine.py`)
- Compiled engine → Model families: uses (evidence: `python/aisimulate/src/aisimulate_core/sdk/engine.py`)
- Operator → Replay entry: replays workload
- Replay entry → Replay orchestration: runs (evidence: `crates/core/src/replay/engine.rs`)
- Replay orchestration → Replay engine component: dispatches
- Replay orchestration → Round-robin core: uses
- Replay orchestration → Disaggregated replay: supports topology
- Replay engine component → Scheduler contracts: implements (evidence: `crates/core/src/replay/components/engine.rs`)
- Replay engine component → Replay protocol: accounts requests (evidence: `crates/core/src/replay/components/engine.rs`)
- Operator → Artifact generator: generates artifacts
- Operator → Compatibility workflows: runs workflows
- Compatibility workflows → Artifact generator: generates configs (evidence: `python/aisimulate/src/aisimulate/legacy_cli/main.py`)
- Operator → Configuration sweeper: searches configurations
- Performance collectors → Performance database: supplies measurements
- Forward-pass collection → Performance estimators: provides observations
- Replay engine component → KV eviction policy: uses eviction policy
- Replay engine component → KV hashing: uses cache keys
