# semianalysisai/inferencex architecture

> Architecture diagram of the GitHub repository semianalysisai/inferencex, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/semianalysisai/inferencex
- Repository: https://github.com/semianalysisai/inferencex

## Overview

InferenceX is a multi-project inference research platform: its end-to-end tooling launches serving benchmarks, gathers and analyzes results, and supports evaluations; CollectiveX benchmarks communication, while OperatorX benchmarks individual operators across hardware backends. The graph preserves those distinct product branches and their principal stages. Some subsystems are represented by their source directories because their implementation was not sampled; those branches are documented capabilities, not claims that their internal calls were inspected.

Main entry points: [`inferencex-e2e/infx/launch/context.py`](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/context.py) (Launch context), [`inferencex-e2e/infx/launch/backends/base.py`](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/backends/base.py) (Launch backend), [`operatorx/main.py`](https://github.com/semianalysisai/inferencex/blob/main/operatorx/main.py) (Operator runner), [`inferencex-e2e/infx/clusters/__init__.py`](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/clusters/__init__.py) (Cluster records)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_launch["Benchmark orchestration"]
  node_e2e_config["Configuration<br/>[config.py]"]
  node_matrix["Benchmark matrix"]
  node_cluster["Cluster records<br/>[__init__.py]"]
  node_launch_request["Launch request<br/>[request.py]"]
  node_launch_policy["Launch policy<br/>[policy.py]"]
  node_launch_context["Launch context<br/>[context.py]"]
  node_launch_backend["Launch backend<br/>[base.py]"]
  node_cluster_scheduler["Cluster scheduler<br/>[slurm.py]"]
end

subgraph group_measurement["Serving and analysis"]
  node_serving_bench["Serving benchmark"]
  node_server["Inference server"]
  node_result_collection["Result collection"]
  node_agentic_metrics["Agentic metrics"]
  node_result_analysis["Result analysis"]
end

subgraph group_evaluation["Evaluation workflows"]
  node_evals["Evaluations"]
  node_workflow["Benchmark workflows"]
end

subgraph group_collectivex["CollectiveX"]
  node_collective_matrix["Collective sweep matrix<br/>[sweep_matrix.py]"]
  node_collective_runtime["Collective runtime"]
  node_collective_summary["Bandwidth summary<br/>[summarize.py]"]
end

subgraph group_operatorx["OperatorX"]
  node_operator_cli["Operator runner<br/>[main.py]"]
  node_operator_runtime["Runtime snapshot<br/>[runtime.py]"]
  node_operator_registry["Operation registry<br/>[op_registry.py]"]
  node_operator_spec["Operation model<br/>[op.py]"]
  node_operator_backends["Hardware backends"]
  node_operator_result["Run results<br/>[result.py]"]
end

node_user(("Researcher"))

node_user -->|"configures"| node_e2e_config
node_e2e_config -.->|"configures"| node_matrix
node_matrix -->|"reads topology"| node_cluster
node_launch_context -->|"uses"| node_cluster
node_launch_context -->|"uses"| node_launch_backend
node_launch_context -->|"uses"| node_launch_request
node_launch_context -->|"uses"| node_launch_policy
node_launch_backend -->|"uses"| node_cluster
node_launch_backend -->|"uses"| node_launch_request
node_launch_backend -.->|"dispatches"| node_cluster_scheduler
node_launch_policy -->|"reads"| node_cluster
node_launch_policy -->|"uses"| node_launch_request
node_launch_context -.->|"launches"| node_serving_bench
node_serving_bench -.->|"sends requests"| node_server
node_server -.->|"produces metrics"| node_result_collection
node_result_collection -.->|"adapts metrics"| node_agentic_metrics
node_result_collection -.->|"feeds analysis"| node_result_analysis
node_workflow -.->|"coordinates runs"| node_matrix
node_workflow -.->|"coordinates"| node_evals
node_user -.->|"selects sweep"| node_collective_matrix
node_collective_matrix -.->|"creates shards"| node_collective_runtime
node_collective_runtime -.->|"summarizes"| node_collective_summary
node_collective_summary -->|"imports bandwidth"| node_collective_runtime
node_user -->|"runs benchmarks"| node_operator_cli
node_operator_cli -->|"snapshots runtime"| node_operator_runtime
node_operator_cli -->|"runs operations"| node_operator_backends
node_operator_cli -.->|"records results"| node_operator_result
node_operator_backends -->|"executes"| node_operator_spec
node_operator_registry -->|"registers"| node_operator_spec

click node_e2e_config "https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/config.py"
click node_matrix "https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/matrix"
click node_cluster "https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/clusters/__init__.py"
click node_launch_request "https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/request.py"
click node_launch_policy "https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/policy.py"
click node_launch_context "https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/context.py"
click node_launch_backend "https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/backends/base.py"
click node_cluster_scheduler "https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/clusters/slurm.py"
click node_serving_bench "https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/bench_serving/benchmark_serving.py"
click node_result_collection "https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/results"
click node_agentic_metrics "https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/results/agentic/backends"
click node_result_analysis "https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/results"
click node_evals "https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/evals"
click node_workflow "https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/workflows"
click node_collective_matrix "https://github.com/semianalysisai/inferencex/blob/main/collectivex/sweep_matrix.py"
click node_collective_runtime "https://github.com/semianalysisai/inferencex/tree/main/collectivex/runtime"
click node_collective_summary "https://github.com/semianalysisai/inferencex/blob/main/collectivex/summarize.py"
click node_operator_cli "https://github.com/semianalysisai/inferencex/blob/main/operatorx/main.py"
click node_operator_runtime "https://github.com/semianalysisai/inferencex/blob/main/operatorx/runtime.py"
click node_operator_registry "https://github.com/semianalysisai/inferencex/blob/main/operatorx/core/op_registry.py"
click node_operator_spec "https://github.com/semianalysisai/inferencex/blob/main/operatorx/core/op.py"
click node_operator_backends "https://github.com/semianalysisai/inferencex/tree/main/operatorx/runners"
click node_operator_result "https://github.com/semianalysisai/inferencex/blob/main/operatorx/core/result.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_e2e_config,node_matrix,node_cluster,node_launch_request,node_launch_policy,node_launch_context,node_launch_backend,node_cluster_scheduler toneBlue
class node_serving_bench,node_server,node_result_collection,node_agentic_metrics,node_result_analysis toneAmber
class node_evals,node_workflow toneMint
class node_collective_matrix,node_collective_runtime,node_collective_summary toneRose
class node_operator_cli,node_operator_runtime,node_operator_registry,node_operator_spec,node_operator_backends,node_operator_result,node_user toneIndigo
```

## Components

### Benchmark orchestration

- [Configuration](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/config.py): `inferencex-e2e/infx/config.py`
- [Benchmark matrix](https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/matrix): `inferencex-e2e/infx/matrix`
- [Cluster records](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/clusters/__init__.py): `inferencex-e2e/infx/clusters/__init__.py`
- [Launch request](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/request.py): `inferencex-e2e/infx/launch/request.py`
- [Launch policy](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/policy.py): `inferencex-e2e/infx/launch/policy.py`
- [Launch context](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/context.py): `inferencex-e2e/infx/launch/context.py`
- [Launch backend](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/launch/backends/base.py): `inferencex-e2e/infx/launch/backends/base.py`
- [Cluster scheduler](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/clusters/slurm.py): `inferencex-e2e/infx/clusters/slurm.py`

### Serving and analysis

- [Serving benchmark](https://github.com/semianalysisai/inferencex/blob/main/inferencex-e2e/infx/bench_serving/benchmark_serving.py): `inferencex-e2e/infx/bench_serving/benchmark_serving.py`
- **Inference server**
- [Result collection](https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/results): `inferencex-e2e/infx/results`
- [Agentic metrics](https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/results/agentic/backends): `inferencex-e2e/infx/results/agentic/backends`
- [Result analysis](https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/results): `inferencex-e2e/infx/results`

### Evaluation workflows

- [Evaluations](https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/evals): `inferencex-e2e/infx/evals`
- [Benchmark workflows](https://github.com/semianalysisai/inferencex/tree/main/inferencex-e2e/infx/workflows): `inferencex-e2e/infx/workflows`

### CollectiveX

- [Collective sweep matrix](https://github.com/semianalysisai/inferencex/blob/main/collectivex/sweep_matrix.py): `collectivex/sweep_matrix.py`
- [Collective runtime](https://github.com/semianalysisai/inferencex/tree/main/collectivex/runtime): `collectivex/runtime`
- [Bandwidth summary](https://github.com/semianalysisai/inferencex/blob/main/collectivex/summarize.py): `collectivex/summarize.py`

### OperatorX

- [Operator runner](https://github.com/semianalysisai/inferencex/blob/main/operatorx/main.py): `operatorx/main.py`
- [Runtime snapshot](https://github.com/semianalysisai/inferencex/blob/main/operatorx/runtime.py): `operatorx/runtime.py`
- [Operation registry](https://github.com/semianalysisai/inferencex/blob/main/operatorx/core/op_registry.py): `operatorx/core/op_registry.py`
- [Operation model](https://github.com/semianalysisai/inferencex/blob/main/operatorx/core/op.py): `operatorx/core/op.py`
- [Hardware backends](https://github.com/semianalysisai/inferencex/tree/main/operatorx/runners): `operatorx/runners`
- [Run results](https://github.com/semianalysisai/inferencex/blob/main/operatorx/core/result.py): `operatorx/core/result.py`

### Other components

- **Researcher**

## Connections

- Researcher → Configuration: configures (evidence: `README.md`)
- Configuration → Benchmark matrix: configures
- Benchmark matrix → Cluster records: reads topology (evidence: `inferencex-e2e/infx/clusters/__init__.py`)
- Launch context → Cluster records: uses (evidence: `inferencex-e2e/infx/launch/context.py`)
- Launch context → Launch backend: uses (evidence: `inferencex-e2e/infx/launch/context.py`)
- Launch context → Launch request: uses (evidence: `inferencex-e2e/infx/launch/context.py`)
- Launch context → Launch policy: uses (evidence: `inferencex-e2e/infx/launch/context.py`)
- Launch backend → Cluster records: uses (evidence: `inferencex-e2e/infx/launch/backends/base.py`)
- Launch backend → Launch request: uses (evidence: `inferencex-e2e/infx/launch/backends/base.py`)
- Launch backend → Cluster scheduler: dispatches
- Launch policy → Cluster records: reads (evidence: `inferencex-e2e/infx/launch/policy.py`)
- Launch policy → Launch request: uses (evidence: `inferencex-e2e/infx/launch/policy.py`)
- Launch context → Serving benchmark: launches
- Serving benchmark → Inference server: sends requests
- Inference server → Result collection: produces metrics
- Result collection → Agentic metrics: adapts metrics (evidence: `inferencex-e2e/infx/results/agentic/backends/vllm.py`)
- Result collection → Result analysis: feeds analysis
- Benchmark workflows → Benchmark matrix: coordinates runs (evidence: `inferencex-e2e/infx/workflows/recover_failed_ingest.py`)
- Benchmark workflows → Evaluations: coordinates
- Researcher → Collective sweep matrix: selects sweep
- Collective sweep matrix → Collective runtime: creates shards
- Collective runtime → Bandwidth summary: summarizes
- Bandwidth summary → Collective runtime: imports bandwidth
- Researcher → Operator runner: runs benchmarks (evidence: `README.md`)
- Operator runner → Runtime snapshot: snapshots runtime (evidence: `operatorx/main.py`)
- Operator runner → Hardware backends: runs operations (evidence: `operatorx/main.py`)
- Operator runner → Run results: records results
- Hardware backends → Operation model: executes (evidence: `operatorx/runners/amd/backends/torch.py`)
- Operation registry → Operation model: registers (evidence: `operatorx/core/op_registry.py`)