# lawrencel05/topology-aware-gpu-scheduling architecture

> Architecture diagram of the GitHub repository lawrencel05/topology-aware-gpu-scheduling, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lawrencel05/topology-aware-gpu-scheduling
- Repository: https://github.com/lawrencel05/topology-aware-gpu-scheduling

## Overview

This is an experimental GPU placement and scheduling research toolkit. A user discovers cluster GPU inventory, supplies a workload profile and inter-node link costs, plans a placement, and executes it through Ray or submits it through KAI; Dynamo supports a separate managed inference lifecycle. Comparison records and matched traces evaluate scheduling outcomes. The README documents topology discovery, NIC inventory, baseline policies and integrations; unsampled subsystems are shown only where their documented role is clear, and wiring is limited to supported calls.

Main entry points: [`topology_scheduler/kai_backend.py`](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/kai_backend.py) (KAI Adapter), [`topology_scheduler/comparison.py`](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/comparison.py) (Comparison Runner), [`topology_scheduler/ray_backend.py`](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/ray_backend.py) (Ray Executor), [`topology_scheduler/policy.py`](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/policy.py) (Placement Policies)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_discovery["Cluster Discovery"]
  node_gpu_inventory["GPU Inventory<br/>[inventory.py]"]
  node_nic_inventory["NIC Inventory<br/>[nic_inventory.py]"]
end

subgraph group_planning["Placement Planning"]
  node_policy["Placement Policies<br/>[policy.py]"]
  node_planner_nodes["Planner Nodes<br/>[policy.py]"]
  node_plan["Placement Plan<br/>[policy.py]"]
end

subgraph group_execution["Execution Backends"]
  node_ray_backend["Ray Executor<br/>[ray_backend.py]"]
  node_ray_tasks["Ray Worker Tasks<br/>[ray_backend.py]"]
  node_kai_backend["KAI Adapter<br/>[kai_backend.py]"]
  node_kai_objects["PodGroup and Pods<br/>[kai_backend.py]"]
  node_dynamo_backend["Dynamo Lifecycle<br/>[dynamo_backend.py]"]
  node_dynamo_guardian["Process Guardian"]
end

subgraph group_evaluation["Evaluation"]
  node_comparison["Comparison Runner<br/>[comparison.py]"]
  node_records["JCT Records<br/>[comparison.py]"]
end

node_user(("Researcher"))
node_gpu_cluster(("Ray GPU Cluster"))
node_host_sysfs[("Host Sysfs")]
node_workload_inputs["Workload and Links"]
node_kubernetes(("Kubernetes KAI"))
node_dynamo_runtime(("Dynamo Runtime"))

node_user -->|"requests discovery"| node_gpu_inventory
node_gpu_inventory -->|"probes nodes"| node_gpu_cluster
node_gpu_inventory -->|"derives inputs"| node_planner_nodes
node_user -->|"requests discovery"| node_nic_inventory
node_nic_inventory -->|"probes nodes"| node_gpu_cluster
node_nic_inventory -->|"reads attributes"| node_host_sysfs
node_user -->|"supplies profile"| node_workload_inputs
node_user -->|"chooses policy"| node_policy
node_planner_nodes -->|"provides nodes"| node_policy
node_workload_inputs -->|"provides costs"| node_policy
node_policy -->|"selects placement"| node_plan
node_user -->|"runs comparison"| node_comparison
node_comparison -->|"plans jobs"| node_policy
node_comparison -->|"dispatches backend"| node_ray_backend
node_comparison -->|"dispatches backend"| node_kai_backend
node_comparison -->|"captures outcomes"| node_records
node_user -->|"runs plan"| node_ray_backend
node_plan -->|"reserves placement"| node_ray_backend
node_ray_backend -->|"reserves resources"| node_gpu_cluster
node_ray_backend -->|"launches tasks"| node_ray_tasks
node_user -->|"submits plan"| node_kai_backend
node_plan -->|"maps workers"| node_kai_backend
node_kai_backend -->|"reads and manages"| node_kubernetes
node_kai_backend -->|"builds manifests"| node_kai_objects
node_kai_backend -->|"submits and watches"| node_kubernetes
node_user -->|"starts service"| node_dynamo_backend
node_plan -->|"assigns replicas"| node_dynamo_backend
node_dynamo_backend -->|"reserves actors"| node_gpu_cluster
node_dynamo_backend -->|"launches guardian"| node_dynamo_guardian
node_dynamo_guardian -->|"starts and stops"| node_dynamo_runtime

click node_gpu_inventory "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/inventory.py"
click node_nic_inventory "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/nic_inventory.py"
click node_policy "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/policy.py"
click node_planner_nodes "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/policy.py"
click node_plan "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/policy.py"
click node_comparison "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/comparison.py"
click node_records "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/comparison.py"
click node_ray_backend "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/ray_backend.py"
click node_ray_tasks "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/ray_backend.py"
click node_kai_backend "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/kai_backend.py"
click node_kai_objects "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/kai_backend.py"
click node_dynamo_backend "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/dynamo_backend.py"
click node_dynamo_guardian "https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/_dynamo_guardian.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_gpu_inventory,node_nic_inventory toneBlue
class node_policy,node_planner_nodes,node_plan,node_host_sysfs toneAmber
class node_ray_backend,node_ray_tasks,node_kai_backend,node_kai_objects,node_dynamo_backend,node_dynamo_guardian toneMint
class node_comparison,node_records toneRose
class node_user,node_gpu_cluster,node_workload_inputs,node_kubernetes,node_dynamo_runtime toneIndigo
```

## Components

### Cluster Discovery

- [GPU Inventory](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/inventory.py): `topology_scheduler/inventory.py`
- [NIC Inventory](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/nic_inventory.py): `topology_scheduler/nic_inventory.py`

### Placement Planning

- [Placement Policies](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/policy.py): `topology_scheduler/policy.py`
- [Planner Nodes](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/policy.py): `topology_scheduler/policy.py`
- [Placement Plan](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/policy.py): `topology_scheduler/policy.py`

### Execution Backends

- [Ray Executor](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/ray_backend.py): `topology_scheduler/ray_backend.py`
- [Ray Worker Tasks](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/ray_backend.py): `topology_scheduler/ray_backend.py`
- [KAI Adapter](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/kai_backend.py): `topology_scheduler/kai_backend.py`
- [PodGroup and Pods](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/kai_backend.py): `topology_scheduler/kai_backend.py`
- [Dynamo Lifecycle](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/dynamo_backend.py): `topology_scheduler/dynamo_backend.py`
- [Process Guardian](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/_dynamo_guardian.py): `topology_scheduler/_dynamo_guardian.py`

### Evaluation

- [Comparison Runner](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/comparison.py): `topology_scheduler/comparison.py`
- [JCT Records](https://github.com/lawrencel05/topology-aware-gpu-scheduling/blob/main/topology_scheduler/comparison.py): `topology_scheduler/comparison.py`

### Other components

- **Researcher**
- **Ray GPU Cluster**
- **Host Sysfs**
- **Workload and Links**
- **Kubernetes KAI**
- **Dynamo Runtime**

## Connections

- Researcher → GPU Inventory: requests discovery
- GPU Inventory → Ray GPU Cluster: probes nodes
- GPU Inventory → Planner Nodes: derives inputs
- Researcher → NIC Inventory: requests discovery
- NIC Inventory → Ray GPU Cluster: probes nodes
- NIC Inventory → Host Sysfs: reads attributes
- Researcher → Workload and Links: supplies profile
- Researcher → Placement Policies: chooses policy
- Planner Nodes → Placement Policies: provides nodes
- Workload and Links → Placement Policies: provides costs
- Placement Policies → Placement Plan: selects placement
- Researcher → Comparison Runner: runs comparison
- Comparison Runner → Placement Policies: plans jobs
- Comparison Runner → Ray Executor: dispatches backend
- Comparison Runner → KAI Adapter: dispatches backend
- Comparison Runner → JCT Records: captures outcomes
- Researcher → Ray Executor: runs plan
- Placement Plan → Ray Executor: reserves placement
- Ray Executor → Ray GPU Cluster: reserves resources
- Ray Executor → Ray Worker Tasks: launches tasks
- Researcher → KAI Adapter: submits plan
- Placement Plan → KAI Adapter: maps workers
- KAI Adapter → Kubernetes KAI: reads and manages
- KAI Adapter → PodGroup and Pods: builds manifests
- KAI Adapter → Kubernetes KAI: submits and watches
- Researcher → Dynamo Lifecycle: starts service
- Placement Plan → Dynamo Lifecycle: assigns replicas
- Dynamo Lifecycle → Ray GPU Cluster: reserves actors
- Dynamo Lifecycle → Process Guardian: launches guardian
- Process Guardian → Dynamo Runtime: starts and stops