How lawrencel05/topology-aware-gpu-scheduling works
This is an experimental GPU placement and scheduling research toolkit. A user discovers cluster GPU inventory, supplies a workload profile and inter-node link costs, plans a placement, and executes it through Ray or submits it through KAI; Dynamo supports a separate managed inference lifecycle. Comparison records and matched traces evaluate scheduling outcomes. The README documents topology discovery, NIC inventory, baseline policies and integrations; unsampled subsystems are shown only where their documented role is clear, and wiring is limited to supported calls.
Main entry points: kai_backend.py KAI Adapter · comparison.py Comparison Runner · ray_backend.py Ray Executor · policy.py Placement Policies
Components 19
Cluster Discovery
- GPU Inventory
topology_scheduler/inventory.py - NIC Inventory
topology_scheduler/nic_inventory.py
Placement Planning
- Placement Policies
topology_scheduler/policy.py - Planner Nodes
topology_scheduler/policy.py - Placement Plan
topology_scheduler/policy.py
Execution Backends
- Ray Executor
topology_scheduler/ray_backend.py - Ray Worker Tasks
topology_scheduler/ray_backend.py - KAI Adapter
topology_scheduler/kai_backend.py - PodGroup and Pods
topology_scheduler/kai_backend.py - Dynamo Lifecycle
topology_scheduler/dynamo_backend.py - Process Guardian
topology_scheduler/_dynamo_guardian.py
Evaluation
- Comparison Runner
topology_scheduler/comparison.py - JCT Records
topology_scheduler/comparison.py
Other components
- Researcher
- Ray GPU Cluster
- Host Sysfs
- Workload and Links
- Kubernetes KAI
- Dynamo Runtime
Connections 30
- Researcher → GPU Inventory: requests discovery
- GPU Inventory → Ray GPU Cluster: probes nodes
- GPU Inventory → Planner Nodes: derives inputs
- Researcher → NIC Inventory: requests discovery
- NIC Inventory → Ray GPU Cluster: probes nodes
- NIC Inventory → Host Sysfs: reads attributes
- Researcher → Workload and Links: supplies profile
- Researcher → Placement Policies: chooses policy
- Planner Nodes → Placement Policies: provides nodes
- Workload and Links → Placement Policies: provides costs
- Placement Policies → Placement Plan: selects placement
- Researcher → Comparison Runner: runs comparison
- Comparison Runner → Placement Policies: plans jobs
- Comparison Runner → Ray Executor: dispatches backend
- Comparison Runner → KAI Adapter: dispatches backend
- Comparison Runner → JCT Records: captures outcomes
- Researcher → Ray Executor: runs plan
- Placement Plan → Ray Executor: reserves placement
- Ray Executor → Ray GPU Cluster: reserves resources
- Ray Executor → Ray Worker Tasks: launches tasks
- Researcher → KAI Adapter: submits plan
- Placement Plan → KAI Adapter: maps workers
- KAI Adapter → Kubernetes KAI: reads and manages
- KAI Adapter → PodGroup and Pods: builds manifests
- KAI Adapter → Kubernetes KAI: submits and watches
- Researcher → Dynamo Lifecycle: starts service
- Placement Plan → Dynamo Lifecycle: assigns replicas
- Dynamo Lifecycle → Ray GPU Cluster: reserves actors
- Dynamo Lifecycle → Process Guardian: launches guardian
- Process Guardian → Dynamo Runtime: starts and stops