How rocm/cvs works
CVS is an SSH-based suite for end-to-end AMD AI cluster validation, with CLI-driven tests, cluster orchestration, and training/inference workloads producing reports. The repository also contains two substantial monitoring products: a cluster health dashboard and a GPU fleet manager. The graph emphasizes those runtime capabilities and their domain stages. Several links to unsampled subsystems are omitted where the excerpts and index do not establish a caller relationship.
Main entry points: main.py Fleet manager API · _base_runner.py Suite runner · container.py Container orchestration · baremetal.py Cluster orchestration
Components 26
CLI and inputs
- CVS CLI
cvs/main.py - CLI commands
cvs/cli_plugins - Config generation
cvs/input/generate
Validation runtime
- Suite runner
cvs/runners/_base_runner.py - Aorta runner
cvs/runners/aorta.py - Cluster orchestration
cvs/core/orchestrators/baremetal.py - Container orchestration
cvs/core/orchestrators/container.py - Host scheduler
cvs/core/scheduler.py - Docker runtime
cvs/core/runtimes/docker.py - SSH execution
cvs/lib/parallel_ssh_lib.py
Workload suites
- Training suites
cvs/lib/training - Inference suites
cvs/lib/inference - vLLM benchmark
cvs/lib/inference/vllm_job.py - Model endpoint checks
cvs/lib/utils/model_query_lib.py - Inference reports
cvs/lib/report/inference_payload.py
Cluster monitor
- Cluster dashboard
cvs/monitors/cluster-mon/frontend/src/App.tsx - Monitor API
cvs/monitors/cluster-mon/backend/app/main.py - Health collectors
cvs/monitors/cluster-mon/backend/app/collectors - Monitor SSH manager
cvs/monitors/cluster-mon/backend/app/core/ssh_manager.py - Go collector
cvs/monitors/cluster-mon/backend/app/core/go_collector.py
GPU fleet manager
- Fleet manager UI
cvs/monitors/metrics_exp/server/ui/src/App.tsx - Fleet manager API
cvs/monitors/metrics_exp/server/fleet_manager/main.py - Fleet API routes
cvs/monitors/metrics_exp/server/fleet_manager/api/routes - Fleet database
cvs/monitors/metrics_exp/server/fleet_manager/models/database.py - Fleet services
cvs/monitors/metrics_exp/server/fleet_manager/services
Other components
- Operator
Connections 24
- Operator → CVS CLI: invokes
- CVS CLI → CLI commands: discovers and dispatches
cvs/main.py - CLI commands → Config generation: generates configs
cvs/input/generate/cluster_json.py - Operator → Suite runner: runs suites
- Aorta runner → Suite runner: uses base runner
cvs/runners/aorta.py - Aorta runner → Docker runtime: uses runtime
cvs/runners/aorta.py - Cluster orchestration → Host scheduler: uses
cvs/core/orchestrators/baremetal.py - Container orchestration → Cluster orchestration: reuses
cvs/core/orchestrators/container.py - Container orchestration → Host scheduler: uses
cvs/core/orchestrators/container.py - Container orchestration → Docker runtime: selects runtime
cvs/core/orchestrators/container.py - Cluster orchestration → SSH execution: executes remotely
- Suite runner → Training suites: runs workloads
- Suite runner → Inference suites: runs workloads
- Inference suites → vLLM benchmark: includes job
cvs/lib/inference/vllm_job.py - vLLM benchmark → Model endpoint checks: checks served model
cvs/lib/inference/vllm_job.py - Inference suites → Inference reports: builds report payload
cvs/lib/report/inference_payload.py - Cluster dashboard → Monitor API: requests monitor data
- Monitor API → Health collectors: configures collectors
cvs/monitors/cluster-mon/backend/app/main.py - Monitor API → Monitor SSH manager: creates manager
cvs/monitors/cluster-mon/backend/app/main.py - Monitor SSH manager → Go collector: delegates execution
cvs/monitors/cluster-mon/backend/app/core/ssh_manager.py - Fleet manager UI → Fleet manager API: uses API
- Fleet manager API → Fleet API routes: registers routes
cvs/monitors/metrics_exp/server/fleet_manager/main.py - Fleet manager API → Fleet database: queries database
cvs/monitors/metrics_exp/server/fleet_manager/main.py - Fleet manager API → Fleet services: uses services
cvs/monitors/metrics_exp/server/fleet_manager/main.py