# volcano-sh/volcano architecture

> Architecture diagram of the GitHub repository volcano-sh/volcano, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/volcano-sh/volcano
- Repository: https://github.com/volcano-sh/volcano

## Overview

##### Project Type and Purpose
- Volcano is a Kubernetes-native batch scheduling system (an “operator” plus custom scheduler) that extends kube-scheduler via CustomResourceDefinitions (CRDs) and admission webhooks to manage high-performance, AI/ML, Big Data, and HPC workloads at scale.

##### File‐Structure Analysis
- Top‐level “cmd/…”: entrypoints for each microservice
- cmd/agent            – node-level agent running on each worker
- cmd/controller-manager – controllers watching CRDs (Job, PodGroup, Queue, JobFlow, JobTemplate)
- cmd/scheduler        – custom scheduler implementing plugin framework
- cmd/webhook-manager – admission webhook server for mutation/validation of CRs
- cmd/cli              – kubectl-style CLI (vcctl, vsub, vjobs, etc.)
- “config/crd”: CRD YAML definitions (bases & v1beta1) for Job, PodGroup, Queue, JobFlow, JobTemplate
- “pkg/…”: implementation packages for each component:
- pkg/agent, pkg/networkqos, pkg/resourceusage, pkg/metriccollect – node-side metrics, QoS, resource reporting
- pkg/controllers – reconciliation loops (operator controllers)
- pkg/scheduler – scheduling framework, plugins, cache, actions
- pkg/webhooks – admission mutations & validations
- pkg/cli – API client, command logic
- “installer/helm” & YAML: deployment artifacts (namespace, Deployments, Services) and Helm charts
- “docs/images/volcano-architecture.png”: high-level architecture reference

##### Key Components & Their Interactions
A. Client Layer
- CLI (vcctl/vsub/vjobs…)
↔ Kubernetes API Server (reads/writes Volcano CRs)
B. Kubernetes Control Plane
- API Server with CRDs (Job, PodGroup, Queue, JobFlow, JobTemplate)
- Admission Webhook (volcano-admission) intercepts CR creations/updates for validation & defaulting
C. Volcano Control Plane Services (in “volcano-system” namespace)
1. Controller-Manager
- Watches Volcano CRs
- Reconciles desired state: creates/deletes Pods, PodGroups, JobFlow steps, manages Queue status
2. Custom Scheduler
- Watches unscheduled Pods with schedulerName=volcano
- Uses plugin framework (enqueue, allocate, preempt, backfill, reclaim, shuffle, etc.)
- Binds Pods to Nodes via API Server
D. Node Agent
- Runs on each worker node
- Collects node metrics, QoS events, oversubscription information
- Reports to API Server/Controller side via Status fields
E. External Systems
- etcd (Kubernetes datastore)
- Prometheus & Grafana (optional monitoring stack installed via installer)
- Benchmark scripts and e2e tests (under test/ & benchmark/)

##### Architectural Patterns & Design Principles
- Operator Pattern: custom controllers reconciling CR state
- Microservices: each “cmd/…” component is a separate Deployment
- Plugin Framework: scheduler loaded with independent “actions” & “plugins” (binpack, drf, capacity, cdp, nodegroup, task-topology, etc.)
- Admission-Control: validate & mutate Volcano custom objects at creation time
- Event-Driven: controllers and scheduler driven by informers/list-watch on CRs and core objects
- Separation of Concerns:
- CLI: presentation & client
- Admission: request shaping
- Controllers: control loops
- Scheduler: placement logic
- Agent: node internals

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    User["User"]:::external
    CLI["CLI (vcctl/vsub/vjobs...)"]:::cli

    %% Kubernetes Cluster Boundary
    subgraph "Kubernetes Cluster"
        direction TB
        APIServer["API Server + CRDs"]:::control
        Admission["Admission Webhook"]:::control
        Controller["Controller-Manager"]:::control
        Scheduler["Scheduler (plugins/actions)"]:::control
        NodeAgent["Node Agent"]:::node
        etcd["etcd"]:::external
    end

    %% Monitoring
    Prometheus["Prometheus"]:::external
    Grafana["Grafana"]:::external

    %% Flows
    User -->|"uses"| CLI
    CLI -->|"kubectl API calls"| APIServer
    APIServer -->|"validate & mutate"| Admission
    Admission -->|"admission result"| APIServer
    APIServer -->|"watches CRs"| Controller
    Controller -->|"creates/updates Pods & Status"| APIServer
    APIServer -->|"unscheduled Pods"| Scheduler
    Scheduler -->|"binds Pods"| APIServer
    NodeAgent -->|"reports metrics/events"| APIServer
    Prometheus -->|"scrapes metrics"| CLI
    Prometheus -->|"scrapes metrics"| Admission
    Prometheus -->|"scrapes metrics"| Controller
    Prometheus -->|"scrapes metrics"| Scheduler
    Prometheus -->|"scrapes metrics"| NodeAgent
    Grafana -->|"visualizes"| Prometheus

    %% Click Events
    click CLI "https://github.com/volcano-sh/volcano/tree/master/cmd/cli/"
    click CLI "https://github.com/volcano-sh/volcano/tree/master/pkg/cli/"
    click APIServer "https://github.com/volcano-sh/volcano/tree/master/config/crd/volcano/bases/"
    click APIServer "https://github.com/volcano-sh/volcano/tree/master/config/crd/volcano/v1beta1/"
    click APIServer "https://github.com/volcano-sh/volcano/tree/master/config/crd/jobflow/bases/"
    click APIServer "https://github.com/volcano-sh/volcano/tree/master/config/crd/jobflow/v1beta1/"
    click Admission "https://github.com/volcano-sh/volcano/tree/master/cmd/webhook-manager/"
    click Admission "https://github.com/volcano-sh/volcano/tree/master/pkg/webhooks/"
    click Controller "https://github.com/volcano-sh/volcano/tree/master/cmd/controller-manager/"
    click Controller "https://github.com/volcano-sh/volcano/tree/master/pkg/controllers/"
    click Scheduler "https://github.com/volcano-sh/volcano/tree/master/cmd/scheduler/"
    click Scheduler "https://github.com/volcano-sh/volcano/tree/master/pkg/scheduler/"
    click NodeAgent "https://github.com/volcano-sh/volcano/tree/master/cmd/agent/"
    click NodeAgent "https://github.com/volcano-sh/volcano/tree/master/pkg/agent/"

    %% Styles
    classDef control fill:#cce5ff,stroke:#000,color:#000
    classDef node fill:#d4edda,stroke:#000,color:#000
    classDef external fill:#ffe5b4,stroke:#000,color:#000
    classDef cli fill:#f0f0f0,stroke:#000,color:#000
```
