# cylc/cylc-flow architecture

> Architecture diagram of the GitHub repository cylc/cylc-flow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cylc/cylc-flow
- Repository: https://github.com/cylc/cylc-flow

## Overview

Cylc Flow is a general-purpose workflow engine with explicit support for cycling workflows, branching, HPC execution, and runtime monitoring and control. The principal path is workflow authoring and validation, installation and launch, dependency-aware scheduling, task dispatch to execution platforms, and persisted/observable workflow state. The graph includes the CLI, scheduler and task runtime, configuration and cycling, job execution, state, and communications/UI branches. README and tree establish product capabilities; sampled excerpts specifically support graph parsing, network client/server behavior, and the TUI. Other internal wiring is shown only where the product structure supports it.

Main entry points: [`cylc/flow/scheduler.py`](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scheduler.py) (Scheduler), [`cylc/flow/config.py`](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/config.py) (Config loading), [`cylc/flow/task_pool.py`](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/task_pool.py) (Task pool), [`cylc/flow/network/client.py`](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/network/client.py) (Runtime API client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cli["Workflow CLI"]
  node_validate["Validate command<br/>[validate.py]"]
  node_install["Install command<br/>[install.py]"]
  node_play["Play command<br/>[play.py]"]
end

subgraph group_definition["Workflow Definition"]
  node_definition_files["Workflow files<br/>[workflow_files.py]"]
  node_configuration["Config loading<br/>[config.py]"]
  node_config_spec["Workflow config spec<br/>[workflow.py]"]
  node_config_validation["Config validation<br/>[validate.py]"]
  node_graph_parser["Dependency graph parser<br/>[graph_parser.py]"]
  node_cycling["Cycle arithmetic<br/>[loader.py]"]
end

subgraph group_runtime["Scheduler Runtime"]
  node_scheduler["Scheduler<br/>[scheduler.py]"]
  node_task_pool["Task pool<br/>[task_pool.py]"]
  node_task_proxy["Task state and proxy<br/>[task_proxy.py]"]
  node_xtriggers["External triggers<br/>[xtrigger_mgr.py]"]
end

subgraph group_execution["Task Execution"]
  node_job_manager["Job manager<br/>[task_job_mgr.py]"]
  node_job_runners["Platform job runners<br/>[job_runner_mgr.py]"]
end

subgraph group_state["State and Control"]
  node_workflow_db[("Workflow database<br/>[workflow_db_mgr.py]")]
  node_data_store["Workflow data store<br/>[data_store_mgr.py]"]
  node_runtime_commands["Runtime commands<br/>[trigger.py]"]
end

subgraph group_comms["Communications and UI"]
  node_network_server["Scheduler API server<br/>[server.py]"]
  node_network_client["Runtime API client<br/>[client.py]"]
  node_tui["Terminal interface<br/>[app.py]"]
end

node_user(("Workflow user"))
node_hpc{{"Execution platform"}}
node_uiserver(("Web UI service"))

node_user -->|"validates"| node_validate
node_user -->|"installs"| node_install
node_user -->|"starts"| node_play
node_validate -->|"loads config"| node_configuration
node_install -->|"installs workflow"| node_definition_files
node_play -->|"starts scheduler"| node_scheduler
node_configuration -->|"reads definition"| node_definition_files
node_configuration -->|"uses schema"| node_config_spec
node_config_spec -->|"validates values"| node_config_validation
node_configuration -->|"parses dependencies"| node_graph_parser
node_configuration -->|"resolves cycling"| node_cycling
node_scheduler -->|"loads workflow"| node_configuration
node_scheduler -->|"schedules tasks"| node_task_pool
node_scheduler -->|"persists state"| node_workflow_db
node_scheduler -->|"updates workflow data"| node_data_store
node_scheduler -->|"hosts API"| node_network_server
node_task_pool -->|"manages tasks"| node_task_proxy
node_task_pool -->|"checks triggers"| node_xtriggers
node_task_pool -->|"dispatches jobs"| node_job_manager
node_job_manager -->|"selects runner"| node_job_runners
node_job_runners -->|"submits jobs"| node_hpc
node_hpc -->|"reports job results"| node_job_manager
node_runtime_commands -->|"issues requests"| node_network_client
node_network_client -->|"sends requests"| node_network_server
node_network_server -->|"reads workflow data"| node_data_store
node_user -->|"monitors workflow"| node_tui
node_tui -->|"requests updates"| node_network_client
node_uiserver -.->|"connects optionally"| node_network_client

click node_validate "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scripts/validate.py"
click node_install "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scripts/install.py"
click node_play "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scripts/play.py"
click node_definition_files "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/workflow_files.py"
click node_configuration "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/config.py"
click node_config_spec "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/cfgspec/workflow.py"
click node_config_validation "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/parsec/validate.py"
click node_graph_parser "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/graph_parser.py"
click node_cycling "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/cycling/loader.py"
click node_scheduler "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scheduler.py"
click node_task_pool "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/task_pool.py"
click node_task_proxy "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/task_proxy.py"
click node_xtriggers "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/xtrigger_mgr.py"
click node_job_manager "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/task_job_mgr.py"
click node_job_runners "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/job_runner_mgr.py"
click node_workflow_db "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/workflow_db_mgr.py"
click node_data_store "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/data_store_mgr.py"
click node_runtime_commands "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scripts/trigger.py"
click node_network_server "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/network/server.py"
click node_network_client "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/network/client.py"
click node_tui "https://github.com/cylc/cylc-flow/blob/master/cylc/flow/tui/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_validate,node_install,node_play,node_user,node_uiserver toneBlue
class node_definition_files,node_configuration,node_config_spec,node_config_validation,node_graph_parser,node_cycling toneAmber
class node_scheduler,node_task_pool,node_task_proxy,node_xtriggers toneMint
class node_job_manager,node_job_runners toneRose
class node_workflow_db,node_data_store,node_runtime_commands,node_hpc toneIndigo
class node_network_server,node_network_client,node_tui toneTeal
```

## Components

### Workflow CLI

- [Validate command](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scripts/validate.py): `cylc/flow/scripts/validate.py`
- [Install command](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scripts/install.py): `cylc/flow/scripts/install.py`
- [Play command](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scripts/play.py): `cylc/flow/scripts/play.py`

### Workflow Definition

- [Workflow files](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/workflow_files.py): `cylc/flow/workflow_files.py`
- [Config loading](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/config.py): `cylc/flow/config.py`
- [Workflow config spec](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/cfgspec/workflow.py): `cylc/flow/cfgspec/workflow.py`
- [Config validation](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/parsec/validate.py): `cylc/flow/parsec/validate.py`
- [Dependency graph parser](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/graph_parser.py): `cylc/flow/graph_parser.py`
- [Cycle arithmetic](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/cycling/loader.py): `cylc/flow/cycling/loader.py`

### Scheduler Runtime

- [Scheduler](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scheduler.py): `cylc/flow/scheduler.py`
- [Task pool](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/task_pool.py): `cylc/flow/task_pool.py`
- [Task state and proxy](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/task_proxy.py): `cylc/flow/task_proxy.py`
- [External triggers](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/xtrigger_mgr.py): `cylc/flow/xtrigger_mgr.py`

### Task Execution

- [Job manager](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/task_job_mgr.py): `cylc/flow/task_job_mgr.py`
- [Platform job runners](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/job_runner_mgr.py): `cylc/flow/job_runner_mgr.py`

### State and Control

- [Workflow database](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/workflow_db_mgr.py): `cylc/flow/workflow_db_mgr.py`
- [Workflow data store](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/data_store_mgr.py): `cylc/flow/data_store_mgr.py`
- [Runtime commands](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/scripts/trigger.py): `cylc/flow/scripts/trigger.py`

### Communications and UI

- [Scheduler API server](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/network/server.py): `cylc/flow/network/server.py`
- [Runtime API client](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/network/client.py): `cylc/flow/network/client.py`
- [Terminal interface](https://github.com/cylc/cylc-flow/blob/master/cylc/flow/tui/app.py): `cylc/flow/tui/app.py`

### Other components

- **Workflow user**
- **Execution platform**
- **Web UI service**

## Connections

- Workflow user → Validate command: validates
- Workflow user → Install command: installs
- Workflow user → Play command: starts
- Validate command → Config loading: loads config
- Install command → Workflow files: installs workflow
- Play command → Scheduler: starts scheduler
- Config loading → Workflow files: reads definition
- Config loading → Workflow config spec: uses schema
- Workflow config spec → Config validation: validates values
- Config loading → Dependency graph parser: parses dependencies
- Config loading → Cycle arithmetic: resolves cycling
- Scheduler → Config loading: loads workflow
- Scheduler → Task pool: schedules tasks
- Scheduler → Workflow database: persists state
- Scheduler → Workflow data store: updates workflow data
- Scheduler → Scheduler API server: hosts API
- Task pool → Task state and proxy: manages tasks
- Task pool → External triggers: checks triggers
- Task pool → Job manager: dispatches jobs
- Job manager → Platform job runners: selects runner
- Platform job runners → Execution platform: submits jobs
- Execution platform → Job manager: reports job results
- Runtime commands → Runtime API client: issues requests
- Runtime API client → Scheduler API server: sends requests
- Scheduler API server → Workflow data store: reads workflow data
- Workflow user → Terminal interface: monitors workflow
- Terminal interface → Runtime API client: requests updates
- Web UI service → Runtime API client: connects optionally