# nextflow-io/nextflow architecture

> Architecture diagram of the GitHub repository nextflow-io/nextflow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nextflow-io/nextflow
- Repository: https://github.com/nextflow-io/nextflow

## Overview

Nextflow turns a user-authored workflow and configuration into a compiled dataflow program, schedules process tasks, manages reproducibility, and materializes outputs. The sampled code strongly supports configuration parsing, AST construction, task hashing, file handling, and S3 transfer control; broader runtime and platform capabilities are represented from the README and directory structure, with unsampled links kept at subsystem boundaries. Optional cloud and scheduler integrations remain separate from the core workflow path.

Main entry points: [`modules/nextflow`](https://github.com/nextflow-io/nextflow/tree/master/modules/nextflow) (Task executor), [`modules/nextflow/src/main/groovy/nextflow/script/parser/v2/ScriptCompiler.java`](https://github.com/nextflow-io/nextflow/blob/master/modules/nextflow/src/main/groovy/nextflow/script/parser/v2/ScriptCompiler.java) (Script compiler), [`modules/nf-lang/src/main/java/nextflow/config/control/ConfigParser.java`](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/config/control/ConfigParser.java) (Config parser), [`modules/nf-lang/src/main/java/nextflow/script/control/ScriptParser.java`](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/script/control/ScriptParser.java) (Script parser)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Language frontend"]
  node_script_compiler["Script compiler"]
  node_script_parser["Script parser<br/>[ScriptParser.java]"]
  node_script_ast["Script AST<br/>[ScriptNode.java]"]
  node_config_parser["Config parser<br/>[ConfigParser.java]"]
  node_config_ast["Config AST"]
  node_config_resolver["Config resolver"]
end

subgraph group_runtime["Workflow runtime"]
  node_workflow_runtime["Workflow runtime"]
  node_workflow_dsl["Workflow DSL<br/>[WorkflowDsl.java]"]
  node_channels["Dataflow channels<br/>[Channel.java]"]
  node_task_closure["Task closure<br/>[TaskClosure.java]"]
  node_task_executor["Task executor"]
end

subgraph group_repro["Reproducibility"]
  node_task_hash["Task hashing<br/>[HashBuilder.java]"]
  node_task_cache["Task cache<br/>[CacheHelper.java]"]
  node_file_outputs["File outputs"]
  node_software_env["Software environments"]
end

subgraph group_execution["Execution integrations"]
  node_local_execution["Local execution"]
  node_aws_execution["AWS execution"]
  node_azure_execution["Azure execution"]
  node_google_execution["Google execution"]
  node_k8s_execution["Kubernetes execution"]
  node_s3_storage["S3 storage<br/>[S3Client.java]"]
end

node_user(("Workflow author"))
node_local_machine{{"Local machine"}}
node_hpc_scheduler{{"HPC scheduler"}}
node_aws_batch{{"AWS Batch"}}
node_azure_batch{{"Azure Batch"}}
node_google_batch{{"Google Cloud Batch"}}
node_kubernetes_cluster{{"Kubernetes cluster"}}
node_s3_service{{"Amazon S3"}}

node_user -->|"provides script"| node_script_compiler
node_user -->|"provides config"| node_config_parser
node_script_compiler -->|"parses script"| node_script_parser
node_script_parser -->|"builds AST"| node_script_ast
node_script_compiler -->|"starts workflow"| node_workflow_runtime
node_config_parser -->|"builds AST"| node_config_ast
node_config_parser -->|"resolves config"| node_config_resolver
node_config_resolver -->|"supplies settings"| node_workflow_runtime
node_workflow_runtime -->|"interprets DSL"| node_workflow_dsl
node_workflow_runtime -->|"creates channels"| node_channels
node_workflow_runtime -->|"dispatches tasks"| node_task_closure
node_workflow_runtime -->|"schedules tasks"| node_task_executor
node_task_executor -->|"computes hashes"| node_task_hash
node_task_executor -->|"reads cache"| node_task_cache
node_task_executor -->|"prepares environment"| node_software_env
node_task_executor -->|"writes outputs"| node_file_outputs
node_task_executor -->|"runs locally"| node_local_execution
node_task_executor -.->|"dispatches tasks"| node_aws_execution
node_task_executor -.->|"dispatches tasks"| node_azure_execution
node_task_executor -.->|"dispatches tasks"| node_google_execution
node_task_executor -.->|"dispatches tasks"| node_k8s_execution
node_local_execution -->|"runs on"| node_local_machine
node_local_execution -.->|"submits to"| node_hpc_scheduler
node_aws_execution -->|"submits to"| node_aws_batch
node_azure_execution -->|"submits to"| node_azure_batch
node_google_execution -->|"submits to"| node_google_batch
node_k8s_execution -->|"submits to"| node_kubernetes_cluster
node_task_executor -.->|"transfers files"| node_s3_storage
node_s3_storage -->|"reads/writes"| node_s3_service
node_file_outputs -->|"returns results"| node_user

click node_script_compiler "https://github.com/nextflow-io/nextflow/blob/master/modules/nextflow/src/main/groovy/nextflow/script/parser/v2/ScriptCompiler.java"
click node_script_parser "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/script/control/ScriptParser.java"
click node_script_ast "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/script/ast/ScriptNode.java"
click node_config_parser "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/config/control/ConfigParser.java"
click node_config_ast "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/config/parser/ConfigParserPlugin.java"
click node_config_resolver "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/config/control/ConfigResolveVisitor.java"
click node_workflow_runtime "https://github.com/nextflow-io/nextflow/tree/master/modules/nextflow"
click node_workflow_dsl "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/script/dsl/WorkflowDsl.java"
click node_channels "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/script/types/Channel.java"
click node_task_closure "https://github.com/nextflow-io/nextflow/blob/master/modules/nextflow/src/main/groovy/nextflow/script/TaskClosure.java"
click node_task_executor "https://github.com/nextflow-io/nextflow/tree/master/modules/nextflow"
click node_task_hash "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-commons/src/main/nextflow/util/HashBuilder.java"
click node_task_cache "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-commons/src/main/nextflow/util/CacheHelper.java"
click node_file_outputs "https://github.com/nextflow-io/nextflow/blob/master/modules/nf-commons/src/main/nextflow/file/CopyMoveHelper.java"
click node_software_env "https://github.com/nextflow-io/nextflow/tree/master/modules/nextflow"
click node_local_execution "https://github.com/nextflow-io/nextflow/tree/master/modules/nextflow"
click node_aws_execution "https://github.com/nextflow-io/nextflow/tree/master/plugins/nf-amazon"
click node_azure_execution "https://github.com/nextflow-io/nextflow/tree/master/plugins/nf-azure"
click node_google_execution "https://github.com/nextflow-io/nextflow/tree/master/plugins/nf-google"
click node_k8s_execution "https://github.com/nextflow-io/nextflow/tree/master/plugins/nf-k8s"
click node_s3_storage "https://github.com/nextflow-io/nextflow/blob/master/plugins/nf-amazon/src/main/nextflow/cloud/aws/nio/S3Client.java"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_script_compiler,node_script_parser,node_script_ast,node_config_parser,node_config_ast,node_config_resolver toneBlue
class node_workflow_runtime,node_workflow_dsl,node_channels,node_task_closure,node_task_executor toneAmber
class node_task_hash,node_task_cache,node_file_outputs,node_software_env toneMint
class node_local_execution,node_aws_execution,node_azure_execution,node_google_execution,node_k8s_execution,node_s3_storage,node_hpc_scheduler toneRose
class node_user,node_local_machine,node_aws_batch,node_azure_batch,node_google_batch,node_kubernetes_cluster,node_s3_service toneIndigo
```

## Components

### Language frontend

- [Script compiler](https://github.com/nextflow-io/nextflow/blob/master/modules/nextflow/src/main/groovy/nextflow/script/parser/v2/ScriptCompiler.java): `modules/nextflow/src/main/groovy/nextflow/script/parser/v2/ScriptCompiler.java`
- [Script parser](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/script/control/ScriptParser.java): `modules/nf-lang/src/main/java/nextflow/script/control/ScriptParser.java`
- [Script AST](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/script/ast/ScriptNode.java): `modules/nf-lang/src/main/java/nextflow/script/ast/ScriptNode.java`
- [Config parser](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/config/control/ConfigParser.java): `modules/nf-lang/src/main/java/nextflow/config/control/ConfigParser.java`
- [Config AST](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/config/parser/ConfigParserPlugin.java): `modules/nf-lang/src/main/java/nextflow/config/parser/ConfigParserPlugin.java`
- [Config resolver](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/config/control/ConfigResolveVisitor.java): `modules/nf-lang/src/main/java/nextflow/config/control/ConfigResolveVisitor.java`

### Workflow runtime

- [Workflow runtime](https://github.com/nextflow-io/nextflow/tree/master/modules/nextflow): `modules/nextflow`
- [Workflow DSL](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/script/dsl/WorkflowDsl.java): `modules/nf-lang/src/main/java/nextflow/script/dsl/WorkflowDsl.java`
- [Dataflow channels](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-lang/src/main/java/nextflow/script/types/Channel.java): `modules/nf-lang/src/main/java/nextflow/script/types/Channel.java`
- [Task closure](https://github.com/nextflow-io/nextflow/blob/master/modules/nextflow/src/main/groovy/nextflow/script/TaskClosure.java): `modules/nextflow/src/main/groovy/nextflow/script/TaskClosure.java`
- [Task executor](https://github.com/nextflow-io/nextflow/tree/master/modules/nextflow): `modules/nextflow`

### Reproducibility

- [Task hashing](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-commons/src/main/nextflow/util/HashBuilder.java): `modules/nf-commons/src/main/nextflow/util/HashBuilder.java`
- [Task cache](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-commons/src/main/nextflow/util/CacheHelper.java): `modules/nf-commons/src/main/nextflow/util/CacheHelper.java`
- [File outputs](https://github.com/nextflow-io/nextflow/blob/master/modules/nf-commons/src/main/nextflow/file/CopyMoveHelper.java): `modules/nf-commons/src/main/nextflow/file/CopyMoveHelper.java`
- [Software environments](https://github.com/nextflow-io/nextflow/tree/master/modules/nextflow): `modules/nextflow`

### Execution integrations

- [Local execution](https://github.com/nextflow-io/nextflow/tree/master/modules/nextflow): `modules/nextflow`
- [AWS execution](https://github.com/nextflow-io/nextflow/tree/master/plugins/nf-amazon): `plugins/nf-amazon`
- [Azure execution](https://github.com/nextflow-io/nextflow/tree/master/plugins/nf-azure): `plugins/nf-azure`
- [Google execution](https://github.com/nextflow-io/nextflow/tree/master/plugins/nf-google): `plugins/nf-google`
- [Kubernetes execution](https://github.com/nextflow-io/nextflow/tree/master/plugins/nf-k8s): `plugins/nf-k8s`
- [S3 storage](https://github.com/nextflow-io/nextflow/blob/master/plugins/nf-amazon/src/main/nextflow/cloud/aws/nio/S3Client.java): `plugins/nf-amazon/src/main/nextflow/cloud/aws/nio/S3Client.java`

### Other components

- **Workflow author**
- **Local machine**
- **HPC scheduler**
- **AWS Batch**
- **Azure Batch**
- **Google Cloud Batch**
- **Kubernetes cluster**
- **Amazon S3**

## Connections

- Workflow author → Script compiler: provides script
- Workflow author → Config parser: provides config
- Script compiler → Script parser: parses script
- Script parser → Script AST: builds AST
- Script compiler → Workflow runtime: starts workflow
- Config parser → Config AST: builds AST
- Config parser → Config resolver: resolves config
- Config resolver → Workflow runtime: supplies settings
- Workflow runtime → Workflow DSL: interprets DSL
- Workflow runtime → Dataflow channels: creates channels
- Workflow runtime → Task closure: dispatches tasks
- Workflow runtime → Task executor: schedules tasks
- Task executor → Task hashing: computes hashes
- Task executor → Task cache: reads cache
- Task executor → Software environments: prepares environment
- Task executor → File outputs: writes outputs
- Task executor → Local execution: runs locally
- Task executor → AWS execution: dispatches tasks
- Task executor → Azure execution: dispatches tasks
- Task executor → Google execution: dispatches tasks
- Task executor → Kubernetes execution: dispatches tasks
- Local execution → Local machine: runs on
- Local execution → HPC scheduler: submits to
- AWS execution → AWS Batch: submits to
- Azure execution → Azure Batch: submits to
- Google execution → Google Cloud Batch: submits to
- Kubernetes execution → Kubernetes cluster: submits to
- Task executor → S3 storage: transfers files
- S3 storage → Amazon S3: reads/writes
- File outputs → Workflow author: returns results
