How nf-core/bacass works
This repository is an nf-core Nextflow DSL2 pipeline for bacterial genome assembly and annotation, centered on a single orchestrated workflow with many reusable process modules underneath.
High-level architecture
The repo is organized into five main layers:
1. Pipeline entry and orchestration
2. Reusable process modules
3. Subworkflows that connect modules into higher-level stages
4. Configuration, schemas, and auxiliary scripts/templates
5. Testing and CI/CD support
The core runtime model is: a samplesheet is parsed, samples are branched by sequencing strategy, reads are QC’d/trimmed, assembled, optionally contamination-checked and reference-grouped, then annotated and summarized into final reports.
Core orchestration boundary
The main workflow logic lives in:
• main.nf
• workflows/bacass.nf
These are the top-level Nextflow orchestration points. They are responsible for:
• reading pipeline parameters and the input samplesheet
• selecting one of the supported assembly modes:
- short-read
- long-read
- hybrid
• wiring together the module and subworkflow graph
• collecting outputs for final reporting and publication artifacts
The workflow is not a monolith: most actual work is delegated to modules and subworkflows, which keeps the topology composable and testable.
Input model and data flow
The pipeline expects a tabular samplesheet with columns like:
• ID
• R1
• R2
• LongFastQ
• Fast5
• GenomeSize
This input model drives branching through the pipeline:
• paired short reads are routed through trimming/QC and short-read assembly paths
• long-read samples are routed through long-read QC and assembler/polisher paths
• samples with both short and long reads are routed through hybrid assembly paths
The GenomeSize field is relevant for long-read assembly tools that benefit from an expected genome size.
Primary processing stages
1. Read preprocessing and QC
Short-read path:
• fastp for trimming/filtering
• fastqc for read-level QC
Long-read path:
• porechop or filtlong for read cleanup/filtering
• nanoplot and pycoqc for long-read QC summaries
• gunzip / untar where needed to unpack compressed inputs or databases
There is also a combined QC subworkflow:
• subworkflows/local/qc_nanoplot_toulliqc
• modules/nf-core/toulligqc
This indicates a long-read QC aggregation branch that combines outputs from multiple tools into a coherent sample-level report.
2. Assembly
Short-read assembly:
• unicycler
Long-read assembly options:
• dragonflye
• miniasm
• canu
Long-read polishing:
• racon
• medaka
• nanopolish
Hybrid assembly:
• unicycler can be used with both read types
• dragonflye can also be used in a hybrid context according to the README, though operationally it still sits in the long-read assembly ecosystem and is fed additional short-read information for polishing
This assembly layer is one of the key architectural boundaries: the pipeline branches heavily here based on assembly_type and assembler.
3. Post-assembly QC and contamination checks
After assembly, the pipeline performs assembly validation and contamination screening:
• quast for assembly quality statistics
• busco for completeness assessment
• kraken2 for taxonomic contamination screening
• kmerfinder for reference-genome identification / purity checking
These tools feed into the downstream grouping and reporting logic.
4. Annotation
Assembled contigs can be annotated using one of:
• prokka
• bakta
• dfast
This is another explicit branch point in the workflow. Annotation happens after assembly QC and is part of the final deliverables for the bacterial genome.
5. Reporting and aggregation
Final reporting is built from several layers:
• multiqc for consolidated QC reporting
• custom multiqc configuration under assets/
• bin/multiqc_to_custom_csv.py for translating MultiQC outputs into a custom tabular summary
• modules/nf-core/custom/dumpsoftwareversions for provenance capture
• bin/kmerfinder_summary.py and bin/find_common_reference.py for Kmerfinder-oriented aggregation
• bin/csv_to_yaml.py and bin/download_reference.py for metadata/reference handling
There is also a reference-grouping flow when Kmerfinder is used:
• samples are grouped by estimated reference genome
• one QUAST pass runs on samples without reference genomes
• a second QUAST pass runs “by reference genome” for grouped samples
This creates a distinct post-processing branch that only exists when Kmerfinder-driven grouping is enabled.
Module organization
Most computational steps are provided as nf-core modules under modules/nf-core/.
These are standard reusable DSL2 modules, typically one tool per module, with their own:
• main.nf
• meta.yml
• environment.yml
• module-specific tests
This is the “tool execution” layer.
Examples of major nf-core modules used here:
• fastp
• fastqc
• unicycler
• dragonflye
• miniasm
• canu
• racon
• medaka
• nanopolish
• kraken2
• quast
• busco
• prokka
• bakta
• liftoff
• multiqc
• samtools
• porechop
• filtlong
• nanoplot
• toulligqc
• untar
• gunzip
Local/custom modules
The repository also includes several custom modules under modules/local/ for behavior that is pipeline-specific:
• kmerfinder submodules:
- download_reference
- kmerfinder
- summary
• kraken2/db_preparation
• dfast
• medaka
• nanopolish
• pycoqc
• custom/multiqc
These local modules are important architectural markers because they represent logic not directly covered by upstream nf-core modules, especially around:
• database download/preparation
• custom report formatting
• pipeline-specific wrappers around tools
• specialized QC and summary generation
Subworkflows as composition units
The subworkflows/ directory shows the pipeline’s intermediate composition layer. These are not low-level tool wrappers; they stitch modules into reusable stages.
Key subworkflows include:
• subworkflows/nf-core/fastq_trim_fastp_fastqc
- a short-read preprocessing unit that chains trimming and QC
• subworkflows/local/qc_nanoplot_toulliqc
- a long-read QC aggregation subworkflow
• subworkflows/local/kmerfinder_summary_download
- combines Kmerfinder summarization with reference download logic
• subworkflows/local/bakta_dbdownload_run
- encapsulates Bakta database acquisition and annotation execution
• subworkflows/local/utils_nfcore_bacass_pipeline
- likely a pipeline-specific utility composition layer for common data handling
• subworkflows/nf-core/utils_nfcore_pipeline
• subworkflows/nf-core/utils_nextflow_pipeline
• subworkflows/nf-core/utils_nfschema_plugin
- nf-core utility subworkflows used for standard pipeline plumbing, schema integration, and shared helper behavior
This layering makes the architecture diagram clearer:
• top-level workflow decides what branch to run
• subworkflows define the stage boundaries
• modules perform the actual tool invocations
Configuration and environment boundary
The runtime is controlled by:
• nextflow.config
• conf/base.config
• multiple profile configs for test variants and platform behavior:
- conf/test.config
- conf/test_full.config
- conf/test_hybrid.config
- conf/test_long.config
- conf/test_dfast.config
- conf/test_liftoff.config
- and others
The pipeline is designed to run across execution backends via Nextflow with container support:
• Docker
• Singularity
• Conda
This is a significant architectural feature: compute is abstracted away from the workflow logic, and the same pipeline can run on local workstations, HPC, or cloud.
The schema and asset files are also part of the configuration boundary:
• nextflow_schema.json defines parameter validation
• assets/schema_input.json and assets/samplesheet.tsv support input structure and documentation
• assets/multiqc_config*.yml, assets/adaptivecard.json, assets/slackreport.json, and email templates support output/report formatting
• assets/methods_description_template.yml suggests structured methods text generation
External dependencies and database flow
Several tools require external reference data or databases:
• kraken2db
• Kmerfinder references
• Bakta databases
The pipeline contains explicit download/preparation logic for these, so database handling is not left to the user alone. This creates a second data plane alongside sample data:
• sample reads flow through assembly/analysis
• reference/database artifacts flow through download and preparation steps into selected modules
A notable example is the Kmerfinder branch, where reference genome identification can influence downstream grouping and reporting.
Output architecture
The outputs are a mixture of:
• assembled contigs/scaffolds
• annotation files
• QC reports
• contamination summaries
• MultiQC aggregate reports
• software version manifests
• custom tabular summaries
The docs/output.md and docs/usage.md files suggest that output contracts are documented and are a meaningful part of the repo design, not just incidental artifacts.
Testing and CI/CD layer
The repository has a substantial validation system, which is architecturally important because it reflects the workflow’s modular decomposition:
• tests/ contains end-to-end workflow tests for multiple pipeline modes:
- default
- short/long/hybrid
- dfast
- dragonflye
- liftoff
- miniasm
- prokka
• module tests exist alongside many module definitions
• nf-test.config and GitHub Actions workflows run those tests
• .github/workflows/awsfulltest.yml indicates full-sized AWS validation
• linting and release workflows enforce repo consistency
This indicates that the pipeline is treated as a compositional workflow system with strong regression coverage across many mode combinations.
What to emphasize in an architecture diagram
If you draw this repository, the most important boundaries are:
• Input boundary: samplesheet plus runtime parameters and optional database URLs
• Orchestration boundary: main.nf / workflows/bacass.nf
• Branching boundary: short vs long vs hybrid assembly
• Tool-execution boundary: nf-core modules vs local custom modules
• Aggregation boundary: QC summaries, Kmerfinder reference grouping, MultiQC, software versions
• External-data boundary: Kraken2/Kmerfinder/Bakta database downloads
• Validation boundary: nf-test, GitHub Actions, and AWS full-test workflows
In short, this is a Nextflow-based bacterial assembly pipeline whose architecture is centered on mode-based branching and modular tool composition, with custom subworkflows for database handling, long-read QC aggregation, Kmerfinder-driven grouping, and final reporting.
Main entry points: bacass.nf bacass flow · multiqc/main.nf Final reports · fastq_trim_fastp_fastqc/main.nf Trim & QC · qc_nanoplot_toulliqc/main.nf Long-read QC
Components 34
Orchestration
Pipeline entrypoint and top-level workflow wiring
- Main entry · Loads pipeline parameters and starts the workflow.
main.nf - bacass flow · Top-level orchestration for mode selection and stage wiring.
workflows/bacass.nf - Samplesheet · Tabular sample metadata driving read-type branching.
assets/samplesheet.tsv - Pipeline utils · Shared pipeline plumbing and data-handling helpers.
subworkflows/nf-core/utils_nfcore_pipeline/main.nf - Nextflow utils · Common Nextflow helper logic and channel utilities.
subworkflows/nf-core/utils_nextflow_pipeline/main.nf - Schema utils · Schema/plugin integration helpers for parameter validation.
subworkflows/nf-core/utils_nfschema_plugin/main.nf
Processing
Read QC, assembly, polishing, and QC stages
- Trim & QC · Short-read preprocessing stage combining trimming and QC.
subworkflows/nf-core/fastq_trim_fastp_fastqc/main.nf - fastp · Read trimming and filtering for short reads.
modules/nf-core/fastp/main.nf - fastqc · Per-sample short-read quality control.
modules/nf-core/fastqc/main.nf - Long-read QC · Aggregates long-read QC outputs into sample-level reports.
subworkflows/local/qc_nanoplot_toulliqc/main.nf - nanoplot · Read-length and quality summary generation.
modules/nf-core/nanoplot/main.nf - toulligqc · Long-read sequencing QC aggregation and reporting.
modules/nf-core/toulligqc/main.nf - Assembly modes · Selects short-read, long-read, or hybrid assembly paths.
workflows/bacass.nf - unicycler · Short-read and hybrid assembly path.
modules/nf-core/unicycler/main.nf - long assemblers · Long-read assembly options including dragonflye, miniasm, and canu.
modules/nf-core/dragonflye/main.nf - Polishing · Long-read polishing with racon, medaka, and nanopolish.
modules/nf-core/racon/main.nf
Reporting
Aggregation, annotation, and final summaries
- Assembly QC · Post-assembly quality and completeness assessment.
modules/nf-core/quast/main.nf - busco · Assembly completeness evaluation.
modules/nf-core/busco/busco/main.nf - Contamination · Taxonomic contamination and reference identification branch.
modules/nf-core/kraken2/kraken2/main.nf - Kmerfinder flow · Reference identification, grouping, and summary generation.
subworkflows/local/kmerfinder_summary_download/main.nf - kmerfinder · Reference matching and genome identification.
modules/local/kmerfinder/kmerfinder/main.nf - Reference download · Fetches Kmerfinder reference data.
modules/local/kmerfinder/download_reference/main.nf - Annotation · Selects Prokka, Bakta, or DFAST annotation paths.
workflows/bacass.nf - Bakta run · Downloads Bakta databases and runs annotation.
subworkflows/local/bakta_dbdownload_run/main.nf - bakta · Bacterial genome annotation with Bakta.
modules/nf-core/bakta/bakta/main.nf - prokka · Bacterial genome annotation with Prokka.
modules/nf-core/prokka/main.nf - dfast · Pipeline-specific annotation wrapper for DFAST.
modules/local/dfast/main.nf - Final reports · MultiQC, custom CSV summaries, provenance, and publication artifacts.
modules/nf-core/multiqc/main.nf - Custom MultiQC · Pipeline-specific MultiQC formatting and report export.
modules/local/custom/multiqc/main.nf - Versions · Captures software versions for final output.
modules/nf-core/custom/dumpsoftwareversions/templates/dumpsoftwareversions.py
Config & Validation
Schemas, profiles, tests, and CI support
- Schema · Parameter and input validation contract.
nextflow_schema.json - Nextflow config · Global execution and profile configuration.
nextflow.config - Base config · Shared executor and module settings.
conf/base.config - Workflow tests · End-to-end mode coverage across short, long, hybrid, and annotation variants.
tests/default.nf.test
Connections 32
- Main entry → Samplesheet: reads · Consumes sample metadata and input paths.
- Main entry → bacass flow: starts · Delegates execution to the main workflow.
- bacass flow → Pipeline utils: plumbing · Uses shared pipeline helpers.
- bacass flow → Nextflow utils: channels · Uses standard Nextflow utility helpers.
- bacass flow → Schema utils: validate · Connects schema and parameter handling.
- bacass flow → Trim & QC: short reads · Routes paired short reads into preprocessing.
- Trim & QC → fastp: trim · Trims and filters short reads.
- Trim & QC → fastqc: qc · Runs per-sample read QC.
- bacass flow → Long-read QC: long reads · Routes long-read samples to QC aggregation.
- Long-read QC → nanoplot: summarize · Generates long-read QC summaries.
- Long-read QC → toulligqc: aggregate · Combines long-read QC metrics.
- bacass flow → Assembly modes: select · Branches by assembly strategy.
- Assembly modes → unicycler: short/hybrid · Uses Unicycler for short-read or hybrid assembly.
- Assembly modes → long assemblers: long · Uses long-read assemblers such as Dragonflye, Miniasm, or Canu.
- Assembly modes → Polishing: polish · Applies long-read polishing steps where needed.
- Assembly modes → Assembly QC: post-assembly · Evaluates assembly quality.
- Assembly QC → busco: completeness · Adds completeness assessment.
- Assembly QC → Contamination: screen · Checks for contamination and taxonomic purity.
- Contamination → Kmerfinder flow: group · Feeds reference grouping and summary generation.
- Kmerfinder flow → kmerfinder: identify · Runs Kmerfinder reference matching.
- Kmerfinder flow → Reference download: db · Downloads required reference data.
- bacass flow → Annotation: select · Chooses annotation toolchain.
- Annotation → Bakta run: bakta · Downloads Bakta database and annotates.
- Bakta run → bakta: run · Executes Bakta annotation.
- Annotation → prokka: prokka · Annotates assemblies with Prokka.
- Annotation → dfast: dfast · Annotates assemblies with DFAST.
- bacass flow → Final reports: collect · Aggregates final artifacts and reports.
- Final reports → Custom MultiQC: format · Creates custom summary outputs from MultiQC data.
- Final reports → Versions: provenance · Captures software version manifests.
- Final reports → Schema: validated by · Output and parameter conventions are governed by schema.
- Nextflow config → Base config: includes · Base runtime settings feed the main Nextflow config.
- Workflow tests → bacass flow: regresses · Tests exercise major workflow modes and branches.