How gene2dis/bugbuster works
Main Components
• Nextflow Orchestrator and Pipeline Entry: The central engine is the “main.nf” file at the project root, which controls the overall workflow.
• Modules: Under the “modules” directory, there is a clear modular breakdown—each subdirectory (such as “arg_fasta_formatter”, “fastp”, “bowtie2”, “metabat2”, etc.) contains its own Nextflow process script (main.nf). Each module represents a step in the pipeline (e.g., read QC, host contamination removal, taxonomic profiling, genome assembly, binning, etc.).
• Configuration Files: The “nextflow.config” and files in the “config” directory (e.g., “databases.config”, “modules.config”) hold parameters, resource allocation defaults, and paths to external databases.
• Input/Output Artifacts: The pipeline uses a “samplesheet.csv” as its input, and it produces multiple outputs (reports, intermediate files, final results) based on the different stages.
Component Relationships and Interactions
• The Nextflow engine (main.nf) acts as a central controller. It dynamically invokes each module as a process, ensuring that the output from one step becomes the input to the next.
• Each module (packed in its own container) runs specific tasks (e.g., FastP for read QC, Bowtie2 for host contamination removal, Kraken2 and Sourmash for taxonomic profiling, Megahit for assembly, BlobTools for contig annotation) and may interact with external APIs/databases.
• The configuration files supply pipeline parameters, including database paths for various tools (Bowtie2, Kraken2, nt DB, etc.), which are key to ensuring reproducibility and accurate resource usage.
• Read-level analysis, assembly, and binning components interact sequentially in the workflow, with “assembly_mode” configuration controlling how assembly is conducted (coassembly vs individual assembly).
Architectural Patterns and Design Principles
• Modular Design: Every analytical step is compartmentalized into its own module. This makes the pipeline easy to update or extend.
• Containerized Execution: Every process is launched in its individual Docker container. This encapsulation ensures that all software dependencies remain consistent and promotes reproducibility.
• Workflow Orchestration: Nextflow’s design ensures that processes are executed according to dependencies and configured resource constraints across compute infrastructures.
• Configuration-driven Behavior: The numerous configuration files separate code from settings such as database paths or parameter defaults—this is central for flexible deployment across different environments.
Relevant Technologies and Frameworks
• Nextflow DSL2 – the workflow engine.
• Docker – containerization for each process.
• A suite of bioinformatics tools: FastP, Bowtie2, Kraken2, Bracken, Megahit, BlobTools, MetaCerberus, Prodigal, DeepARG, Metabat2, SemiBin, COMEBin, MetaWrap, CheckM2, GTDB-TK, and MMseqs2.
• Various external databases (e.g., nt, gtdbtk data, human_db, and more) as part of the configuration.