How nf-core/rnavar works
Project type and purpose
• This is a DSL2 Nextflow pipeline (nf-core/rnavar) for RNA variant calling following GATK4 best practices.
• It orchestrates a set of bioinformatics tools (STAR, FastQC, GATK4, SAMtools, Picard, snpEff, VEP, MultiQC, etc.) into a reproducible workflow.
High-level file structure
• main.nf: Entry point – defines the overall workflow and channel/workflow definitions.
• modules/: Library of process‐level modules (each tool or command wrapped as a Nextflow module with its main.nf, environment.yml, meta.yml, tests).
• subworkflows/: Reusable DSL2 workflows calling sets of modules for higher‐level tasks (e.g., “prepare_genome,” “cram_qc_mosdepth_samtools”).
• conf/: Configuration fragments (base.config, modules.config, igenomes.config, test configs) loaded by nextflow.config.
• nextflow.config: Main configuration injecting profiles (docker/singularity/conda), resource specs, module paths.
• assets/ and bin/: Templates and helper scripts for reports, emails, Slack notifications.
• tests/: nf-test‐driven unit tests covering both modules and the main pipeline.
• .github/, .gitlab-ci, .github/actions, workflows/: CI‐as‐code definitions triggering lint, unit tests, full‐size runs, release announcements.
Key insights from README
• Lists 16 sequential pipeline steps (merging, QC, UMI extraction, alignment, BQSR, variant calling, annotation, QC reports).
• Emphasizes support for Conda, Docker, Singularity, Seqera Platform.
• States DSL2 modular architecture, nf-core best practices, and testing standards.
a. Main components
• Input Layer: samplesheet.csv, user parameters, nextflow profiles (docker/conda).
• Orchestration Layer: Nextflow Engine (DSL2) + main.nf coordinating channels, workflows, modules.
• Module Library: individual tool modules under modules/ (each with a container/Conda environment).
• Subworkflow Library: high‐level workflows under subworkflows/ calling multiple modules.
• Configuration Layer: nextflow.config, conf/*.config – selects profiles, resource settings, module overrides.
• Execution Environments: Docker/Singularity containers or Conda environments per module.
• Output Layer: BAMs, VCFs, QC reports (HTML/JSON), MultiQC, logs.
• CI/CD & Testing: GitHub Actions, nf-test unit tests, nf-core linting, AWS full‐size validation.
• Notification & Reporting: Slack, email, adaptive cards via templates in assets//bin/.
b. Relationships & interactions
• Inputs → Nextflow Engine: user provides samplesheet + params → engine kicks off main.nf.
• Nextflow Engine → Config: loads nextflow.config → merges in conf/ fragments → sets up executor, containers, resources.
• Nextflow Engine → Module Library: invokes each process in sequence or scatter/gather via DSL2 workflows.
• Module processes → Execution Environments: each process runs in specified container/Conda env.
• Outputs → Results directory → MultiQC collects metrics → final reports.
• CI/CD triggers: pushes or PRs → GitHub Actions → runs lint, unit tests, full tests.
c. Architectural patterns & principles
• Modularity: Each tool abstracted as a DSL2 module with its own environment, tests, metadata.
• Reusability: Subworkflows stitch modules for common tasks (e.g., alignment preparation).
• Configurability: Profiles and config fragments allow easy switching of executors (local, AWS Batch, Kubernetes).
• Test-driven development: nf-test and YAML‐based test definitions for both modules and workflows.
• Immutable environments: Docker/Singularity or pinned Conda env ensures reproducibility.
d. Key technologies & frameworks
• Nextflow DSL2 (pipeline orchestration)
• nf-core framework conventions
• Docker, Singularity, Conda (environment management)
• GitHub Actions & nf-test for CI
• GATK4, STAR, Samtools, Picard, etc. (bioinformatics tools)