# nf-core/rnavar architecture

> Architecture diagram of the GitHub repository nf-core/rnavar, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nf-core/rnavar
- Repository: https://github.com/nf-core/rnavar

## Overview

##### Project type and purpose
- This is a DSL2 Nextflow pipeline (nf-core/rnavar) for RNA variant calling following GATK4 best practices.
- It orchestrates a set of bioinformatics tools (STAR, FastQC, GATK4, SAMtools, Picard, snpEff, VEP, MultiQC, etc.) into a reproducible workflow.

##### High-level file structure
- **main.nf**: Entry point – defines the overall workflow and channel/workflow definitions.
- **modules/**: Library of process‐level modules (each tool or command wrapped as a Nextflow module with its `main.nf`, `environment.yml`, `meta.yml`, tests).
- **subworkflows/**: Reusable DSL2 workflows calling sets of modules for higher‐level tasks (e.g., “prepare_genome,” “cram_qc_mosdepth_samtools”).
- **conf/**: Configuration fragments (`base.config`, `modules.config`, `igenomes.config`, test configs) loaded by `nextflow.config`.
- **nextflow.config**: Main configuration injecting profiles (docker/singularity/conda), resource specs, module paths.
- **assets/** and **bin/**: Templates and helper scripts for reports, emails, Slack notifications.
- **tests/**: nf-test‐driven unit tests covering both modules and the main pipeline.
- **.github/**, **.gitlab-ci**, **.github/actions**, **workflows/**: CI‐as‐code definitions triggering lint, unit tests, full‐size runs, release announcements.

##### Key insights from README
- Lists 16 sequential pipeline steps (merging, QC, UMI extraction, alignment, BQSR, variant calling, annotation, QC reports).
- Emphasizes support for Conda, Docker, Singularity, Seqera Platform.
- States DSL2 modular architecture, nf-core best practices, and testing standards.

a. Main components
- Input Layer: samplesheet.csv, user parameters, nextflow profiles (docker/conda).
- Orchestration Layer: Nextflow Engine (DSL2) + `main.nf` coordinating channels, workflows, modules.
- Module Library: individual tool modules under `modules/` (each with a container/Conda environment).
- Subworkflow Library: high‐level workflows under `subworkflows/` calling multiple modules.
- Configuration Layer: `nextflow.config`, `conf/*.config` – selects profiles, resource settings, module overrides.
- Execution Environments: Docker/Singularity containers or Conda environments per module.
- Output Layer: BAMs, VCFs, QC reports (HTML/JSON), MultiQC, logs.
- CI/CD & Testing: GitHub Actions, nf-test unit tests, nf-core linting, AWS full‐size validation.
- Notification & Reporting: Slack, email, adaptive cards via templates in `assets/`/`bin/`.

b. Relationships & interactions
- Inputs → Nextflow Engine: user provides samplesheet + params → engine kicks off `main.nf`.
- Nextflow Engine → Config: loads `nextflow.config` → merges in `conf/` fragments → sets up executor, containers, resources.
- Nextflow Engine → Module Library: invokes each process in sequence or scatter/gather via DSL2 workflows.
- Module processes → Execution Environments: each process runs in specified container/Conda env.
- Outputs → Results directory → MultiQC collects metrics → final reports.
- CI/CD triggers: pushes or PRs → GitHub Actions → runs lint, unit tests, full tests.

c. Architectural patterns & principles
- **Modularity**: Each tool abstracted as a DSL2 module with its own environment, tests, metadata.
- **Reusability**: Subworkflows stitch modules for common tasks (e.g., alignment preparation).
- **Configurability**: Profiles and config fragments allow easy switching of executors (local, AWS Batch, Kubernetes).
- **Test-driven development**: nf-test and YAML‐based test definitions for both modules and workflows.
- **Immutable environments**: Docker/Singularity or pinned Conda env ensures reproducibility.

d. Key technologies & frameworks
- Nextflow DSL2 (pipeline orchestration)
- nf-core framework conventions
- Docker, Singularity, Conda (environment management)
- GitHub Actions & nf-test for CI
- GATK4, STAR, Samtools, Picard, etc. (bioinformatics tools)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input Layer
    subgraph "Input Layer"
        direction TB
        Samplesheet["samplesheet.csv"]:::input
        Schema["input schema (JSON)"]:::input
    end

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        direction TB
        MainNf["main.nf"]:::orchestration
        RnavarWorkflow["workflows/rnavar.nf"]:::orchestration
        NextflowConfig["nextflow.config"]:::orchestration
        NextflowSchema["nextflow_schema.json"]:::orchestration
    end

    %% Module Library
    subgraph "Module Library"
        direction TB
        LocalModules["modules/local/"]:::module
        NfCoreModules["modules/nf-core/"]:::module
    end

    %% Subworkflow Library
    Subworkflows["subworkflows/"]:::module

    %% Configuration Layer
    subgraph "Configuration Layer"
        direction TB
        BaseConfig["conf/base.config"]:::orchestration
        ModulesConfig["conf/modules.config"]:::orchestration
        IgenomesConfig["conf/igenomes.config"]:::orchestration
        TestConfig["conf/test.config\nconf/test_full.config"]:::orchestration
    end

    %% Execution Environments
    subgraph "Execution Environments"
        direction TB
        CondaEnvs["modules/*/*/environment.yml"]:::env
        ContainerProfiles["conf/modules/*.config"]:::env
    end

    %% Core Workflow
    subgraph "Core Workflow (Stages 1–16)"
        direction TB
        Preprocessing["1. FastQC\n2. UMI extraction"]:::module
        Alignment["3. STAR alignment"]:::module
        Recalibration["4. BQSR (GATK4)"]:::module
        VariantCalling["5. HaplotypeCaller\n6. MergeVCFs"]:::module
        Annotation["7. snpEff/VEP annotation"]:::module
        QCReports["8. MultiQC & JSON reports"]:::module
    end

    %% Output Layer
    subgraph "Output Layer"
        direction TB
        BAMs["BAM files"]:::output
        VCFs["VCF files"]:::output
        Reports["QC reports (HTML/JSON)"]:::output
        MultiQCConfig["assets/multiqc_config.yml"]:::output
        MethodsTemplate["assets/methods_description_template.yml"]:::output
    end

    %% CI/CD & Testing
    subgraph "CI/CD & Testing"
        direction TB
        GHWorkflows[".github/workflows/"]:::cicd
        GHActions[".github/actions/"]:::cicd
        NFTestConfig["nf-test.config"]:::cicd
        Tests["tests/ & modules/*/*/tests/\n& subworkflows/*/*/tests/"]:::cicd
    end

    %% Notification & Reporting
    subgraph "Notification & Reporting"
        direction TB
        EmailSlackTemplates["assets/email_template.html\nassets/email_template.txt\nassets/slackreport.json"]:::cicd
        HelperScripts["bin/remove_unknown_regions.py"]:::cicd
    end

    %% Connections
    Samplesheet -->|provides| MainNf
    Schema -->|validates| MainNf
    MainNf -->|defines workflows| RnavarWorkflow
    MainNf -->|loads| NextflowConfig
    NextflowConfig -->|includes| BaseConfig
    NextflowConfig -->|includes| ModulesConfig
    NextflowConfig -->|includes| IgenomesConfig
    NextflowConfig -->|includes| TestConfig
    MainNf -->|invokes| LocalModules
    MainNf -->|invokes| NfCoreModules
    MainNf -->|uses| Subworkflows
    Subworkflows -->|calls| LocalModules
    Subworkflows -->|calls| NfCoreModules
    LocalModules -->|run in| CondaEnvs
    NfCoreModules -->|run in| ContainerProfiles
    MainNf -->|executes stages| Preprocessing
    Preprocessing -->|FASTQ→FASTQC| Alignment
    Alignment -->|BAM| Recalibration
    Recalibration -->|Recalibrated BAM| VariantCalling
    VariantCalling -->|VCF| Annotation
    Annotation -->|Annotated VCF| QCReports
    QCReports -->|reports| BAMs
    QCReports -->|reports| VCFs
    QCReports -->|generates| Reports
    Reports -->|uses| MultiQCConfig
    Reports -->|uses| MethodsTemplate

    MainNf -->|tested by| Tests
    GHWorkflows -->|runs lint & tests| Tests
    GHWorkflows -->|triggers| NFTestConfig
    GHWorkflows -->|on push/PR| MainNf

    EmailSlackTemplates -->|templates| HelperScripts
    MainNf -->|sends notifications via| EmailSlackTemplates

    %% Click Events
    click Samplesheet "https://github.com/nf-core/rnavar/blob/master/assets/samplesheet.csv"
    click Schema "https://github.com/nf-core/rnavar/blob/master/assets/schema_input.json"
    click MainNf "https://github.com/nf-core/rnavar/blob/master/main.nf"
    click RnavarWorkflow "https://github.com/nf-core/rnavar/blob/master/workflows/rnavar.nf"
    click NextflowConfig "https://github.com/nf-core/rnavar/blob/master/nextflow.config"
    click NextflowSchema "https://github.com/nf-core/rnavar/blob/master/nextflow_schema.json"
    click LocalModules "https://github.com/nf-core/rnavar/tree/master/modules/local/"
    click NfCoreModules "https://github.com/nf-core/rnavar/tree/master/modules/nf-core/"
    click Subworkflows "https://github.com/nf-core/rnavar/tree/master/subworkflows/"
    click BaseConfig "https://github.com/nf-core/rnavar/blob/master/conf/base.config"
    click ModulesConfig "https://github.com/nf-core/rnavar/blob/master/conf/modules.config"
    click IgenomesConfig "https://github.com/nf-core/rnavar/blob/master/conf/igenomes.config"
    click TestConfig "https://github.com/nf-core/rnavar/blob/master/conf/test.config"
    click CondaEnvs "https://github.com/nf-core/rnavar/blob/master/modules/*/*/environment.yml"
    click ContainerProfiles "https://github.com/nf-core/rnavar/blob/master/conf/modules/*.config"
    click MultiQCConfig "https://github.com/nf-core/rnavar/blob/master/assets/multiqc_config.yml"
    click MethodsTemplate "https://github.com/nf-core/rnavar/blob/master/assets/methods_description_template.yml"
    click GHWorkflows "https://github.com/nf-core/rnavar/tree/master/.github/workflows/"
    click GHActions "https://github.com/nf-core/rnavar/tree/master/.github/actions/"
    click NFTestConfig "https://github.com/nf-core/rnavar/blob/master/nf-test.config"
    click Tests "https://github.com/nf-core/rnavar/tree/master/tests/"
    click EmailSlackTemplates "https://github.com/nf-core/rnavar/blob/master/assets/email_template.html"
    click HelperScripts "https://github.com/nf-core/rnavar/blob/master/bin/remove_unknown_regions.py"

    %% Styles
    classDef input fill:#F0F0F0,stroke:#333,stroke-width:1px
    classDef orchestration fill:#D0E8FF,stroke:#333,stroke-width:1px
    classDef module fill:#D1FFD1,stroke:#333,stroke-width:1px
    classDef env fill:#FFE4B3,stroke:#333,stroke-width:1px
    classDef output fill:#FFF2CC,stroke:#333,stroke-width:1px
    classDef cicd fill:#E3D1FF,stroke:#333,stroke-width:1px
```
