# openvar/nextflow_rd_genomic_vqr architecture

> Architecture diagram of the GitHub repository openvar/nextflow_rd_genomic_vqr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openvar/nextflow_rd_genomic_vqr
- Repository: https://github.com/openvar/nextflow_rd_genomic_vqr

## Overview

##### Project type and purpose
- This is a domain-specific workflow/pipeline built on the Nextflow engine for Rare Disease & Germline variant calling.
- It orchestrates a series of bioinformatics “processes” (QC, alignment, recalibration, variant calling, filtering, analysis) to test down-sampling effects.

##### File structure
- Top-level:
- main.nf: Nextflow entrypoint
- nextflow.config & nextflow_schema.json: parameter definitions & defaults
- conf/base.config: environment/profiles for Docker, Singularity, DNAnexus
- modules/*.nf: one file per pipeline step
- dockerfiles/*: custom container definitions for FASTQC, GATK, etc.
- data/: genome FASTA, indexes, reference VCFs
- samplesheet.tsv: sample metadata
- Key patterns:
- Modular pipeline (each tool in its own .nf)
- Containerized tool execution (Docker/Singularity images)
- Declarative config driving parameterization & profiles

##### README for additional insights
- Confirms toolchain (Nextflow + Docker/Singularity + optional DNAnexus)
- Lists required inputs (FASTQ, reference genome, VQSR resources)
- Describes run profiles (–profile docker | singularity | dx)

##### Main components
a. Workflow Orchestrator
- Nextflow Engine (main.nf + modules + config + schema)
b. Modules (Processes)
1. FASTQC
2. Adapter Trimming / mapDamage
3. BWA-aln & BWA-mem alignment
4. sortBam / markDuplicates / indexBam
5. BQSR (Base Recalibration)
6. HaplotypeCaller & processGVCFs
7. VariantRecalibrator / filterVCF
8. identityAnalysis
9. indexGenome (one‐time prep)
c. Execution Environments
- Local or HPC cluster
- Docker daemon or Singularity runtime
- (Optional) DNAnexus applet via dx-toolkit
d. Data Stores & Inputs
- FASTQ inputs (samplesheet)
- Reference genome & indexes (data/genome)
- VQSR resource VCFs & indices
e. Outputs
- QC reports, recalibrated BAMs, VCFs, summary metrics

##### Relationships & interactions
- Nextflow reads config + samplesheet → instantiates channels of input files
- Each module receives one or more channels (FASTQ → QC → trimmed FASTQ → aligner → BAM → post-processing → VCF)
- Nextflow launches each process inside its respective Docker/Singularity image
- Artifacts flow via the file system (or remote storage) between processes
- Final reporting/analysis (identityAnalysis) aggregates metrics

##### Architectural patterns & principles
- Modular decomposition: each .nf file is a self-contained process
- Containerization: isolating dependencies per tool
- Declarative configuration: profiles for environment portability
- Dataflow paradigm: channels → processes → outputs
- Reproducibility & portability (local, cloud, DNAnexus)

- Clear labels for components (e.g., “Nextflow Engine,” “FASTQC Container,” “Reference Genome Store”)
- FASTQ → FASTQC → trimmed FASTQ → BWA → BAM → … → VCF
- Rectangles for modules/processes

##### Final advice
- How Nextflow channels connect modules.
- Container boundaries to highlight reproducibility.
- Config profiles to indicate environment choices.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Inputs & Config
    subgraph "Inputs & Config"
        direction TB
        Samplesheet["samplesheet.tsv"]:::config
        MainNF["main.nf"]:::config
        NextflowConfig["nextflow.config"]:::config
        Schema["nextflow_schema.json"]:::config
        BaseConfig["conf/base.config"]:::config
        SamplesheetWGS["samplesheet_wgs.tsv"]:::config
    end

    %% Orchestration Layer
    subgraph "Orchestration"
        direction TB
        NFengine["Nextflow Engine"]:::engine
    end

    %% Processing Modules
    subgraph "Processing Modules"
        direction TB
        FASTQC["FASTQC"]:::process
        Adapter["Adapter Trimming / mapDamage"]:::process
        BWA_ALN["BWA-aln Alignment"]:::process
        BWA_MEM["BWA-mem Alignment"]:::process
        SortBam["sortBam"]:::process
        MarkDup["markDuplicates"]:::process
        IndexBam["indexBam"]:::process
        BQSR["BQSR"]:::process
        Haplotype["HaplotypeCaller"]:::process
        ProcessGVCF["processGVCFs"]:::process
        VarRecal["VariantRecalibrator"]:::process
        FilterVCF["filterVCF"]:::process
        Identity["identityAnalysis"]:::process
        IndexGenome["indexGenome (prep)"]:::process
    end

    %% Execution Environments
    subgraph "Execution Environments"
        direction TB
        Docker["Docker Runtime"]:::exec
        Singularity["Singularity Runtime"]:::exec
        DNAnexus["DNAnexus"]:::exec
    end

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        Genome["Reference Genome & Indexes"]:::data
        VQSR["VQSR Resource VCFs"]:::data
    end

    %% Outputs
    subgraph "Outputs & Reports"
        direction TB
        Results["results/ & work/"]:::output
        QCReports["QC Reports"]:::output
        FinalVCF["Final VCFs"]:::output
        Metrics["Summary Metrics"]:::output
    end

    %% Connections
    Samplesheet --> NFengine
    SamplesheetWGS --> NFengine
    MainNF --> NFengine
    NextflowConfig --> NFengine
    Schema --> NFengine
    BaseConfig --> NFengine

    NFengine --> FASTQC
    FASTQC --> Adapter
    Adapter --> BWA_ALN
    Adapter --> BWA_MEM
    BWA_ALN --> SortBam
    BWA_MEM --> SortBam
    SortBam --> MarkDup
    MarkDup --> IndexBam
    IndexBam --> BQSR
    BQSR --> Haplotype
    Haplotype --> ProcessGVCF
    ProcessGVCF --> VarRecal
    VarRecal --> FilterVCF
    FilterVCF --> Identity
    NFengine --> IndexGenome

    Genome --> BWA_ALN
    Genome --> BWA_MEM
    Genome --> IndexGenome
    VQSR --> VarRecal

    FASTQC --> Docker
    Adapter --> Docker
    BWA_ALN --> Docker
    BWA_MEM --> Docker
    SortBam --> Docker
    MarkDup --> Docker
    IndexBam --> Docker
    BQSR --> Docker
    Haplotype --> Docker
    ProcessGVCF --> Docker
    VarRecal --> Docker
    FilterVCF --> Docker
    Identity --> Docker
    IndexGenome --> Docker

    Docker --> Results
    Singularity --> Results
    DNAnexus --> Results
    Identity --> Metrics
    Identity --> QCReports
    FilterVCF --> FinalVCF

    %% Click Events
    click Samplesheet "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/samplesheet.tsv"
    click SamplesheetWGS "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/samplesheet_wgs.tsv"
    click MainNF "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/main.nf"
    click NextflowConfig "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/nextflow.config"
    click Schema "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/nextflow_schema.json"
    click BaseConfig "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/conf/base.config"
    click FASTQC "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/FASTQC.nf"
    click FASTQC "https://github.com/openvar/nextflow_rd_genomic_vqr/tree/main/dockerfiles/FASTqc/Dockerfile"
    click Adapter "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/mapDamage.nf"
    click BWA_ALN "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/alignReadsBwaAln.nf"
    click BWA_MEM "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/alignReadsBwaMem.nf"
    click SortBam "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/sortBam.nf"
    click MarkDup "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/markDuplicates.nf"
    click IndexBam "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/indexBam.nf"
    click BQSR "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/BQSR.nf"
    click Haplotype "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/haplotypeCaller.nf"
    click ProcessGVCF "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/processGVCFs.nf"
    click VarRecal "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/variantRecalibrator.nf"
    click FilterVCF "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/filterVCF.nf"
    click Identity "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/identityAnalysis.nf"
    click Identity "https://github.com/openvar/nextflow_rd_genomic_vqr/tree/main/dockerfiles/identityAnalysis/Dockerfile"
    click IndexGenome "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/modules/indexGenome.nf"
    click IndexGenome "https://github.com/openvar/nextflow_rd_genomic_vqr/tree/main/dockerfiles/indexGenome/Dockerfile"
    click Genome "https://github.com/openvar/nextflow_rd_genomic_vqr/tree/main/data/genome/"
    click Docker "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/build_images.sh"
    click Singularity "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/environment.yml"
    click DNAnexus "https://github.com/openvar/nextflow_rd_genomic_vqr/blob/main/docs/running_your_nextflow_applet.md"

    %% Styles
    classDef config fill:#f9f7b0,stroke:#333,stroke-width:1px
    classDef engine fill:#dcd0ff,stroke:#333,stroke-width:1px
    classDef process fill:#b3e5fc,stroke:#333,stroke-width:1px
    classDef exec fill:#ffcc80,stroke:#333,stroke-width:1px
    classDef data fill:#c8e6c9,stroke:#333,stroke-width:1px
    classDef output fill:#f8bbd0,stroke:#333,stroke-width:1px
```
