# phcviro/viro-metagenomics architecture

> Architecture diagram of the GitHub repository phcviro/viro-metagenomics, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/phcviro/viro-metagenomics
- Repository: https://github.com/phcviro/viro-metagenomics

## Overview

##### Main components
- Workflow Orchestrator
– Nextflow (main.nf + DSL2 modules + subworkflows)
- Module Library
– QC modules (fastqc, fastp)
– Alignment modules (minimap2, samtools)
– Variant Calling modules (lofreq, medaka)
– Annotation modules (snpEff, SnpSift)
– Reporting modules (MultiQC, custom Python/R scripts in bin/)
- Reference Data
– nextflow-references folder (genomes, mutation tables, SnpEff data, BLAST DB)
- Input Interfaces
– Sample sheet CSV (defines sample IDs, input FASTQ paths, genome type)
– Raw FASTQ (gzipped)
- Compute Environment
– Local / HPC / Cloud executor (Docker/Conda optional)
- Outputs
– Per‐sample output tree: combinedFastq, QC, align, variants, annotations, coverage, report

##### Relationships & data flow
1) Inputs → Nextflow launcher
2) Nextflow parses sample sheet → routes samples into subworkflows (HBV or CMV)
3) Within each subworkflow:
a) Channel raw FASTQs → QC modules (fastqc → fastp) → cleaned FASTQs
b) Cleaned FASTQs + reference genomes → alignment module → BAM files
c) BAM → variant modules (lofreq SNVs → medaka indels) → raw VCFs
d) Raw VCFs + reference mutation tables → annotation modules → annotated VCFs/TSVs
e) All intermediate results → MultiQC module → final report
4) Results collected into structured output directories

##### Architectural patterns & principles
- DSL2 modular design – each step encapsulated in its own .nf file under modules/
- Subworkflow abstraction – cmv.nf and hbv.nf in subworkflows/ for virus‐specific branching
- Data‐flow (DAG) execution – channels connect processes, enabling parallelism
- Reusability – common modules (e.g., fastqc) used by both HBV and CMV branches
- Configuration separation – nextflow.config controls parameters, executors, containers

##### Notable technologies & tools
– Nextflow 24.04.4 (DSL2)
– fastqc, fastp, minimap2, samtools, lofreq, medaka, snpEff/SnpSift, mosdepth, bcftools, vcflib
– MultiQC for reporting
– Python/R scripts in bin/ for custom tables and plots
– Reference genome files, SnpEff databases, BLAST DB

– Clear, human‐readable labels on every component (e.g., “Quality Control (fastqc, fastp)”)
- Orchestrator vs. module vs. data store vs. compute
- Common vs. virus‐specific steps

##### Final advice
- Reference data as a separate repository that all alignment/annotation steps pull from.
- DAG nature—processes run in parallel where possible.
- Don’t overcomplicate—focus on separating the pipeline into logical processing stages and highlighting Nextflow’s role as orchestrator.
- This level of detail ensures maintainability, clarity for new engineers, and easy extension for other viral pipelines.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Input Interfaces
    subgraph "Input Interfaces" 
        A1["Sample Sheet CSV"]:::data
        A2["Raw FASTQ (gzipped)"]:::data
    end

    %% Reference Data Store
    subgraph "Reference Data" 
        R1["Genomes & Mutation Tables"]:::data
        R2["BLAST DB"]:::data
        R3["SnpEff Databases"]:::data
    end

    %% Compute Environment
    subgraph "Compute Environment" 
        C1["Local / HPC / Cloud Executor"]:::compute
        C2["Docker / Conda"]:::compute
    end

    %% Workflow Orchestrator
    subgraph "Workflow Orchestrator" 
        NF["Nextflow Engine"]:::orchestrator
        M1["main.nf"]:::orchestrator
        M2["nextflow.config"]:::orchestrator
    end

    %% Subworkflows (Virus-specific)
    subgraph "Subworkflows (DSL2 Branching)" 
        SW1["hbv.nf"]:::virus
        SW2["cmv.nf"]:::virus
    end

    %% Module Library
    subgraph "DSL2 Module Library"
        subgraph "Utility / Pre-processing"
            U1["combine_fastq.nf"]:::module
            U2["seqtk.nf"]:::module
            U3["dumpsoftwareversions.nf"]:::module
            U4["vcf_concat.nf"]:::module
            U5["vcf_uniq.nf"]:::module
            U6["plot_coverage_regions.nf"]:::module
            U7["bam_generation.nf"]:::module
        end

        subgraph "Quality Control"
            Q1["fastqc.nf"]:::module
            Q2["fastp.nf"]:::module
        end

        subgraph "Alignment"
            AL1["alignment.nf"]:::module
            AL2["bam_generation.nf"]:::module
            AL3["blast.nf"]:::module
        end

        subgraph "Coverage"
            CV1["coverage.nf"]:::module
            CV2["coverage_hbv.nf"]:::module
        end

        subgraph "Variant Calling"
            V1["call_snvs.nf"]:::module
            V2["call_snvs_hbv.nf"]:::virus
            V3["call_deletions.nf"]:::module
            V4["call_deletions_hbv.nf"]:::virus
        end

        subgraph "Annotation"
            AN1["call_annotation.nf"]:::module
            AN2["annotation_extractfields.nf"]:::module
        end

        subgraph "Reporting"
            RPT1["multiqc.nf"]:::module
            RPT2["variants_long_table.nf"]:::module
        end
    end

    %% Custom Reporting & Utilities
    subgraph "Custom Scripts (bin/)"
        B1["dumpsoftwareversions.py"]:::script
        B2["make_variants_long_table.py"]:::script
        B3["multibase_codon_detection.py"]:::script
        B4["plot_mosdepth_regions.r"]:::script
        B5["run_singular.py"]:::script
    end

    %% Outputs
    subgraph "Pipeline Outputs"
        O1["Per-sample Output Tree"]:::output
    end

    %% Data Flow
    A1 -->|Defines Samples| NF
    A2 -->|Raw Reads| NF
    NF -->|parses & routes| SW1
    NF -->|parses & routes| SW2

    SW1 --> U1
    SW2 --> U1

    U1 --> Q1
    U1 --> Q2

    Q1 --> Q2

    Q2 --> AL1
    R1 --> AL1
    Q2 --> AL2
    Q2 --> AL3

    AL1 --> CV1
    AL1 --> CV2
    R1 --> CV1
    R1 --> CV2

    CV1 --> V1
    CV2 --> V2
    CV1 --> V3
    CV2 --> V4

    V1 --> AN1
    V2 --> AN1
    V3 --> AN1
    V4 --> AN1
    R1 --> AN1
    R3 --> AN1

    AN1 --> AN2
    AN2 --> RPT1
    AN2 --> RPT2

    RPT1 --> O1
    RPT2 --> O1

    C1 --> NF
    C2 --> NF

    %% Click Events
    click M1 "https://github.com/phcviro/viro-metagenomics/blob/main/main.nf"
    click M2 "https://github.com/phcviro/viro-metagenomics/blob/main/nextflow.config"
    click Q1 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/fastqc.nf"
    click Q2 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/fastp.nf"
    click AL1 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/alignment.nf"
    click AL2 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/bam_generation.nf"
    click AL3 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/blast.nf"
    click CV1 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/coverage.nf"
    click CV2 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/coverage_hbv.nf"
    click V1 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/call_snvs.nf"
    click V2 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/call_snvs_hbv.nf"
    click V3 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/call_deletions.nf"
    click V4 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/call_deletions_hbv.nf"
    click AN1 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/call_annotation.nf"
    click AN2 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/annotation_extractfields.nf"
    click RPT1 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/multiqc.nf"
    click RPT2 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/variants_long_table.nf"
    click U1 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/combine_fastq.nf"
    click U2 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/seqtk.nf"
    click U3 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/dumpsoftwareversions.nf"
    click U4 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/vcf_concat.nf"
    click U5 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/vcf_uniq.nf"
    click U6 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/plot_coverage_regions.nf"
    click U7 "https://github.com/phcviro/viro-metagenomics/blob/main/modules/bam_generation.nf"
    click SW1 "https://github.com/phcviro/viro-metagenomics/blob/main/subworkflows/hbv.nf"
    click SW2 "https://github.com/phcviro/viro-metagenomics/blob/main/subworkflows/cmv.nf"
    click B1 "https://github.com/phcviro/viro-metagenomics/blob/main/bin/dumpsoftwareversions.py"
    click B2 "https://github.com/phcviro/viro-metagenomics/blob/main/bin/make_variants_long_table.py"
    click B3 "https://github.com/phcviro/viro-metagenomics/blob/main/bin/multibase_codon_detection.py"
    click B4 "https://github.com/phcviro/viro-metagenomics/blob/main/bin/plot_mosdepth_regions.r"
    click B5 "https://github.com/phcviro/viro-metagenomics/blob/main/bin/run_singular.py"

    %% Styles
    classDef data fill:#E8F1F2,stroke:#0B6E99,color:#0B6E99
    classDef orchestrator fill:#FEF3C7,stroke:#C47A00,color:#C47A00
    classDef compute fill:#F2F4F5,stroke:#5A6A72,color:#5A6A72
    classDef module fill:#FCE8E6,stroke:#C32F27,color:#C32F27
    classDef virus fill:#E4F7D2,stroke:#3C763D,color:#3C763D
    classDef script fill:#FFF5E1,stroke:#A56F00,color:#A56F00
    classDef output fill:#EAF6F6,stroke:#1B6B62,color:#1B6B62
```
