# yedomon/pangenome architecture

> Architecture diagram of the GitHub repository yedomon/pangenome, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yedomon/pangenome
- Repository: https://github.com/yedomon/pangenome

## Overview

##### Project type and purpose
- This is a monorepo of multiple bioinformatics pipelines for pangenome and comparative-genome analyses across different species (rice, alfalfa, broomcorn millet, grapevine, wild rice, Elymus, Arabidopsis, etc.).
- Each subdirectory implements a full analysis workflow (assembly → annotation → variation calling → pangenome graph construction → evolutionary and population analyses → trait/GWAS/eQTL analyses).

##### File structure
- Top-level directories are named after the species or published projects.
- Within each species folder you typically see numbered stages (e.g., 1.Assembly, 2.Annotation, 3.Pangenome_analysis, 4.SV_analysis, etc.).
- Scripts come in several flavors: shell/Bash (.sh), Snakemake (.smk), Python (.py), R (.R), Perl (.pl).
- Configuration files (.yaml, .ctl), SLURM/PBS submission wrappers (.slurm, PBS_*.sh), and build/work scripts (work.sh) coordinate cluster execution.

##### README for additional insights
- The root README is essentially a curated index of publications and “code availability” links, pointing to each sub-pipeline.
- There is no single  central code module; rather, the structure is intentionally modular to mirror each paper’s methods section.

a. Main components
- Data Ingestion Layer: raw reads (FASTQ), reference genomes, annotation databases (GO, KEGG, Pfam, etc.)
- Workflow Orchestrator: Snakemake/PBS/SLURM scheduler
- Pipeline Modules (per species/project):
– Genome Assembly (HiCanu, Hifiasm, Flye, Juicer/3D-DNA)
– Repeat/TE Annotation (RepeatModeler, RepeatMasker, LTRharvest)
– Gene Prediction & Functional Annotation (Braker3/AUGUSTUS, Maker, DIAMOND, InterProScan, DeepGO)
– Variation Calling (MUMmer/SVMU for SV, GATK/BCFtools for SNP/indel, VG toolkit for graph genotyping)
– Pangenome Graph Construction (minigraph, seqwish, vg)
– Downstream Analyses (Orthofinder, paraAT, Ka/Ks, population structure, PCA, GWAS, selective sweep, eQTL)
- Support Scripts & Utilities: Python modules, R plotting scripts, summary-builders
- Outputs: QC reports, summary tables, VCFs, graph files, figures for publication

b. Relationships & data flow
1. Raw data → Assembly module → Assembled contigs/scaffolds
2. Assemblies + reference genomes → Repeat & gene annotation
3. Annotated genomes + assemblies → alignment & variation calling
4. Variation calls & assemblies → pangenome graph construction
5. Graph + genotype matrices → population genetics (π, FST), GWAS, eQTL
6. Each module reads configs and writes standardized result folders → next module

c. Architectural patterns & principles
- Modular “stage” directories mirror a pipeline pattern (Stage 1, 2, … 8).
- Configuration-driven: each Snakemake or shell script reads a central config/ctl/yaml.
- Reproducibility via workflow manager (Snakemake) + containerized/cluster-wrapped scripts.
- Separation of compute (HPC scheduler) from code.

d. Key technologies & tools
- Workflow: Snakemake, PBS/SLURM scripts
- Languages: Bash, Python, R, Perl
- Assembly: Hifiasm, HiCanu, Flye, Juicer/3D-DNA
- Annotation: RepeatModeler, Braker3, Maker, DIAMOND, InterProScan, DeepGO
- Variation: MUMmer4 (NUCmer, SVMU), vg toolkit, GATK, BCFtools
- Pangenome graph: minigraph, seqwish, vg
- Downstream: Orthofinder, paraAT, EMMAX/EMMAX2, Matrix eQTL, R packages (ClusterProfiler)

##### Detail is key
- Parallelism across species—e.g., ten lanes side by side—then collapse identical modules into a “shared template” callout.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    RawData["Raw Reads (FASTQ)"]:::data
    RefGenomes["Reference Genomes"]:::data
    RefDBs["Annotation DBs (GO, KEGG, Pfam)"]:::data

    %% Orchestration Layer
    subgraph "Workflow Orchestrator"
        Snakemake["Snakemake Engine"]:::orchestrator
        Scheduler["HPC Scheduler (SLURM/PBS)"]:::orchestrator
    end

    RawData -->|FASTQ| Snakemake
    RefGenomes -->|FASTA/GFF| Snakemake
    RefDBs -->|DB Files| Snakemake
    Snakemake --- Scheduler

    %% Broomcorn Millet Pipeline
    subgraph "Broomcorn Millet Pipeline"
        direction TB
        BM1["1. Assembly\n(HiCanu/Hifiasm/Flye)"]:::assembly
        BM2["2. Annotation\n(RepeatModeler, Braker3)"]:::annotation
        BM3["3. Pangenome_analysis\n(Orthofinder, Ka/Ks)"]:::pangenome
        BM4["4. SV_analysis\n(MUMmer, vg)"]:::variation
        BM5["5. Selective_sweep"]:::downstream
        BM6["6. GWAS"]:::downstream
    end
    Snakemake --> BM1
    BM1 --> BM2 --> BM3 --> BM4 --> BM5 --> BM6

    click BM1 "https://github.com/yedomon/pangenome/blob/main/Broomcorn millet pangenome scripts/1.Assembly"
    click BM2 "https://github.com/yedomon/pangenome/blob/main/Broomcorn millet pangenome scripts/2.Annotation"
    click BM3 "https://github.com/yedomon/pangenome/blob/main/Broomcorn millet pangenome scripts/3.Pangenome_analysis"
    click BM4 "https://github.com/yedomon/pangenome/blob/main/Broomcorn millet pangenome scripts/4.SV_analysis"
    click BM5 "https://github.com/yedomon/pangenome/blob/main/Broomcorn millet pangenome scripts/5.Selective_sweep"
    click BM6 "https://github.com/yedomon/pangenome/blob/main/Broomcorn millet pangenome scripts/6.GWAS"

    %% Elymus nutans Pipeline
    subgraph "Elymus nutans Pipeline"
        direction TB
        EN1["01_assembly"]:::assembly
        EN2["02_annotation"]:::annotation
        EN3["03_genome_analysis"]:::pangenome
        EN4["04_variation_calling"]:::variation
        EN5["05_population_analysis"]:::downstream
        EN6["06_environmental adaptation"]:::downstream
        EN7["07_UVB_GWAS"]:::downstream
    end
    Snakemake --> EN1
    EN1 --> EN2 --> EN3 --> EN4 --> EN5 --> EN6 --> EN7

    click EN1 "https://github.com/yedomon/pangenome/tree/main/Elymus_nutans-genome-origin-and-evolution-main/.../01_assembly"
    click EN2 "https://github.com/yedomon/pangenome/tree/main/Elymus_nutans-genome-origin-and-evolution-main/.../02_annotation"
    click EN3 "https://github.com/yedomon/pangenome/tree/main/Elymus_nutans-genome-origin-and-evolution-main/.../03_genome_analysis"
    click EN4 "https://github.com/yedomon/pangenome/tree/main/Elymus_nutans-genome-origin-and-evolution-main/.../04_variation_calling"
    click EN5 "https://github.com/yedomon/pangenome/tree/main/Elymus_nutans-genome-origin-and-evolution-main/.../05_population_analysis"
    click EN6 "https://github.com/yedomon/pangenome/tree/main/Elymus_nutans-genome-origin-and-evolution-main/.../06_environmental adaptation"
    click EN7 "https://github.com/yedomon/pangenome/tree/main/Elymus_nutans-genome-origin-and-evolution-main/.../07_UVB_GWAS"

    %% Grapevine Pipeline
    subgraph "Grapevine Pipeline"
        direction TB
        GV1["Genome Assembly"]:::assembly
        GV2["NLR Gene Annotation"]:::annotation
        GV3["SV & Graph Genotyping"]:::variation
        GV4["eQTL Mapping"]:::downstream
        GV5["Gene Family Evolution"]:::pangenome
        GV6["RNA-seq Expression"]:::downstream
    end
    Snakemake --> GV1
    GV1 --> GV2 --> GV3 --> GV4 --> GV5 --> GV6

    click GV1 "https://github.com/yedomon/pangenome/tree/main/SuperPangenomeofGrapevines-main/genomeAssembly"
    click GV2 "https://github.com/yedomon/pangenome/tree/main/SuperPangenomeofGrapevines-main/NLR_genes"
    click GV3 "https://github.com/yedomon/pangenome/tree/main/SuperPangenomeofGrapevines-main/SV"
    click GV4 "https://github.com/yedomon/pangenome/tree/main/SuperPangenomeofGrapevines-main/UpdateEqtl"
    click GV5 "https://github.com/yedomon/pangenome/tree/main/SuperPangenomeofGrapevines-main/panGeneFamilies"
    click GV6 "https://github.com/yedomon/pangenome/tree/main/SuperPangenomeofGrapevines-main/rnaseq"

    %% Wild Rice Pipeline
    subgraph "Wild Rice Pipeline"
        direction TB
        WR1["1. Genome_assembly"]:::assembly
        WR2["2. Gene_annotation"]:::annotation
        WR3["3. TE_annotation"]:::annotation
        WR4["4. Variation_calling"]:::variation
        WR5["5. Pangenome_analysis"]:::pangenome
        WR6["6. Evolutionary_analysis"]:::downstream
        WR7["7. Domestication_analysis"]:::downstream
        WR8["8. Indica-japonica_differentiated"]:::downstream
    end
    Snakemake --> WR1
    WR1 --> WR2 --> WR3 --> WR4 --> WR5 --> WR6 --> WR7 --> WR8

    click WR1 "https://github.com/yedomon/pangenome/blob/main/Wild-rice-Pangenome-Project-main/.../1. Genome_assembly"
    click WR2 "https://github.com/yedomon/pangenome/blob/main/Wild-rice-Pangenome-Project-main/.../2. Gene_annotation"
    click WR3 "https://github.com/yedomon/pangenome/blob/main/Wild-rice-Pangenome-Project-main/.../3. TE_annotation"
    click WR4 "https://github.com/yedomon/pangenome/blob/main/Wild-rice-Pangenome-Project-main/.../4. Variation_calling"
    click WR5 "https://github.com/yedomon/pangenome/blob/main/Wild-rice-Pangenome-Project-main/.../5. Pangenome_analysis"
    click WR6 "https://github.com/yedomon/pangenome/blob/main/Wild-rice-Pangenome-Project-main/.../6. Evolutionary_analysis"
    click WR7 "https://github.com/yedomon/pangenome/blob/main/Wild-rice-Pangenome-Project-main/.../7. Domestication_analysis"
    click WR8 "https://github.com/yedomon/pangenome/blob/main/Wild-rice-Pangenome-Project-main/.../8. Indica-japonica_differentiated"

    %% Legend
    subgraph "Legend"
        direction TB
        A["Assembly"]:::assembly
        B["Annotation"]:::annotation
        C["Pangenome"]:::pangenome
        D["Variation"]:::variation
        E["Downstream"]:::downstream
        F["Data Source"]:::data
        G["Orchestration"]:::orchestrator
    end

    classDef assembly fill:#cce5ff,stroke:#004085;
    classDef annotation fill:#d4edda,stroke:#155724;
    classDef pangenome fill:#fff3cd,stroke:#856404;
    classDef variation fill:#f5c6cb,stroke:#721c24;
    classDef downstream fill:#e2e3e5,stroke:#6c757d;
    classDef data fill:#d1ecf1,stroke:#0c5460;
    classDef orchestrator fill:#e8d7f3,stroke:#4e148c;
```
