How yedomon/pangenome works
Project type and purpose
• This is a monorepo of multiple bioinformatics pipelines for pangenome and comparative-genome analyses across different species (rice, alfalfa, broomcorn millet, grapevine, wild rice, Elymus, Arabidopsis, etc.).
• Each subdirectory implements a full analysis workflow (assembly → annotation → variation calling → pangenome graph construction → evolutionary and population analyses → trait/GWAS/eQTL analyses).
File structure
• Top-level directories are named after the species or published projects.
• Within each species folder you typically see numbered stages (e.g., 1.Assembly, 2.Annotation, 3.Pangenome_analysis, 4.SV_analysis, etc.).
• Scripts come in several flavors: shell/Bash (.sh), Snakemake (.smk), Python (.py), R (.R), Perl (.pl).
• Configuration files (.yaml, .ctl), SLURM/PBS submission wrappers (.slurm, PBS_*.sh), and build/work scripts (work.sh) coordinate cluster execution.
README for additional insights
• The root README is essentially a curated index of publications and “code availability” links, pointing to each sub-pipeline.
• There is no single central code module; rather, the structure is intentionally modular to mirror each paper’s methods section.
a. Main components
• Data Ingestion Layer: raw reads (FASTQ), reference genomes, annotation databases (GO, KEGG, Pfam, etc.)
• Workflow Orchestrator: Snakemake/PBS/SLURM scheduler
• Pipeline Modules (per species/project):
– Genome Assembly (HiCanu, Hifiasm, Flye, Juicer/3D-DNA)
– Repeat/TE Annotation (RepeatModeler, RepeatMasker, LTRharvest)
– Gene Prediction & Functional Annotation (Braker3/AUGUSTUS, Maker, DIAMOND, InterProScan, DeepGO)
– Variation Calling (MUMmer/SVMU for SV, GATK/BCFtools for SNP/indel, VG toolkit for graph genotyping)
– Pangenome Graph Construction (minigraph, seqwish, vg)
– Downstream Analyses (Orthofinder, paraAT, Ka/Ks, population structure, PCA, GWAS, selective sweep, eQTL)
• Support Scripts & Utilities: Python modules, R plotting scripts, summary-builders
• Outputs: QC reports, summary tables, VCFs, graph files, figures for publication
b. Relationships & data flow
1. Raw data → Assembly module → Assembled contigs/scaffolds
2. Assemblies + reference genomes → Repeat & gene annotation
3. Annotated genomes + assemblies → alignment & variation calling
4. Variation calls & assemblies → pangenome graph construction
5. Graph + genotype matrices → population genetics (π, FST), GWAS, eQTL
6. Each module reads configs and writes standardized result folders → next module
c. Architectural patterns & principles
• Modular “stage” directories mirror a pipeline pattern (Stage 1, 2, … 8).
• Configuration-driven: each Snakemake or shell script reads a central config/ctl/yaml.
• Reproducibility via workflow manager (Snakemake) + containerized/cluster-wrapped scripts.
• Separation of compute (HPC scheduler) from code.
d. Key technologies & tools
• Workflow: Snakemake, PBS/SLURM scripts
• Languages: Bash, Python, R, Perl
• Assembly: Hifiasm, HiCanu, Flye, Juicer/3D-DNA
• Annotation: RepeatModeler, Braker3, Maker, DIAMOND, InterProScan, DeepGO
• Variation: MUMmer4 (NUCmer, SVMU), vg toolkit, GATK, BCFtools
• Pangenome graph: minigraph, seqwish, vg
• Downstream: Orthofinder, paraAT, EMMAX/EMMAX2, Matrix eQTL, R packages (ClusterProfiler)
Detail is key
• Parallelism across species—e.g., ten lanes side by side—then collapse identical modules into a “shared template” callout.