How liaob21/colora works
Project Type and Purpose
• This is an open-source, Snakemake-based bioinformatics workflow named “colora” for *de novo* genome assembly.
• It orchestrates a series of third-party tools (Hifiasm, Fastp, KMC, GenomeScope2, Arima Genomics pipeline, YaHS, Oatk, QUAST, BUSCO, etc.) to go from raw sequencing reads (HiFi, ONT, Hi-C) to polished, chromosome-scale assemblies with quality reports.
File-Structure Analysis
• Top-level:
• .github/workflows: CI, PR checks, automated releases
• config/: user-editable global settings (config.yaml, test config)
• workflow/: core Snakemake pipeline
– Snakefile: master workflow definition
– envs/: Conda environment YAMLs per tool
– frozen_envs/: pinned-version envs for reproducibility
– rules/: modular rule files (*.smk) for each step
• scripts/: small helper scripts (e.g., adapter trimming, BAM combiner)
• test_data/: minimal input files and test-specific resources
• .template/: templating scaffolds for new workflows
• Configuration files (config/config.yaml and config/README.md) define inputs, parameters, paths to external DBs (Busco, OatkDB, FCS-GX), and toggles for optional steps.
README Insights
• Outlines the sequential steps:
a. Read QC (NanoPlot for HiFi/ONT; Fastp for Hi-C)
b. k-mer profiling (KMC → GenomeScope2)
c. Assembly (Hifiasm, optional ONT integration, optional Hi-C phasing)
d. Contaminant removal (FCS-GX)
e. Haplotype duplication purge (purge_dups)
f. Hi-C mapping & scaffolding (Arima pipeline → YaHS)
g. Organelle assembly (Oatk) + QC (gfastats, Bandage)
h. Final QC (QUAST, BUSCO)
• Describes Conda/Mamba + Snakemake deployment and optional SLURM profiles.
a. Main Components (use as blocks):
• Snakemake Orchestrator (controller)
• Configuration Layer (config/config.yaml, templates)
• Input Data Repositories (raw FASTQ, external DB symlinks)
• Rule Engines grouped by domain:
• Preprocessing & QC (fastp, NanoPlot)
• k-mer Analysis (KMC → GenomeScope2)
• Assembly (Hifiasm ± ONT)
• Decontamination (FCS-GX)
• Duplication Purge (purge_dups)
• Hi-C Mapping & Scaffolding (Arima → YaHS)
• Organelle Processing (Oatk → gfastats → Bandage)
• Final QC (QUAST → BUSCO)
• Conda Environment Manager (per-rule envs & frozen_envs)
• Custom Scripts (filter_five_end.pl, two_read_bam_combiner.pl, etc.)
• Outputs (assembly FASTA, QC reports, logs)
• Config → Snakemake (parameters feed the orchestrator)
• Raw Reads → Preprocessing → Cleaned Reads
• Cleaned Reads → k-mer Analysis & Assembly
• Assembly → Optional FCS-GX → Purged Assembly
• Purged Assembly + Hi-C Reads → Mapping → Scaffolding
• Assembly & Organelle Reads → Oatk → Organelle Assemblies
• All assemblies → QC tools → Final reports
c. Architectural Patterns & Principles:
• Modular rule-based DAG (Snakemake)
• Environment isolation (Conda env per tool)
• Configuration-driven (“config/config.yaml”)
• Extensibility via adding new .smk rules and envs
• Reproducibility via frozen_envs and test_data
d. Key Technologies / Frameworks:
• Snakemake (workflow engine)
• Conda/Mamba (software environments)
• Hifiasm, Fastp, KMC, GenomeScope2, FCS-GX, purge_dups, Arima Genomics pipeline, YaHS, Oatk, gfastats, Bandage, QUAST, BUSCO
• Perl/AWK scripts for minor data wrangling