# liaob21/colora architecture

> Architecture diagram of the GitHub repository liaob21/colora, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/liaob21/colora
- Repository: https://github.com/liaob21/colora

## Overview

##### Project Type and Purpose
- This is an open-source, Snakemake-based bioinformatics workflow named “colora” for *de novo* genome assembly.
- It orchestrates a series of third-party tools (Hifiasm, Fastp, KMC, GenomeScope2, Arima Genomics pipeline, YaHS, Oatk, QUAST, BUSCO, etc.) to go from raw sequencing reads (HiFi, ONT, Hi-C) to polished, chromosome-scale assemblies with quality reports.

##### File-Structure Analysis
- Top-level:
- `.github/workflows`: CI, PR checks, automated releases
- `config/`: user-editable global settings (`config.yaml`, test config)
- `workflow/`: core Snakemake pipeline
– `Snakefile`: master workflow definition
– `envs/`: Conda environment YAMLs per tool
– `frozen_envs/`: pinned-version envs for reproducibility
– `rules/`: modular rule files (*.smk) for each step
- `scripts/`: small helper scripts (e.g., adapter trimming, BAM combiner)
- `test_data/`: minimal input files and test-specific resources
- `.template/`: templating scaffolds for new workflows
- Configuration files (`config/config.yaml` and `config/README.md`) define inputs, parameters, paths to external DBs (Busco, OatkDB, FCS-GX), and toggles for optional steps.

##### README Insights
- Outlines the sequential steps:
a. Read QC (NanoPlot for HiFi/ONT; Fastp for Hi-C)
b. k-mer profiling (KMC → GenomeScope2)
c. Assembly (Hifiasm, optional ONT integration, optional Hi-C phasing)
d. Contaminant removal (FCS-GX)
e. Haplotype duplication purge (purge_dups)
f. Hi-C mapping & scaffolding (Arima pipeline → YaHS)
g. Organelle assembly (Oatk) + QC (gfastats, Bandage)
h. Final QC (QUAST, BUSCO)
- Describes Conda/Mamba + Snakemake deployment and optional SLURM profiles.

a. Main Components (use as blocks):
- Snakemake Orchestrator (controller)
- Configuration Layer (`config/config.yaml`, templates)
- Input Data Repositories (raw FASTQ, external DB symlinks)
- Rule Engines grouped by domain:
- Preprocessing & QC (fastp, NanoPlot)
- k-mer Analysis (KMC → GenomeScope2)
- Assembly (Hifiasm ± ONT)
- Decontamination (FCS-GX)
- Duplication Purge (purge_dups)
- Hi-C Mapping & Scaffolding (Arima → YaHS)
- Organelle Processing (Oatk → gfastats → Bandage)
- Final QC (QUAST → BUSCO)
- Conda Environment Manager (per-rule envs & frozen_envs)
- Custom Scripts (filter_five_end.pl, two_read_bam_combiner.pl, etc.)
- Outputs (assembly FASTA, QC reports, logs)

- Config → Snakemake (parameters feed the orchestrator)
- Raw Reads → Preprocessing → Cleaned Reads
- Cleaned Reads → k-mer Analysis & Assembly
- Assembly → Optional FCS-GX → Purged Assembly
- Purged Assembly + Hi-C Reads → Mapping → Scaffolding
- Assembly & Organelle Reads → Oatk → Organelle Assemblies
- All assemblies → QC tools → Final reports

c. Architectural Patterns & Principles:
- Modular rule-based DAG (Snakemake)
- Environment isolation (Conda env per tool)
- Configuration-driven (“config/config.yaml”)
- Extensibility via adding new `.smk` rules and envs
- Reproducibility via frozen_envs and test_data

d. Key Technologies / Frameworks:
- Snakemake (workflow engine)
- Conda/Mamba (software environments)
- Hifiasm, Fastp, KMC, GenomeScope2, FCS-GX, purge_dups, Arima Genomics pipeline, YaHS, Oatk, gfastats, Bandage, QUAST, BUSCO
- Perl/AWK scripts for minor data wrangling

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input & Config Layer
    subgraph "Input & Config" 
        direction TB
        CFG1["config/config.yaml"]:::input
        CFG2["config/config_test.yaml"]:::input
        CFG3["config/README.md"]:::input
        CFG4[".template/config/config.yaml.tmpl.tmpl"]:::input
        RAW_HIFI[(test_data/raw_hifi/)]:::input
        RAW_ONT[(test_data/raw_ont/)]:::input
        RAW_HIC[(test_data/raw_hic/)]:::input
    end

    %% Snakemake Orchestrator
    subgraph "Orchestrator"
        direction TB
        SNAKE["Snakemake Orchestrator"]:::controller
    end

    %% Preprocessing & QC
    subgraph "Preprocessing & QC" 
        direction TB
        FASTP["Fastp"]:::pre
        NANO["NanoPlot"]:::pre
    end

    %% k-mer Analysis
    subgraph "k-mer Analysis"
        direction TB
        KMC["KMC"]:::analysis
        GSCOPE["GenomeScope2"]:::analysis
    end

    %% Assembly & Scaffolding
    subgraph "Assembly & Scaffolding"
        direction TB
        subgraph "Assembly"
            direction TB
            HIFIPREP["Hifi_prep"]:::assem
            HIFIASM["Hifiasm"]:::assem
        end
        subgraph "Decontamination & Purge"
            direction TB
            FCSGX["FCS-GX"]:::assem
            PURGE1["purge_dups"]:::assem
            PURGE2["purge_dups2"]:::assem
        end
        subgraph "Hi-C Mapping & Scaffolding"
            direction TB
            BWA_INDEX["bwa_index"]:::assem
            BWA_MEM["bwa_mem"]:::assem
            YAHS["YaHS"]:::assem
        end
        subgraph "Organelle Processing"
            direction TB
            OATK["Oatk"]:::assem
            GFAST["gfastats"]:::assem
            BANDAGE["Bandage"]:::assem
        end
    end

    %% Final QC
    subgraph "Post-assembly QC"
        direction TB
        QUAST["QUAST"]:::post
        BUSCO["BUSCO"]:::post
    end

    %% Env Manager & CI
    subgraph "Environment & CI"
        direction TB
        ENV_MGMT["Conda Env Manager"]:::env
        CI1[".github/workflows/main.yml"]:::ci
        CI2[".github/workflows/conventional-prs.yml"]:::ci
        CI3[".github/workflows/release-please.yml"]:::ci
    end

    %% Outputs
    ASSEMBLY[(Assembly FASTA)]:::output
    REPORTS[(QC Reports & Logs)]:::output

    %% Connections
    CFG1 --> SNAKE
    CFG2 --> SNAKE
    CFG3 --> SNAKE
    CFG4 --> SNAKE

    RAW_HIFI --> FASTP
    RAW_ONT --> FASTP
    RAW_HIC --> FASTP
    FASTP --> NANO
    NANO --> SNAKE

    FASTP --> KMC
    KMC --> GSCOPE
    GSCOPE --> SNAKE

    SNAKE --> HIFIPREP
    HIFIPREP --> HIFIASM
    HIFIASM --> FCSGX
    FCSGX --> PURGE1
    PURGE1 --> PURGE2
    PURGE2 --> BWA_INDEX
    BWA_INDEX --> BWA_MEM
    BWA_MEM --> YAHS
    YAHS --> ASSEMBLY

    RAW_HIC --> BWA_INDEX

    HIFIASM --> OATK
    OATK --> GFAST
    GFAST --> BANDAGE
    BANDAGE --> ASSEMBLY

    ASSEMBLY --> QUAST
    ASSEMBLY --> BUSCO
    QUAST --> REPORTS
    BUSCO --> REPORTS

    ENV_MGMT --> FASTP
    ENV_MGMT --> NANO
    ENV_MGMT --> KMC
    ENV_MGMT --> GSCOPE
    ENV_MGMT --> HIFIASM
    ENV_MGMT --> FCSGX
    ENV_MGMT --> PURGE1
    ENV_MGMT --> PURGE2
    ENV_MGMT --> BWA_INDEX
    ENV_MGMT --> BWA_MEM
    ENV_MGMT --> YAHS
    ENV_MGMT --> OATK
    ENV_MGMT --> GFAST
    ENV_MGMT --> BANDAGE
    ENV_MGMT --> QUAST
    ENV_MGMT --> BUSCO

    CI1 --> SNAKE
    CI2 --> SNAKE
    CI3 --> SNAKE

    %% Click Events
    click SNAKE "https://github.com/liaob21/colora/tree/main/workflow/Snakefile"
    click CFG1 "https://github.com/liaob21/colora/blob/main/config/config.yaml"
    click CFG2 "https://github.com/liaob21/colora/blob/main/config/config_test.yaml"
    click CFG3 "https://github.com/liaob21/colora/blob/main/config/README.md"
    click CFG4 "https://github.com/liaob21/colora/blob/main/.template/config/config.yaml.tmpl.tmpl"
    click CI1 "https://github.com/liaob21/colora/blob/main/.github/workflows/main.yml"
    click CI2 "https://github.com/liaob21/colora/blob/main/.github/workflows/conventional-prs.yml"
    click CI3 "https://github.com/liaob21/colora/blob/main/.github/workflows/release-please.yml"
    click RAW_HIFI "https://github.com/liaob21/colora/tree/main/test_data/raw_hifi/"
    click RAW_ONT "https://github.com/liaob21/colora/tree/main/test_data/raw_ont/"
    click RAW_HIC "https://github.com/liaob21/colora/tree/main/test_data/raw_hic/"
    click FASTP "https://github.com/liaob21/colora/blob/main/workflow/rules/fastp.smk"
    click NANO "https://github.com/liaob21/colora/blob/main/workflow/rules/nanoplot.smk"
    click KMC "https://github.com/liaob21/colora/blob/main/workflow/rules/kmc.smk"
    click GSCOPE "https://github.com/liaob21/colora/blob/main/workflow/rules/genomescope2.smk"
    click HIFIPREP "https://github.com/liaob21/colora/blob/main/workflow/rules/hifi_prep.smk"
    click HIFIASM "https://github.com/liaob21/colora/blob/main/workflow/rules/hifiasm.smk"
    click FCSGX "https://github.com/liaob21/colora/blob/main/workflow/rules/ncbi_fcsgx.smk"
    click PURGE1 "https://github.com/liaob21/colora/blob/main/workflow/rules/purge_dups.smk"
    click PURGE2 "https://github.com/liaob21/colora/blob/main/workflow/rules/purge_dups2.smk"
    click BWA_INDEX "https://github.com/liaob21/colora/blob/main/workflow/rules/bwa_index.smk"
    click BWA_MEM "https://github.com/liaob21/colora/blob/main/workflow/rules/bwa_mem.smk"
    click YAHS "https://github.com/liaob21/colora/blob/main/workflow/rules/yahs.smk"
    click OATK "https://github.com/liaob21/colora/blob/main/workflow/rules/oatk.smk"
    click GFAST "https://github.com/liaob21/colora/blob/main/workflow/rules/gfastats.smk"
    click BANDAGE "https://github.com/liaob21/colora/blob/main/workflow/rules/bandage.smk"
    click QUAST "https://github.com/liaob21/colora/blob/main/workflow/rules/quast.smk"
    click BUSCO "https://github.com/liaob21/colora/blob/main/workflow/rules/busco.smk"
    click ENV_MGMT "https://github.com/liaob21/colora/tree/main/workflow/envs/"
    click ENV_MGMT "https://github.com/liaob21/colora/tree/main/workflow/frozen_envs/"

    %% Styles
    classDef input fill:#cce5ff,stroke:#004085
    classDef controller fill:#e2e3e5,stroke:#6c757d
    classDef pre fill:#d4edda,stroke:#155724
    classDef analysis fill:#d1ecf1,stroke:#0c5460
    classDef assem fill:#fff3cd,stroke:#856404
    classDef post fill:#f5c6cb,stroke:#721c24
    classDef env fill:#e2e3e5,stroke:#6c757d
    classDef ci fill:#e2e3e5,stroke:#6c757d
    classDef output fill:#d6d8db,stroke:#1b1e21
```
