# pacificbiosciences/pb-metagenomics-tools architecture

> Architecture diagram of the GitHub repository pacificbiosciences/pb-metagenomics-tools, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pacificbiosciences/pb-metagenomics-tools
- Repository: https://github.com/pacificbiosciences/pb-metagenomics-tools

## Overview

This repository is a collection of Snakemake workflows and companion utilities for HiFi metagenomics, not one single analysis pipeline. Its main user workflows produce quality-filtered MAGs, compare and consolidate MAG sets, or profile taxa from reads using Sourmash, DIAMOND/MEGAN, or minimap2/MEGAN. Separate scripts convert profiling outputs into standard reports and compare those reports. The README documents the major stages and outputs; sampled source supports selected report-conversion and filtering behavior, so unsampled workflow wiring is represented only at the documented level.

Main entry points: [`HiFi-MAG-Pipeline/Snakefile-hifimags.smk`](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/Snakefile-hifimags.smk) (MAG workflow), [`Taxonomic-Profiling-Sourmash/Snakefile-sourmash`](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/Taxonomic-Profiling-Sourmash/Snakefile-sourmash) (Sourmash workflow), [`Taxonomic-Profiling-Diamond-Megan/Snakefile-diamond-megan.smk`](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/Taxonomic-Profiling-Diamond-Megan/Snakefile-diamond-megan.smk) (DIAMOND MEGAN workflow), [`Taxonomic-Profiling-Minimap-Megan/Snakefile-minimap-megan`](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/Taxonomic-Profiling-Minimap-Megan/Snakefile-minimap-megan) (Minimap MEGAN workflow)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_mag["MAG recovery"]
  node_magflow["MAG workflow"]
  node_protect["Long-contig protection"]
  node_binning["Binning and merge"]
  node_magqc["MAG QC and taxonomy"]
  node_magresult["MAGs and summaries<br/>[MAG-Summary.py]"]
end

subgraph group_mirror["MAG comparison"]
  node_mirrorflow["Mirror workflow"]
  node_mirrorqc["Filter bin sets"]
  node_compare["Compare bins"]
  node_mirrorresult["Consolidated MAG set"]
end

subgraph group_profiling["Taxonomic profiling"]
  node_sourmashflow["Sourmash workflow"]
  node_sourmashresult["Sourmash profile"]
  node_diamondflow["DIAMOND MEGAN workflow"]
  node_minimapflow["Minimap MEGAN workflow"]
  node_profiletools["Alignment and classification"]
  node_taxonomydb[("Taxonomy databases")]
  node_profileformats["Taxonomic and functional reports"]
end

subgraph group_reports["Report utilities"]
  node_converters["Format converters"]
  node_taxonomy[("NCBI taxonomy lookup")]
  node_comparekr["Compare kreports"]
  node_krresults["Standard reports and plots"]
end

node_analyst(("Analyst"))
node_assembly["HiFi assembly"]
node_reads["HiFi reads"]
node_megan["MEGAN-LR"]
node_sourmash["Sourmash"]
node_reportuser(("Report user"))
node_profilefiles["Profiler outputs"]

node_analyst -->|"runs"| node_magflow
node_assembly -->|"provides assembly"| node_magflow
node_magflow -->|"protects contigs"| node_protect
node_magflow -->|"bins and merges"| node_binning
node_magflow -->|"runs QC"| node_magqc
node_magflow -->|"produces"| node_magresult
node_protect -->|"passes contigs"| node_binning
node_binning -->|"passes bins"| node_magqc
node_magqc -->|"records quality"| node_magresult
node_analyst -->|"runs"| node_mirrorflow
node_mirrorflow -->|"filters bins"| node_mirrorqc
node_mirrorflow -->|"compares sets"| node_compare
node_mirrorqc -->|"provides passing bins"| node_compare
node_compare -->|"consolidates sets"| node_mirrorresult
node_reads -->|"provides reads"| node_sourmashflow
node_analyst -->|"runs"| node_sourmashflow
node_sourmashflow -->|"classifies"| node_sourmash
node_sourmash -.->|"queries"| node_taxonomydb
node_sourmashflow -->|"writes profile"| node_sourmashresult
node_reads -->|"provides reads"| node_diamondflow
node_analyst -->|"runs"| node_diamondflow
node_diamondflow -->|"aligns and classifies"| node_profiletools
node_profiletools -->|"summarizes"| node_megan
node_megan -.->|"uses annotations"| node_taxonomydb
node_diamondflow -->|"produces reports"| node_profileformats
node_reads -->|"provides reads"| node_minimapflow
node_analyst -->|"runs"| node_minimapflow
node_minimapflow -->|"aligns reads"| node_profiletools
node_profiletools -->|"summarizes"| node_megan
node_minimapflow -->|"produces reports"| node_profileformats
node_reportuser -->|"requests conversion"| node_converters
node_profilefiles -->|"provides input"| node_converters
node_converters -->|"resolves lineages"| node_taxonomy
node_converters -->|"writes reports"| node_krresults
node_reportuser -->|"requests comparison"| node_comparekr
node_profilefiles -->|"provides kreports"| node_comparekr
node_comparekr -->|"creates plots"| node_krresults

click node_magflow "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/Snakefile-hifimags.smk"
click node_protect "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/Snakefile-hifimags.smk"
click node_binning "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/Snakefile-hifimags.smk"
click node_magqc "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/Snakefile-hifimags.smk"
click node_magresult "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/scripts/MAG-Summary.py"
click node_mirrorflow "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/pb-MAG-mirror/Snakefile-pb-mag-mirror.smk"
click node_mirrorqc "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/pb-MAG-mirror/scripts/Filter-Checkm2-Bins.py"
click node_compare "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/pb-MAG-mirror/scripts/Perform-bin-comparisons.py"
click node_mirrorresult "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/pb-MAG-mirror/scripts/Copy-consolidated-bins.py"
click node_sourmashflow "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/Taxonomic-Profiling-Sourmash/Snakefile-sourmash"
click node_sourmashresult "https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/Taxonomic-Profiling-Sourmash"
click node_diamondflow "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/Taxonomic-Profiling-Diamond-Megan/Snakefile-diamond-megan.smk"
click node_minimapflow "https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/Taxonomic-Profiling-Minimap-Megan/Snakefile-minimap-megan"
click node_profiletools "https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/Taxonomic-Profiling-Diamond-Megan"
click node_profileformats "https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/Taxonomic-Profiling-Diamond-Megan"
click node_converters "https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/pb-metagenomics-scripts/Convert-to-kreport-mpa"
click node_comparekr "https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/pb-metagenomics-scripts/Compare-kreport-taxonomic-profiles"
click node_krresults "https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/pb-metagenomics-scripts/Compare-kreport-taxonomic-profiles"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_magflow,node_protect,node_binning,node_magqc,node_magresult,node_reportuser toneBlue
class node_mirrorflow,node_mirrorqc,node_compare,node_mirrorresult toneAmber
class node_sourmashflow,node_sourmashresult,node_diamondflow,node_minimapflow,node_profiletools,node_taxonomydb,node_profileformats toneMint
class node_converters,node_taxonomy,node_comparekr,node_krresults toneRose
class node_analyst,node_assembly,node_reads,node_megan,node_sourmash,node_profilefiles toneIndigo
```

## Components

### MAG recovery

- [MAG workflow](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/Snakefile-hifimags.smk): `HiFi-MAG-Pipeline/Snakefile-hifimags.smk`
- [Long-contig protection](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/Snakefile-hifimags.smk): `HiFi-MAG-Pipeline/Snakefile-hifimags.smk`
- [Binning and merge](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/Snakefile-hifimags.smk): `HiFi-MAG-Pipeline/Snakefile-hifimags.smk`
- [MAG QC and taxonomy](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/Snakefile-hifimags.smk): `HiFi-MAG-Pipeline/Snakefile-hifimags.smk`
- [MAGs and summaries](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/HiFi-MAG-Pipeline/scripts/MAG-Summary.py): `HiFi-MAG-Pipeline/scripts/MAG-Summary.py`

### MAG comparison

- [Mirror workflow](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/pb-MAG-mirror/Snakefile-pb-mag-mirror.smk): `pb-MAG-mirror/Snakefile-pb-mag-mirror.smk`
- [Filter bin sets](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/pb-MAG-mirror/scripts/Filter-Checkm2-Bins.py): `pb-MAG-mirror/scripts/Filter-Checkm2-Bins.py`
- [Compare bins](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/pb-MAG-mirror/scripts/Perform-bin-comparisons.py): `pb-MAG-mirror/scripts/Perform-bin-comparisons.py`
- [Consolidated MAG set](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/pb-MAG-mirror/scripts/Copy-consolidated-bins.py): `pb-MAG-mirror/scripts/Copy-consolidated-bins.py`

### Taxonomic profiling

- [Sourmash workflow](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/Taxonomic-Profiling-Sourmash/Snakefile-sourmash): `Taxonomic-Profiling-Sourmash/Snakefile-sourmash`
- [Sourmash profile](https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/Taxonomic-Profiling-Sourmash): `Taxonomic-Profiling-Sourmash`
- [DIAMOND MEGAN workflow](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/Taxonomic-Profiling-Diamond-Megan/Snakefile-diamond-megan.smk): `Taxonomic-Profiling-Diamond-Megan/Snakefile-diamond-megan.smk`
- [Minimap MEGAN workflow](https://github.com/pacificbiosciences/pb-metagenomics-tools/blob/master/Taxonomic-Profiling-Minimap-Megan/Snakefile-minimap-megan): `Taxonomic-Profiling-Minimap-Megan/Snakefile-minimap-megan`
- [Alignment and classification](https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/Taxonomic-Profiling-Diamond-Megan): `Taxonomic-Profiling-Diamond-Megan`
- **Taxonomy databases**
- [Taxonomic and functional reports](https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/Taxonomic-Profiling-Diamond-Megan): `Taxonomic-Profiling-Diamond-Megan`

### Report utilities

- [Format converters](https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/pb-metagenomics-scripts/Convert-to-kreport-mpa): `pb-metagenomics-scripts/Convert-to-kreport-mpa`
- **NCBI taxonomy lookup**
- [Compare kreports](https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/pb-metagenomics-scripts/Compare-kreport-taxonomic-profiles): `pb-metagenomics-scripts/Compare-kreport-taxonomic-profiles`
- [Standard reports and plots](https://github.com/pacificbiosciences/pb-metagenomics-tools/tree/master/pb-metagenomics-scripts/Compare-kreport-taxonomic-profiles): `pb-metagenomics-scripts/Compare-kreport-taxonomic-profiles`

### Other components

- **Analyst**
- **HiFi assembly**
- **HiFi reads**
- **MEGAN-LR**
- **Sourmash**
- **Report user**
- **Profiler outputs**

## Connections

- Analyst → MAG workflow: runs
- HiFi assembly → MAG workflow: provides assembly
- MAG workflow → Long-contig protection: protects contigs
- MAG workflow → Binning and merge: bins and merges
- MAG workflow → MAG QC and taxonomy: runs QC
- MAG workflow → MAGs and summaries: produces
- Long-contig protection → Binning and merge: passes contigs
- Binning and merge → MAG QC and taxonomy: passes bins
- MAG QC and taxonomy → MAGs and summaries: records quality
- Analyst → Mirror workflow: runs
- Mirror workflow → Filter bin sets: filters bins
- Mirror workflow → Compare bins: compares sets
- Filter bin sets → Compare bins: provides passing bins
- Compare bins → Consolidated MAG set: consolidates sets
- HiFi reads → Sourmash workflow: provides reads
- Analyst → Sourmash workflow: runs
- Sourmash workflow → Sourmash: classifies
- Sourmash → Taxonomy databases: queries
- Sourmash workflow → Sourmash profile: writes profile
- HiFi reads → DIAMOND MEGAN workflow: provides reads
- Analyst → DIAMOND MEGAN workflow: runs
- DIAMOND MEGAN workflow → Alignment and classification: aligns and classifies
- Alignment and classification → MEGAN-LR: summarizes
- MEGAN-LR → Taxonomy databases: uses annotations
- DIAMOND MEGAN workflow → Taxonomic and functional reports: produces reports
- HiFi reads → Minimap MEGAN workflow: provides reads
- Analyst → Minimap MEGAN workflow: runs
- Minimap MEGAN workflow → Alignment and classification: aligns reads
- Alignment and classification → MEGAN-LR: summarizes
- Minimap MEGAN workflow → Taxonomic and functional reports: produces reports
- Report user → Format converters: requests conversion
- Profiler outputs → Format converters: provides input
- Format converters → NCBI taxonomy lookup: resolves lineages
- Format converters → Standard reports and plots: writes reports
- Report user → Compare kreports: requests comparison
- Profiler outputs → Compare kreports: provides kreports
- Compare kreports → Standard reports and plots: creates plots