# biopus/barcode-kit architecture

> Architecture diagram of the GitHub repository biopus/barcode-kit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/biopus/barcode-kit
- Repository: https://github.com/biopus/barcode-kit

## Overview

1. `barcode-kit` is a local-first DNA barcode pipeline centered on one persistent workspace under `data_dir`. The core loop is: query GenBank, cache raw records, normalize taxonomy, parse marker sequences, then export FASTA datasets and JSON reports from local state.

2. The main user boundary is the Typer CLI in `src/barcode_kit/cli.py`. It exposes distinct operational commands for `sync`, `build`, `qc`, and `db` management, so the repository is organized around command-driven workflows rather than a long-running service.

3. Configuration is loaded from `config.py` and is part of the architecture, not just convenience. It controls filesystem layout, collector behavior, GenBank credentials, and QC parameters. The default workspace includes a SQLite database plus `cache/genbank/` and `logs/`, which makes local state an explicit subsystem.

4. The ingestion path is `genbank.py` plus `blast.py`/`itsxrust.py`/`parser.py` around record acquisition and sequence interpretation. `sync` contacts NCBI E-utilities, fetches accession records, and stores the downloaded GenBank text locally. The README indicates this step can use taxon names or NCBI TaxId, and it supports the markers `its`, `its2`, `matk`, and `rbcl`.

5. Taxonomy is a separate normalization boundary in `taxonomy.py`. It uses ETE’s `NCBITaxa` local taxonomy database to turn NCBI taxon hints or scientific names into canonical names and lineages. This is important because the system treats taxonomy resolution as a local enrichment step before data is written to storage.

6. `storage.py` is the persistence layer and the main integration point between modules. It maintains the SQLite schema for accession-level cache records and taxonomy rows, and it is also the source of truth for later build and cleanup operations. The `db` commands operate directly on this layer for status, info, remove, clear, and prune.

7. `builder.py` implements the dataset construction boundary. It reads only from local cache and produces marker-specific FASTA outputs plus `dataset.json` and `build_report.json`. The build stage is intentionally separate from synchronization: it does not re-query NCBI and does not apply QC filters unless the later `qc` command is run.

8. `validation.py` is the QC layer that consumes a built dataset and emits a filtered FASTA plus `qc_report.json`. It supports sequence-level rules like length and ambiguity thresholds, as well as taxonomy-based exclusions. This stage is designed to be repeatable from `raw/`, not incremental over prior QC outputs.

9. The phylogeny subsystem in `phylogeny.py` is a thin wrapper around external bioinformatics tools. It orchestrates alignment, trimming, tree building, and TreeShrink-based outlier detection through subprocess runners. The README shows dependencies on `mafft`, `iqtree`, `trimal`, and `run_treeshrink.py`, so this module is an execution boundary to system-installed tools rather than an in-process algorithm library.

10. The domain layer in `models.py`, `exceptions.py`, and `__init__.py` provides shared objects and error semantics used across the pipeline. That keeps the CLI, storage, parser, and QC code aligned on the same record and report shapes, which is important for generating consistent JSON outputs.

11. The repository also has a clear testing boundary. The `tests/` suite covers CLI behavior, GenBank download logic, parser validation, taxonomy, storage, phylogeny, and module exports, which mirrors the architecture of the package and suggests the intended contracts between modules are stable and explicitly verified.

Main entry points: [`src/barcode_kit/builder.py`](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/builder.py) (Builder), [`src/barcode_kit/cli.py`](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py) (CLI), [`src/barcode_kit/storage.py`](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/storage.py) (Storage), [`src/barcode_kit/validation.py`](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/validation.py) (QC)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User Interface"]
  node_cli["CLI<br/>typer commands<br/>[cli.py]"]
  node_sync_cmd(("Sync<br/>workflow command<br/>[cli.py]"))
  node_build_cmd(("Build<br/>workflow command<br/>[cli.py]"))
  node_qc_cmd(("QC command<br/>workflow command<br/>[cli.py]"))
  node_db_cmd(("DB command<br/>workflow command<br/>[cli.py]"))
end

subgraph group_pipeline["Pipeline Core"]
  node_genbank["GenBank<br/>NCBI fetch<br/>[genbank.py]"]
  node_taxonomy["Taxonomy<br/>NCBITaxa normalization<br/>[taxonomy.py]"]
  node_parser["Parser<br/>[parser.py]"]
  node_blast["BLAST rescue<br/>sequence rescue<br/>[blast.py]"]
  node_itsxrust["ITSxRust<br/>marker finder<br/>[itsxrust.py]"]
  node_builder["Builder<br/>dataset assembly<br/>[builder.py]"]
  node_validation["QC<br/>dataset validation<br/>[validation.py]"]
end

subgraph group_storage["Local State"]
  node_storage[("Storage<br/>sqlite persistence<br/>[storage.py]")]
end

subgraph group_analysis["Phylogeny"]
  node_phylogeny["Phylogeny<br/>tool runner<br/>[phylogeny.py]"]
end

subgraph group_shared["Shared Domain"]
  node_config["Config<br/>[config.py]"]
  node_models["Models<br/>shared records<br/>[models.py]"]
  node_exceptions["Errors<br/>shared exceptions<br/>[exceptions.py]"]
  node_module_init["Package API<br/>exports<br/>[__init__.py]"]
end

node_cli -->|"exposes"| node_sync_cmd
node_cli -->|"exposes"| node_build_cmd
node_cli -->|"exposes"| node_qc_cmd
node_cli -->|"exposes"| node_db_cmd
node_sync_cmd -->|"fetches"| node_genbank
node_sync_cmd -->|"resolves"| node_taxonomy
node_sync_cmd -->|"persists"| node_storage
node_genbank -->|"caches"| node_storage
node_taxonomy -->|"enriches"| node_storage
node_storage -->|"reads"| node_builder
node_builder -->|"interprets"| node_parser
node_builder -->|"rescues"| node_blast
node_builder -->|"detects"| node_itsxrust
node_builder -->|"emits"| node_models
node_builder -->|"hands off"| node_validation
node_qc_cmd -->|"runs"| node_validation
node_validation -->|"emits"| node_models
node_db_cmd -->|"operates on"| node_storage
node_phylogeny -->|"reports"| node_models
node_cli -->|"loads"| node_config
node_builder -->|"reads"| node_config
node_validation -->|"reads"| node_config
node_genbank -->|"reads"| node_config
node_module_init -->|"exports"| node_models
node_module_init -->|"exports"| node_exceptions

click node_cli "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py"
click node_config "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/config.py"
click node_models "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/models.py"
click node_exceptions "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/exceptions.py"
click node_storage "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/storage.py"
click node_genbank "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/genbank.py"
click node_taxonomy "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/taxonomy.py"
click node_parser "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/parser.py"
click node_blast "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/blast.py"
click node_itsxrust "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/itsxrust.py"
click node_builder "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/builder.py"
click node_validation "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/validation.py"
click node_phylogeny "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/phylogeny.py"
click node_module_init "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/__init__.py"
click node_sync_cmd "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py"
click node_build_cmd "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py"
click node_qc_cmd "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py"
click node_db_cmd "https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_sync_cmd,node_build_cmd,node_qc_cmd,node_db_cmd toneBlue
class node_genbank,node_taxonomy,node_parser,node_blast,node_itsxrust,node_builder,node_validation toneAmber
class node_storage toneMint
class node_phylogeny toneRose
class node_config,node_models,node_exceptions,node_module_init toneIndigo
```

## Components

### User Interface

CLI entrypoint and command surfaces

- [CLI](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py): typer commands · Command boundary for sync, build, qc, and db operations · `src/barcode_kit/cli.py`
- [Sync](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py): workflow command · Fetches records, normalizes taxonomy, and populates cache · `src/barcode_kit/cli.py`
- [Build](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py): workflow command · Constructs marker FASTA outputs and JSON summaries · `src/barcode_kit/cli.py`
- [QC command](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py): workflow command · Runs validation over built datasets · `src/barcode_kit/cli.py`
- [DB command](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/cli.py): workflow command · Inspects and cleans the persistent cache database · `src/barcode_kit/cli.py`

### Pipeline Core

Ingestion, parsing, build, and QC workflow

- [GenBank](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/genbank.py): NCBI fetch · Downloads and caches accession records from E-utilities · `src/barcode_kit/genbank.py`
- [Taxonomy](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/taxonomy.py): NCBITaxa normalization · Resolves names and taxids to canonical taxonomy · `src/barcode_kit/taxonomy.py`
- [Parser](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/parser.py): record parser · Interprets cached records and extracts marker sequences · `src/barcode_kit/parser.py`
- [BLAST rescue](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/blast.py): sequence rescue · Supplementary marker identification and rescue logic · `src/barcode_kit/blast.py`
- [ITSxRust](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/itsxrust.py): marker finder · ITS/ITS2 marker detection via external tool integration · `src/barcode_kit/itsxrust.py`
- [Builder](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/builder.py): dataset assembly · Builds FASTA datasets and build reports from local cache · `src/barcode_kit/builder.py`
- [QC](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/validation.py): dataset validation · Applies length, ambiguity, and taxonomy filters to builds · `src/barcode_kit/validation.py`

### Local State

Persistent workspace and SQLite-backed cache

- [Storage](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/storage.py): sqlite persistence · Source of truth for cache records and taxonomy rows · `src/barcode_kit/storage.py`

### Phylogeny

External-tool orchestration for downstream analysis

- [Phylogeny](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/phylogeny.py): tool runner · Runs alignment, trimming, tree building, and TreeShrink · `src/barcode_kit/phylogeny.py`

### Shared Domain

Common models, config, and error types

- [Config](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/config.py): workspace config · Defines workspace layout, credentials, and QC parameters · `src/barcode_kit/config.py`
- [Models](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/models.py): shared records · Common data structures for records and reports · `src/barcode_kit/models.py`
- [Errors](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/exceptions.py): shared exceptions · Pipeline-specific error semantics · `src/barcode_kit/exceptions.py`
- [Package API](https://github.com/biopus/barcode-kit/blob/master/src/barcode_kit/__init__.py): exports · Package-level exports for the public API · `src/barcode_kit/__init__.py`

## Connections

- CLI → Sync: exposes · Defines the sync workflow
- CLI → Build: exposes · Defines the build workflow
- CLI → QC command: exposes · Defines the QC workflow
- CLI → DB command: exposes · Defines database management commands
- Sync → GenBank: fetches · Retrieves accession records from NCBI
- Sync → Taxonomy: resolves · Normalizes names and taxids
- Sync → Storage: persists · Stores downloaded records and taxonomy state
- GenBank → Storage: caches · Writes raw GenBank text into local state
- Taxonomy → Storage: enriches · Stores canonical taxonomy rows
- Storage → Builder: reads · Build stage consumes only local cache
- Builder → Parser: interprets · Uses parsed marker sequences from cached records
- Builder → BLAST rescue: rescues · Can use BLAST-based rescue logic
- Builder → ITSxRust: detects · Can use ITS marker detection tooling
- Builder → Models: emits · Produces dataset and build report structures
- Builder → QC: hands off · QC is a separate later stage
- QC command → QC: runs · Applies filters to built datasets
- QC → Models: emits · Produces QC report structures
- DB command → Storage: operates on · Status, info, remove, clear, and prune use persistence layer
- Phylogeny → Models: reports · Uses shared report shapes for analysis outputs
- CLI → Config: loads · CLI is configured by workspace and QC settings
- Builder → Config: reads · Build behavior depends on workspace layout and markers
- QC → Config: reads · QC thresholds come from configuration
- GenBank → Config: reads · Uses credentials and collector settings
- Package API → Models: exports · Re-exports public package types
- Package API → Errors: exports · Re-exports package error types