# alexandrovlab/dupcaller architecture

> Architecture diagram of the GitHub repository alexandrovlab/dupcaller, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alexandrovlab/dupcaller
- Repository: https://github.com/alexandrovlab/dupcaller

## Overview

##### Project Type and Purpose
- DupCaller is a Python-based CLI pipeline for somatic mutation calling and burden estimation on ecNGS data.
- It wraps several “subcommands” (index, trim, call, learn, summarize, aggregateProfile, estimate) and orchestrates external tools (BWA, GATK, samtools).

##### Main Architectural Components
a. CLI Dispatcher
– src/DupCaller.py: Entry point (console script) that parses top-level arguments and delegates to subcommands.
b. Subcommand Modules
– src/subcommands/{Index, Trim, Caller, Learn, Summarize, AggregateProfile, Estimate}.py
– Each implements one pipeline stage, imports reusable routines from funcs.
c. Core Function Libraries
– src/subcommands/funcs/*.py (call.py, depth.py, indels.py, learn.py, misc.py, prob.py)
– Encapsulate statistical models, file I/O, context indexing logic.
d. External Tools
– BWA-MEM (alignment)
– GATK MarkDuplicates (deduplication with optical vs. PCR UMI tags)
– Samtools (sorting, indexing)
e. Data Artifacts
– Input: FASTQ, BAM, reference.fa, germline.vcf, noise mask bed files
– Intermediate: ref.h5, tn.h5, hp.h5 (indexed references), trimmed fastqs, sorted/marked BAMs
– Output: VCFs (snv.vcf, indel.vcf), burden reports

##### Relationships & Data Flow
- CLI Dispatcher → Subcommand Module → Core Funcs → (optionally) invoke External Tool → write output
- Data flows in sequence:
1) index: reference.fa → {ref.h5, tn.h5, hp.h5}
2) trim: FASTQ → trimmed FASTQ + annotated read names
3) align: trimmed FASTQ → BAM → sorted/indexed BAM (external BWA/samtools)
4) Dup: BAM → mkdped.bam + metrics (external GATK)
5) call: mkdped.bam → snv.vcf + indel.vcf (internal routines, multi‐thread across regions)
6) summarize/aggregateProfile/learn: VCFs + error profiles → trained models + profiles
7) estimate: VCFs → mutational burden report

##### Architectural Patterns & Principles
- Command Pattern: each “subcommand” is a pluggable command class.
- Layered Design: CLI layer → command layer → library layer → external‐tool layer → storage.
- Single Responsibility: each module handles one stage or one aspect of error modeling.
- Extensibility: add new subcommands by dropping a new script under src/subcommands.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Entry
    DupCallerDispatcher["DupCaller CLI Dispatcher"]:::cli
    click DupCallerDispatcher "https://github.com/alexandrovlab/dupcaller/blob/main/src/DupCaller.py"

    %% Application Container
    subgraph "Application (Python package)"
        direction TB

        Index["Index Module"]:::subcmd
        click Index "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/Index.py"

        Trim["Trim Module"]:::subcmd
        click Trim "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/Trim.py"

        AlignEngine["Alignment Engine\n(BWA, Samtools)"]:::external

        MarkDup["MarkDuplicates Engine\n(GATK)"]:::external

        Caller["Call Module\n(multi-thread)"]:::subcmd
        click Caller "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/Caller.py"

        subgraph "Core Functions Library"
            direction TB
            callF["call.py"]:::func
            click callF "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/funcs/call.py"
            depthF["depth.py"]:::func
            click depthF "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/funcs/depth.py"
            indelF["indels.py"]:::func
            click indelF "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/funcs/indels.py"
            learnF["learn.py"]:::func
            click learnF "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/funcs/learn.py"
            miscF["misc.py"]:::func
            click miscF "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/funcs/misc.py"
            probF["prob.py"]:::func
            click probF "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/funcs/prob.py"
        end

        Summarize["Summarize Module"]:::subcmd
        click Summarize "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/Summarize.py"

        Learn["Learn Module"]:::subcmd
        click Learn "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/Learn.py"

        Aggregate["AggregateProfile Module"]:::subcmd
        click Aggregate "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/AggregateProfile.py"

        Estimate["Estimate Module"]:::subcmd
        click Estimate "https://github.com/alexandrovlab/dupcaller/blob/main/src/subcommands/Estimate.py"
    end

    %% Data Stores
    refStore["ref.h5 / tn.h5 / hp.h5"]:::data
    trimmed["trimmed.fastq"]:::data
    bam[".bam / .bai"]:::data
    mkbam["mkdped.bam / metrics"]:::data
    vcf["snv.vcf / indel.vcf"]:::data
    summaryOut["profiles"]:::data
    learnOut["trained models"]:::data
    aggOut["aggregate profiles"]:::data
    report["burden report"]:::data

    %% Package & Docs
    subgraph "Package & Documentation"
        direction TB
        Setup["setup.py"]:::pkg
        click Setup "https://github.com/alexandrovlab/dupcaller/blob/main/setup.py"
        Docker["Dockerfile"]:::pkg
        click Docker "https://github.com/alexandrovlab/dupcaller/tree/main/Dockerfile"
        Docs["README.md\n.pre-commit-config.yaml\n.prettierignore\nLICENSE"]:::pkg
    end

    %% Flows
    DupCallerDispatcher --> Index --> refStore
    refStore --> Trim --> trimmed
    trimmed --> AlignEngine --> bam
    bam --> MarkDup --> mkbam
    mkbam --> Caller --> vcf

    vcf --> Summarize --> summaryOut
    vcf --> Learn --> learnOut
    vcf --> Aggregate --> aggOut
    vcf --> Estimate --> report

    %% Module to Core Funcs dependencies
    Index --> callF
    Trim --> miscF
    Caller --> callF
    Caller --> depthF
    Caller --> indelF
    Summarize --> probF
    Learn --> learnF
    Aggregate --> miscF
    Estimate --> probF

    %% Styles
    classDef cli fill:#D0E6F8,stroke:#0366D6
    classDef subcmd fill:#BDD7EE,stroke:#254A91
    classDef external fill:#FCD5CE,stroke:#D9534F
    classDef data fill:#D5E8D4,stroke:#82B366
    classDef func fill:#E2F0D9,stroke:#6AA84F
    classDef pkg fill:#F0F0F0,stroke:#999999,stroke-dasharray: 5 5
```
