# hodson-bioinformatics/aule architecture

> Architecture diagram of the GitHub repository hodson-bioinformatics/aule, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hodson-bioinformatics/aule
- Repository: https://github.com/hodson-bioinformatics/aule

## Overview

##### Main Components
- Pipeline Orchestration: At the center you have the Snakemake engine that ties together many modular workflows.
- Data & Input Management: Include components for raw data (FASTQ, BAM, BED) ingestion – as seen in the demo data directory and reference files—that feed into the pipelines.
- Environment & Dependencies: Represent the defined Conda/ YAML environment files that manage the dependencies (bcftools, gatk, bwa, picard, samtools, etc.).
- Scripts & Utilities: Group the assorted “lcr-scripts” which include stand-alone scripts for tasks such as munging VCF files, liftover procedures, IGV batch generation, etc.
- Documentation & Schema: Show the documentation and schema directories that define the configuration and expected formats.

##### Relationships and Interactions
- How the Snakemake workflows coordinate interactions between modules. For example, output from “aule_preprocessing” feeds into “aule_error,” and then into plasma or variant caller modules.
- Role of common environment files and configuration scripts that “bind” or “configure” these steps for reproducibility.
- Integration points with external reference files and tools managed via the “workflows/reference_files” directory.

##### Architectural Patterns and Design Principles
- Modularity: Emphasize the module-based design. Each functional step in the pipeline is broken out into submodules with versioned directories (the “1.0”, “1.1”, “1.2”, etc. versions).
- Reusability & Extensibility: Demonstrate that modules have independent schemas and configuration files that allow the system to be integrated as standalone or added-on to an existing pipeline.
- Configuration / Declarative Workflow: Note the use of YAML-based configuration and environment files and how these contribute to reproducibility.
- Workflow Orchestration: Indicate that Snakemake acts as the “conductor” of the overall pipeline, scheduling tasks and managing dependencies between steps.

##### Relevant Technologies, Frameworks, and Libraries
- Snakemake – as the primary orchestrator.
- Conda environments configured via YAML files for managing tool dependencies (e.g., gatk, samtools, bcftools).
- Scripting languages such as Python (for custom scripts) and R (for statistical and data formatting tasks).
- Version-control with Git (as evident from .gitignore and repo structure).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Input Management
    subgraph "Data & Input Management"
        DI["Input Data (FASTQ, BAM, BED)"]:::data
        RF["Reference Files"]:::data
    end

    %% Pipeline Orchestration (Workflow Engine)
    WE["Workflow Engine (Snakemake)"]:::orchestrator

    %% Preprocessing Phase
    subgraph "Preprocessing Phase"
        AP["Preprocessing Module (aule_preprocessing)"]:::module
        AE["Error Correction (aule_error)"]:::module
        AP --> AE
    end

    %% Post Error Branches: Plasma Analysis and Variant Analysis
    subgraph "Plasma Analysis Branch"
        PP["Plasma Analysis (aule_plasma)"]:::module
        OT["Output Tables (aule_out_tables)"]:::module
        PP --> OT
    end

    subgraph "Variant Analysis Branch"
        PV["Phased Variants (aule_pv)"]:::module
        PON["Panel-of-Normals (aule_pv_pon)"]:::module
        SNV["SNV Panel-of-Normals (aule_snv_pon)"]:::module
        UC["UMI Collapse (aule_umi_collapse)"]:::module
        GR["GRIDSS Variant Caller (gridss)"]:::module
        IC["IchorCNA (ichorcna)"]:::module
        ICO["IchorCNA PON (ichorcna_pon)"]:::module
        LF["Liftover Utility (liftover)"]:::module
        SF["STARFISH (starfish)"]:::module
        MT["Variant Caller (Mutect2)"]:::module
        MA["Structural Variant Caller (Manta)"]:::module
        SM["SV Merge & Annotation (sv_merge)"]:::module
        VF["VCF to MAF Conversion (vcf2maf)"]:::module

        PV --> PON
        PON --> SNV
        SNV --> UC
        UC --> GR
        GR --> IC
        IC --> ICO
        ICO --> LF
        LF --> SF
        SF --> MT
        MT --> MA
        MA --> SM
        SM --> VF
    end

    %% Final Results
    FR["Final Results / QC & Reporting"]:::module

    %% Supporting Components
    ENV["Environment & Dependencies (Conda/YAML)"]:::env
    UTIL["Scripts & Utilities"]:::utility
    DOC["Documentation & Schemas"]:::docs

    %% Data Flow Connections
    DI -->|"feeds"| WE
    RF -->|"provides"| WE
    WE -->|"orchestrates"| AP
    AE -->|"branches to"| PP
    AE -->|"branches to"| PV
    OT --> FR
    VF --> FR
    PP --> FR

    %% Supporting Connections (dashed lines for support)
    WE --- ENV
    WE --- UTIL
    WE --- DOC

    %% Click Events
    click DI "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/demo/data"
    click RF "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/workflows/reference_files"
    click WE "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/demo"
    click AP "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/aule_preprocessing"
    click AE "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/aule_error"
    click PP "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/aule_plasma"
    click OT "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/aule_out_tables"
    click PV "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/aule_pv"
    click PON "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/aule_pv_pon"
    click SNV "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/aule_snv_pon"
    click UC "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/aule_umi_collapse"
    click GR "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/gridss"
    click IC "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/ichorcna"
    click ICO "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/ichorcna_pon"
    click LF "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/liftover"
    click SF "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/starfish"
    click MT "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/mutect2"
    click MA "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/manta"
    click SM "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/sv_merge"
    click VF "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/modules/vcf2maf"
    click ENV "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/envs"
    click UTIL "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-scripts"
    click DOC "https://github.com/hodson-bioinformatics/aule/tree/main/lcr-modules/docs"

    %% Styles
    classDef data fill:#AEDFF7,stroke:#333,stroke-width:2px;
    classDef orchestrator fill:#C3FDB8,stroke:#333,stroke-width:2px;
    classDef module fill:#FFF79A,stroke:#333,stroke-width:2px;
    classDef env fill:#D2B4DE,stroke:#333,stroke-width:2px;
    classDef utility fill:#FFB3BA,stroke:#333,stroke-width:2px;
    classDef docs fill:#D4A5A5,stroke:#333,stroke-width:2px;
```
