# oskarvid/selma architecture

> Architecture diagram of the GitHub repository oskarvid/selma, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oskarvid/selma
- Repository: https://github.com/oskarvid/selma

## Overview

##### Main Components

a. Input Layer:
- FASTQ files input directory
- Reference genome files
- Configuration files (config.yaml, samples.tsv)

b. Workflow Management Layer:
- Snakemake workflow engine
- Environment management systems (Conda/Docker/Guix)

c. Processing Pipeline Components:
- Alignment Module (BWA)
- BAM Processing Module (Samtools)
- Quality Control Module (GATK4 tools)
- Variant Calling Module (GATK4 HaplotypeCaller)
- Variant Processing Module (GATK4 GenotypeGVCFs, VariantRecalibrator)

**[Input Layer]**
- ↓
**[Environment Management]**
- ↓
**[Workflow Engine]**
- ↓
**[Processing Pipeline]**

##### Specific Elements

a. Input Section:

b. Environment Section:
- Docker container
- Conda environment
- Guix shell

c. Workflow Engine Section:
- Snakemake as the central orchestrator
- Scatter-gather parallelization mechanism

d. Processing Pipeline Section:
- Create sequential blocks for each major processing step:
1. Read Mapping (BWA)
2. BAM Processing (Samtools)
3. Duplicate Marking (GATK4)
4. Base Recalibration (GATK4)
5. Variant Calling (HaplotypeCaller)
6. Variant Genotyping (GenotypeGVCFs)
7. Variant Recalibration (VQSR)

##### Important Details to Highlight

- Blue: Input/Output data
- Green: Processing tools
- Yellow: Configuration and environment
- Red: Critical processing steps

- Parallel processing capabilities:
- Where scatter-gather operations occur
- Thread distribution

- Resource requirements:
- Memory requirements (64GB RAM recommended)
- Storage requirements (500GB minimum)
- CPU requirements (16 threads optimal)

##### Additional Elements

- Version numbers for key tools
- Temporary file storage locations
- Optimization points for resource usage

##### Performance Characteristics

A small section showing:
- Typical runtime (16-18 hours for 30x coverage)
- Resource scaling information
- Bottleneck points

This system design should effectively communicate both the high-level architecture and the detailed workflow of the Selma pipeline, making it valuable for both new team members and experienced developers.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Input Layer
    subgraph Input
        FASTQ["FASTQ Files"]:::input
        REF["Reference Genomes"]:::input
        CONFIG["Configuration Files"]:::input
    end

    %% Environment Management Layer
    subgraph Environment
        DOCKER["Docker Container"]:::env
        CONDA["Conda Environment"]:::env
        GUIX["Guix Shell"]:::env
    end

    %% Workflow Management Layer
    subgraph Workflow
        SNAKE["Snakemake Engine"]:::workflow
        SETUP["Setup Scripts"]:::workflow
    end

    %% Processing Pipeline Layer
    subgraph Pipeline
        direction TB
        BWA["BWA Alignment"]:::process
        SAMTOOLS["Samtools Processing"]:::process
        GATK_DUP["GATK4 Duplicate Marking"]:::process
        GATK_BQSR["GATK4 Base Recalibration"]:::process
        GATK_HC["GATK4 HaplotypeCaller"]:::process
        GATK_GENO["GATK4 GenotypeGVCFs"]:::process
        GATK_VQSR["GATK4 Variant Recalibration"]:::process
    end

    %% Output Layer
    VCF["Final VCF Output"]:::output

    %% Connections
    FASTQ --> SNAKE
    REF --> SNAKE
    CONFIG --> SNAKE
    
    DOCKER --> SNAKE
    CONDA --> SNAKE
    GUIX --> SNAKE
    
    SNAKE --> SETUP
    SETUP --> BWA
    BWA --> SAMTOOLS
    SAMTOOLS --> GATK_DUP
    GATK_DUP --> GATK_BQSR
    GATK_BQSR --> GATK_HC
    GATK_HC --> GATK_GENO
    GATK_GENO --> GATK_VQSR
    GATK_VQSR --> VCF

    %% Resource Requirements Note
    classDef requirements fill:#f9f,stroke:#333,stroke-width:2px
    RESOURCES["System Requirements<br/>RAM: 64GB<br/>Storage: 500GB<br/>CPU: 16 threads"]:::requirements

    %% Styling
    classDef input fill:#a8d1ff,stroke:#333,stroke-width:2px
    classDef env fill:#ffe0a8,stroke:#333,stroke-width:2px
    classDef workflow fill:#a8ffa8,stroke:#333,stroke-width:2px
    classDef process fill:#ffa8a8,stroke:#333,stroke-width:2px
    classDef output fill:#d1ffa8,stroke:#333,stroke-width:2px

    %% Click Events
    click FASTQ "https://github.com/oskarvid/selma/tree/master/workspace/fastq/"
    click CONFIG "https://github.com/oskarvid/selma/blob/master/workspace/config.yaml"
    click REF "https://github.com/oskarvid/selma/tree/master/utilities/b37-reference-dl-list"
    click SNAKE "https://github.com/oskarvid/selma/tree/master/Snakefile"
    click DOCKER "https://github.com/oskarvid/selma/tree/master/Dockerfile"
    click GUIX "https://github.com/oskarvid/selma/blob/master/manifest.scm"
    click CONDA "https://github.com/oskarvid/selma/blob/master/conda/env.yaml"
    click BWA "https://github.com/oskarvid/selma/blob/master/conda/bwa.yaml"
    click SAMTOOLS "https://github.com/oskarvid/selma/blob/master/conda/samtools.yaml"
    click SETUP "https://github.com/oskarvid/selma/blob/master/utilities/Selma-setup.sh"

    %% Legend
    subgraph Legend
        L1["Input Files"]:::input
        L2["Environment"]:::env
        L3["Workflow"]:::workflow
        L4["Processing"]:::process
        L5["Output"]:::output
    end
```
