# wholegenomesequencinganalysispipeline/bacpipe architecture

> Architecture diagram of the GitHub repository wholegenomesequencinganalysispipeline/bacpipe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wholegenomesequencinganalysispipeline/bacpipe
- Repository: https://github.com/wholegenomesequencinganalysispipeline/bacpipe

## Overview

##### Project Type and Purpose
- BacPipe is a monolithic, command-line/GUI bioinformatics pipeline for bacterial whole-genome sequencing analysis.
- Written in Python 2.7, with a GUI front-end (appJar) and a CLI mode (Pipeline_cmd.py).
- It orchestrates a suite of third-party tools (written in Perl, C/C++, etc.) to perform read trimming, assembly, typing, annotation, resistance/virulence gene detection, outbreak analysis, and report generation.

##### High-Level Components
- User Interface
– GUI: appJar running inside Pipeline.py
– CLI wrapper: Pipeline_cmd.py
- Pipeline Controller
– Pipeline.py reads configure.yaml, manages workflow graph, spawns tasks concurrently, updates progress tab, writes logs.
- Pre-Assembly Module
– TrimGalore (cutadapt-1.12) for quality trimming
- Assembly Module
– SPAdes (SPAdes-3.13.0-Linux) for de-novo assembly and scaffolding
- Post-Assembly Typing / Detection Modules
– MLST (mlst/)
– emm typing (emm_typing/)
– PlasmidFinder (plasmidfinder/)
– ResFinder (resfinder/)
– VirulenceFinder (virulencefinder/)
– ParSNP (parsnp_Linux64_v1_2/) for SNP-based outbreak analysis
- Annotation Module
– Prokka (external requirement) invoked via Main.command or within Pipeline
- Post-Annotation Search Modules
– CARD (cardSearch/)
– VirDB (VirDB/)
– ResFams (ResFam/)
- Data & Logging
– Input: raw reads (*.fastq.gz) or contigs (*.fasta) in mapped host volume
– Intermediates: trimmed reads, assemblies, annotated proteins
– Outputs:
· Overall summary dashboard and Newick tree file
· Per-sample Excel summaries (in Summary folder)
· Detailed per-tool per-sample folders
– Log files: pipeline.log (step-wise), log.txt (tool stdout/err)
- Deployment
– Docker container (mahmed/bacpipe) with X11 forwarding for GUI
– Local install via .run installers
- License & Third-Party Software
– Numerous bundled executables with their own licenses (BLAST, HMMER, cutadapt, etc.)
– License files under LICENSES/ and within each tool’s folder

##### Relationships and Data Flow
1) User invokes Pipeline.py (GUI) or Pipeline_cmd.py (CLI).
2) Pipeline controller parses configure.yaml, builds task DAG.
3) Raw reads → TrimGalore → trimmed reads.
4) Trimmed reads → SPAdes → contigs.fasta.
5) contigs.fasta → parallel execution of typing/detection modules (MLST, emm, plasmidfinder, resfinder, virulencefinder, Parsnp).
6) contigs.fasta → Prokka → annotated proteins/*.faa.
7) *.faa → parallel CARD, VirDB, ResFams searches.
8) Results → aggregator → Excel sheets + HTML dashboard + Newick tree.
9) Pipeline writes logs and signals completion to GUI.

##### Architectural Patterns & Principles
- Modular “step” abstraction: each tool wrapped as an independent task
- Parallel execution: non-conflicting steps (e.g., post-assembly detectors) run concurrently
- Data-flow pipeline: file system as “message bus” between steps
- Separation of concerns: UI layer (appJar), orchestration layer (Pipeline.py), tool wrappers (scripts in each folder)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Docker Container" 
        style DockerContainer stroke-dasharray: 5 5
        DockerContainer

        subgraph "User Interface" 
            GUI["GUI (appJar)"]:::ui
            CLI["CLI (Pipeline_cmd.py)"]:::ui
        end

        subgraph "Orchestrator" 
            Controller["Pipeline.py"]:::orchestrator
            Config["configure.yaml"]:::orchestrator
            Logger["pipeline.log / log.txt"]:::orchestrator
        end

        subgraph "Data Stores"
            Input["Raw Reads (*.fastq.gz) / Contigs (*.fasta)"]:::data
            Interm["Intermediates"]:::data
            Output["Outputs (Excel, HTML, Newick)"]:::data
        end

        subgraph "Toolbox"
            subgraph "Pre-Assembly" 
                TrimGalore["TrimGalore"]:::pre
                Cutadapt["Cutadapt (1.12)"]:::pre
            end
            subgraph "Assembly"
                SPAdes["SPAdes"]:::assembly
            end
            subgraph "Post-Assembly Typing/Detection"
                MLST["MLST"]:::post_assembly
                emm["emm Typing"]:::post_assembly
                Plasmid["PlasmidFinder"]:::post_assembly
                ResFinder["ResFinder"]:::post_assembly
                Virulence["VirulenceFinder"]:::post_assembly
                Parsnp["Parsnp"]:::post_assembly
            end
            subgraph "Annotation"
                Prokka["Prokka (via Main.command)"]:::annotation
            end
            subgraph "Post-Annotation Search"
                CARD["CARD Search"]:::post_annotation
                VirDB["VirDB Search"]:::post_annotation
                ResFams["ResFams Search"]:::post_annotation
            end
        end

        subgraph "Reporting"
            Aggregator["Results Aggregator"]:::result
            ReportScripts["Output_batch.pl<br/>Output_organizer.pl<br/>Newick_visualization.pl"]:::result
        end

        DockerContainer --> GUI
        DockerContainer --> CLI
        GUI --> Controller
        CLI --> Controller
        Config --> Controller
        Controller --> Input
        Input --> TrimGalore
        TrimGalore --> Cutadapt
        Cutadapt --> SPAdes
        SPAdes --> Interm
        Interm --> MLST
        Interm --> emm
        Interm --> Plasmid
        Interm --> ResFinder
        Interm --> Virulence
        Interm --> Parsnp
        SPAdes --> Prokka
        Prokka --> Interm
        Interm --> CARD
        Interm --> VirDB
        Interm --> ResFams
        CARD --> Aggregator
        VirDB --> Aggregator
        ResFams --> Aggregator
        MLST --> Aggregator
        emm --> Aggregator
        Plasmid --> Aggregator
        ResFinder --> Aggregator
        Virulence --> Aggregator
        Parsnp --> Aggregator
        Aggregator --> Output
        Controller --> Logger
        Controller --> GUI

    end

    click GUI "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/appJar/"
    click CLI "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/Pipeline_cmd.py"
    click Controller "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/Pipeline.py"
    click Config "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/configure.yaml"
    click TrimGalore "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/trim_galore_v0.4.2/trim_galore"
    click Cutadapt "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/cutadapt-1.12/"
    click SPAdes "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/SPAdes-3.13.0-Linux/"
    click MLST "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/mlst/"
    click emm "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/emm_typing/"
    click Plasmid "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/plasmidfinder/"
    click ResFinder "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/resfinder/"
    click Virulence "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/virulencefinder/"
    click Parsnp "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/parsnp_Linux64_v1_2/parsnp"
    click Prokka "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/Main.command"
    click CARD "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/cardSearch/Blast_card.pl"
    click VirDB "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/VirDB/Blast_VirDB.pl"
    click ResFams "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/ResFam/ResFam_Batch.pl"
    click DockerContainer "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/Dockerfile"
    click LICENSES "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/tree/master/LICENSES/"
    click ReportScripts "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/Output_batch.pl"
    click ReportScripts "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/Output_organizer.pl"
    click ReportScripts "https://github.com/wholegenomesequencinganalysispipeline/bacpipe/blob/master/Newick_visualization.pl"

    classDef ui fill:#CFF,stroke:#000
    classDef orchestrator fill:#FFC,stroke:#000
    classDef data fill:#EEE,stroke:#000,stroke-dasharray: 5 2
    classDef pre fill:#BBE,stroke:#000
    classDef assembly fill:#BFB,stroke:#000
    classDef post_assembly fill:#FBC,stroke:#000
    classDef annotation fill:#DDB,stroke:#000
    classDef post_annotation fill:#F88,stroke:#000
    classDef result fill:#CCC,stroke:#000
    classDef DockerContainer fill:none,stroke:#999,stroke-dasharray: 5 5
```
