How wholegenomesequencinganalysispipeline/bacpipe works
Project Type and Purpose
• BacPipe is a monolithic, command-line/GUI bioinformatics pipeline for bacterial whole-genome sequencing analysis.
• Written in Python 2.7, with a GUI front-end (appJar) and a CLI mode (Pipeline_cmd.py).
• It orchestrates a suite of third-party tools (written in Perl, C/C++, etc.) to perform read trimming, assembly, typing, annotation, resistance/virulence gene detection, outbreak analysis, and report generation.
High-Level Components
• User Interface
– GUI: appJar running inside Pipeline.py
– CLI wrapper: Pipeline_cmd.py
• Pipeline Controller
– Pipeline.py reads configure.yaml, manages workflow graph, spawns tasks concurrently, updates progress tab, writes logs.
• Pre-Assembly Module
– TrimGalore (cutadapt-1.12) for quality trimming
• Assembly Module
– SPAdes (SPAdes-3.13.0-Linux) for de-novo assembly and scaffolding
• Post-Assembly Typing / Detection Modules
– MLST (mlst/)
– emm typing (emm_typing/)
– PlasmidFinder (plasmidfinder/)
– ResFinder (resfinder/)
– VirulenceFinder (virulencefinder/)
– ParSNP (parsnp_Linux64_v1_2/) for SNP-based outbreak analysis
• Annotation Module
– Prokka (external requirement) invoked via Main.command or within Pipeline
• Post-Annotation Search Modules
– CARD (cardSearch/)
– VirDB (VirDB/)
– ResFams (ResFam/)
• Data & Logging
– Input: raw reads (*.fastq.gz) or contigs (*.fasta) in mapped host volume
– Intermediates: trimmed reads, assemblies, annotated proteins
– Outputs:
· Overall summary dashboard and Newick tree file
· Per-sample Excel summaries (in Summary folder)
· Detailed per-tool per-sample folders
– Log files: pipeline.log (step-wise), log.txt (tool stdout/err)
• Deployment
– Docker container (mahmed/bacpipe) with X11 forwarding for GUI
– Local install via .run installers
• License & Third-Party Software
– Numerous bundled executables with their own licenses (BLAST, HMMER, cutadapt, etc.)
– License files under LICENSES/ and within each tool’s folder
Relationships and Data Flow
1) User invokes Pipeline.py (GUI) or Pipeline_cmd.py (CLI).
2) Pipeline controller parses configure.yaml, builds task DAG.
3) Raw reads → TrimGalore → trimmed reads.
4) Trimmed reads → SPAdes → contigs.fasta.
5) contigs.fasta → parallel execution of typing/detection modules (MLST, emm, plasmidfinder, resfinder, virulencefinder, Parsnp).
6) contigs.fasta → Prokka → annotated proteins/*.faa.
7) *.faa → parallel CARD, VirDB, ResFams searches.
8) Results → aggregator → Excel sheets + HTML dashboard + Newick tree.
9) Pipeline writes logs and signals completion to GUI.
Architectural Patterns & Principles
• Modular “step” abstraction: each tool wrapped as an independent task
• Parallel execution: non-conflicting steps (e.g., post-assembly detectors) run concurrently
• Data-flow pipeline: file system as “message bus” between steps
• Separation of concerns: UI layer (appJar), orchestration layer (Pipeline.py), tool wrappers (scripts in each folder)