How hodson-bioinformatics/aule works
Main Components
• Pipeline Orchestration: At the center you have the Snakemake engine that ties together many modular workflows.
• Data & Input Management: Include components for raw data (FASTQ, BAM, BED) ingestion – as seen in the demo data directory and reference files—that feed into the pipelines.
• Environment & Dependencies: Represent the defined Conda/ YAML environment files that manage the dependencies (bcftools, gatk, bwa, picard, samtools, etc.).
• Scripts & Utilities: Group the assorted “lcr-scripts” which include stand-alone scripts for tasks such as munging VCF files, liftover procedures, IGV batch generation, etc.
• Documentation & Schema: Show the documentation and schema directories that define the configuration and expected formats.
Relationships and Interactions
• How the Snakemake workflows coordinate interactions between modules. For example, output from “aule_preprocessing” feeds into “aule_error,” and then into plasma or variant caller modules.
• Role of common environment files and configuration scripts that “bind” or “configure” these steps for reproducibility.
• Integration points with external reference files and tools managed via the “workflows/reference_files” directory.
Architectural Patterns and Design Principles
• Modularity: Emphasize the module-based design. Each functional step in the pipeline is broken out into submodules with versioned directories (the “1.0”, “1.1”, “1.2”, etc. versions).
• Reusability & Extensibility: Demonstrate that modules have independent schemas and configuration files that allow the system to be integrated as standalone or added-on to an existing pipeline.
• Configuration / Declarative Workflow: Note the use of YAML-based configuration and environment files and how these contribute to reproducibility.
• Workflow Orchestration: Indicate that Snakemake acts as the “conductor” of the overall pipeline, scheduling tasks and managing dependencies between steps.
Relevant Technologies, Frameworks, and Libraries
• Snakemake – as the primary orchestrator.
• Conda environments configured via YAML files for managing tool dependencies (e.g., gatk, samtools, bcftools).
• Scripting languages such as Python (for custom scripts) and R (for statistical and data formatting tasks).
• Version-control with Git (as evident from .gitignore and repo structure).