How dbespiatykh/rdscan works
Project Overview and Main Components
• Project Type & Purpose: RDscan is a bioinformatics pipeline that uses Snakemake to analyze sequencing data for detecting genomic deletions (regions of difference) in Mycobacterium tuberculosis complex. Its primary purpose is to automate the analysis workflow – from raw sequencing inputs through mapping, discovery, and result reporting.
• Main Components:
• Input Data: Raw reads (FASTQ files under the .test/reads directory) and sample configuration files (.test/config/samples.tsv and config samples.tsv).
• Configuration Management: “config” directory (with its README and config.yml) and additional test configuration under “.test/config”. These define thresholds (e.g., for conditional formatting in RD_known.xlsx) and pipeline parameters.
• Scripts & Tools: Custom scripts are located in “workflow/scripts” (such as annotate_novel.py, concatenate_bed.py, makeTables.R, proportions.py) handling specific analyses or formatting tasks.
• Environment & Dependencies: The “environment.yml” and the files under “workflow/envs” specify conda-based environments and dependencies ensuring reproducible computational environments.
• External Resources: The “resources” directory contains reference data (like IS6110.bed, RD.bed, RvD1_TbD1.fa) needed for analysis.
• Testing & CI: The “.github” and “.test” directories indicate continuous integration workflows (GitHub Actions) and testing datasets to validate pipeline performance.
Relationships and Data Flow
• The configuration files (from both “config” and “.test/config”) feed into the Snakemake workflow to specify parameters for each rule.
• The Snakefile orchestrates multiple rule files (mapping, novel discovery, etc.) that process the raw data. These rules call custom scripts for tasks like annotation and generating summary tables.
• Throughout the workflow, external resources and reference files are accessed, and the environment is managed by conda (ensuring all dependencies are available).
• Finally, the output is generated (results tables such as RD_putative.tsv, RD_known.tsv, etc.) and possibly transformed (e.g., conditional formatting in the XLSX file as shown in the README).
Architectural Patterns and Design Principles
• Modular Design: The project is clearly organized into modular components (configuration, workflow rules, scripts, and resources) making it easier to extend or modify parts of the workflow.
• Separation of Concerns: Input data handling, processing logic, and output generation are distinct. The rules in the Snakemake workflow specifically focus on different analysis steps.
• Reproducibility: Use of conda environments (detailed in environment.yml and various env YAML files) ensures that the system is reproducible across environments.
• Workflow Orchestration: The use of Snakemake indicates a dataflow paradigm with clear dependencies between steps managed by the Snakefile.