How sdparekh/zumis works
Project Type & Purpose
• zUMIs is an open-source, command-line bioinformatics pipeline for single-cell RNA-seq UMI processing. It orchestrates a sequence of filtering, demultiplexing, mapping, UMI/barcode correction, counting, and downstream export (including loom) steps via a master Bash script, multi-language helper scripts, and external tools (STAR, samtools, BBMap, velocyto).
File-Structure Analysis & Architectural Patterns
• Top-Level Orchestrator:
• zUMIs.sh (Bash) – entry point that parses zUMIs.yaml, sets up the Conda environment (-c), and dispatches jobs.
• zUMIs.yaml – user-editable config with all pipeline parameters (YAML).
• Core Modules (scripts triggered in sequence):
• Preprocessing: splitfq.sh, fqfilter_v2.pl (Perl), merge_demultiplexed_fastq.R (R), listPrefix.sh (shell)
• Demultiplex & UMI/BC handling: misc/demultiplex_BC.{pl,py}, zUMIs-BCdetection.R, correct_UBtag.{pl,py}, correct_BCtag.pl
• Mapping: zUMIs-mapping.R, zUMIs-bbmap.R orchestrate external mappers (STAR, BBMap) and samtools
• Counting & Stats: zUMIs-stats2.R, runfeatureCountFUN.R, featureCounts.R, statsFUN.R
• Post-processing & Output: zUMIs-dge2.R (matrix assembly), rds2loom.R (loom conversion), runVelocyto.R
• Support Libraries & Helpers:
• Approx.pm, distilReads.pm (Perl modules), misc/fcountsLib* (featureCounts bindings)
• checkyaml.R, readYaml4fqfilter.R for config validation/parsing
• External Tools & Dependencies: bundled Miniconda environment parts (zUMIs-miniconda.part[a-h]), STAR index usage, R, Python, Perl, BBMap, velocyto.
1) Input Layer: FastQ files, STAR genome index, GTF annotation, zUMIs.yaml
2) Orchestration Layer: zUMIs.sh + environment manager (Conda)
3) Preprocessing Module
4) Demultiplexing & UMI/BC Correction Module
5) Mapping Module
6) Counting & Statistics Module
7) Post-processing & Export Module
8) Output Layer: BAMs, count matrices (.rds/.txt), stats plots, loom files
• Input → Orchestrator → Config parsing → Preprocessing → Demultiplexing → Correction → Mapping → Counting → DGE matrix creation → Optional RNA-velocity → Loom conversion → Outputs
c. Architectural Patterns & Principles:
• Pipeline pattern (linear, modular stages)
• Modular multi-language scripts (Bash top-level, R for stats, Perl/Python for fast processing)
• Config-driven (YAML) for reproducibility & extensibility
• Parallelization at key steps (filtering, mapping, counting)
d. Key Technologies / Libraries:
• Bash, Conda, YAML, R (Rscript), Perl, Python (pysam)
• External: STAR, samtools, BBMap, velocyto, Rsubread::featureCounts, loomR