How xinglab/rmats-turbo works
rMATS turbo is a command-line RNA-seq alternative splicing analysis pipeline. The repo combines Python orchestration, a C/Cython counting engine, C/Fortran statistical code, and optional R-based models, packaged for local execution and workflow engines.
1. Entry points and orchestration
- rmats.py is the main CLI.
- run_rmats wraps it with the built environment.
- build_rmats prepares the toolchain and optional conda/R dependencies.
- The main execution modes are prep, post, both, inte, and stat, which split preprocessing, aggregation, and statistics into separable stages.
2. Input boundary
- The pipeline accepts either aligned BAMs (--b1/--b2) or raw FASTQ pairs (--s1/--s2).
- FASTQ input is mapped first using STAR binary indices (--bi), then converted into BAMs before analysis.
- Sample grouping is passed via simple text files, which are the main contract between user input and the pipeline.
3. Two-phase processing model
- Prep phase: processes each BAM independently and writes per-BAM .rmats summaries into --tmp.
- Post phase: reads all .rmats files, merges them, detects/counts splicing events, and produces final outputs in --od.
- This split is the key scale-out boundary: prep can run on multiple machines, while post needs the full intermediate set.
4. Core counting engine
- The performance-critical logic lives in rMATS_pipeline/rmatspipeline.pyx and related Cython declarations.
- This layer interfaces with compiled code for read traversal, splice junction counting, and event bookkeeping.
- It is the main reason the project is much faster than the older Python implementation.
5. BAM and SAM infrastructure
- The repo vendors bamtools, which provides low-level BAM/SAM reading, writing, indexing, and random access.
- Internal bamtools/src/api/internal/... code handles file/device access, BGZF streams, HTTP/FTP/local I/O abstractions, and BAM indexing.
- This is the storage-access boundary for all alignment data.
6. Splicing event model
- The analysis is centered on five event types: SE, MXE, A5SS, A3SS, and RI.
- The pipeline first derives event definitions from the GTF (fromGTF.*.txt) and then counts evidence for each event.
- Output is produced in two counting regimes: JC (junction counts only) and JCEC (junction plus exon-body counts).
7. Statistical analysis layer
- The default differential splicing model is implemented in rMATS_C with C code and a Fortran L-BFGS/LAPACK-style optimization dependency chain.
- --paired-stats switches to an R-based paired model that integrates with PAIRADISE.
- --darts-model enables an alternative R-based DARTS model.
- This is a clear separation between counting and inference.
8. Intermediate and final data products
- Prep writes .rmats files and read-outcome summaries into --tmp.
- Post writes event files, raw count tables, p-values/FDR, inclusion levels, and summary.txt into --od.
- The final outputs are structured per event type and per counting method, which is important for any diagram of downstream consumers.
9. Python utilities around the core
- rMATS_P contains helper scripts for postprocessing and multi-group workflows, such as prepare_stat_inputs.py, summary.py, and FDR.py.
- utils/ contains parsing, filtering, FASTA, pileup, and CLI option helpers.
- These modules are supporting services around the core engine rather than the main compute path.
10. Workflow packaging and deployment
- The repo includes Nextflow (main.nf, nextflow/), WDL (wdl/rmatsTurbo.wdl), and CWL (cwl/rMATS_bam_packed.cwl) wrappers.
- Dockerfile, .dockstore.yml, and conda requirement files indicate containerized and workflow-manager deployment as first-class use cases.
- This matters architecturally because the tool is designed to run both locally and in batch/workflow environments.
11. Testing and validation
- tests/ contains scenario-based integration tests for many splicing cases and edge conditions.
- test_rmats is the test runner entry point.
- The tests validate end-to-end behavior across input parsing, counting, statistical output, and edge cases like strandedness and variable read length.
12. Practical architectural summary
- The repository is organized around a pipeline: input grouping → optional alignment → per-BAM preprocessing → merged event counting → statistical inference → text outputs.
- The major boundaries are Python orchestration vs compiled counting/statistics, and per-sample prep vs global post/stat.
- External dependencies that materially shape the architecture are STAR, PAIRADISE, DARTS, and the C/Fortran scientific stack used by the statistical core.
Main entry points: rmatspipeline.pyx Post stage · rmats.py rmats.py · main.c rMATS_C · runner.py Tests
Components 18
Orchestration
CLI entrypoints, staged execution, and packaging wrappers.
- rmats.py · Main command-line interface and mode dispatcher.
rmats.py - run_rmats · Launches rMATS in the prepared runtime environment.
run_rmats - build_rmats · Prepares toolchain and optional dependencies.
build_rmats
Core analysis
Per-sample preprocessing, event counting, and statistical inference.
- Prep stage · Per-BAM preprocessing that produces intermediate summaries.
rMATS_pipeline/rmatspipeline/rmatspipeline.pyx - Post stage · Merges intermediates, counts events, and writes final outputs.
rMATS_pipeline/rmatspipeline/rmatspipeline.pyx - rMATS pipeline · Performance-critical read traversal, junction counting, and event bookkeeping.
rMATS_pipeline/rmatspipeline/rmatspipeline.pyx - rMATS_C · Compiled statistical inference core for differential splicing.
rMATS_C/src/main.c - L-BFGS · Fortran/C optimization backend used by the statistical core.
rMATS_C/lbfgs_scipy/lbfgsb.f - PAIRED model · Optional paired statistical model integration.
rMATS_R/paired_model.R - DARTS model · Optional alternative statistical model integration.
rMATS_R/darts_model.R - rMATS_P · Helpers for statistical inputs, summaries, and FDR handling.
rMATS_P/prepare_stat_inputs.py - utils · Parsing, filtering, FASTA, pileup, and CLI helper code.
bamtools/src/utils/bamtools_utilities.cpp
BAM access
Vendored BAM/SAM I/O and indexing library.
- BAM tools · Vendored low-level BAM/SAM access, indexing, and random access.
bamtools/src/api/BamReader.cpp - I/O internals · Local, remote, and BGZF stream abstractions for BAM devices.
bamtools/src/api/internal/io/BgzfStream_p.cpp - Indexing · BAM index creation and lookup for random access.
bamtools/src/api/internal/index/BamStandardIndex_p.cpp
Workflow packaging
Workflow and container entrypoints for external runtimes.
Validation
Scenario-based end-to-end tests.
- Tests · End-to-end scenario coverage for splicing cases and edge conditions.
tests/runner.py
Connections 20
- run_rmats → rmats.py: wraps · Runs the CLI through the prepared environment.
- build_rmats → run_rmats: prepares · Builds dependencies and runtime prerequisites for execution.
- rmats.py → Prep stage: dispatches prep · CLI mode routes work into the preprocessing phase.
- rmats.py → Post stage: dispatches post · CLI mode routes work into the merge-and-statistics phase.
- Prep stage → rMATS pipeline: uses · Prep phase relies on the counting engine.
- Post stage → rMATS pipeline: uses · Post phase relies on the counting engine.
- rMATS pipeline → BAM tools: reads alignments · Core engine traverses BAM/SAM records through the vendored library.
- BAM tools → I/O internals: delegates I/O · Library uses internal device and stream abstractions.
- BAM tools → Indexing: random access · Library relies on index support for region queries.
- Post stage → rMATS_C: feeds counts · Merged event counts flow into statistical inference.
- rMATS_C → L-BFGS: optimizes with · C statistical core links to the L-BFGS/Fortran solver stack.
- rMATS_C → PAIRED model: optional paired · Paired inference can switch to the R model.
- rMATS_C → DARTS model: optional darts · Alternative inference can switch to the DARTS model.
- Post stage → rMATS_P: postprocesses · Utilities assemble statistical inputs and summarize outputs.
- Post stage → utils: supports · Shared helpers support parsing and data preparation.
- Nextflow → rmats.py: invokes · Nextflow wrapper drives the main CLI.
- WDL/CWL → rmats.py: invokes · WDL/CWL wrappers execute the same core CLI.
- Tests → rmats.py: validates · Integration tests exercise the full pipeline.
- Tests → Prep stage: covers · Tests verify preprocessing behavior.
- Tests → Post stage: covers · Tests verify postprocessing and outputs.