# xinglab/rmats-turbo architecture

> Architecture diagram of the GitHub repository xinglab/rmats-turbo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xinglab/rmats-turbo
- Repository: https://github.com/xinglab/rmats-turbo

## Overview

rMATS turbo is a command-line RNA-seq alternative splicing analysis pipeline. The repo combines Python orchestration, a C/Cython counting engine, C/Fortran statistical code, and optional R-based models, packaged for local execution and workflow engines.

1. **Entry points and orchestration**
   - `rmats.py` is the main CLI.
   - `run_rmats` wraps it with the built environment.
   - `build_rmats` prepares the toolchain and optional conda/R dependencies.
   - The main execution modes are `prep`, `post`, `both`, `inte`, and `stat`, which split preprocessing, aggregation, and statistics into separable stages.

2. **Input boundary**
   - The pipeline accepts either aligned BAMs (`--b1/--b2`) or raw FASTQ pairs (`--s1/--s2`).
   - FASTQ input is mapped first using STAR binary indices (`--bi`), then converted into BAMs before analysis.
   - Sample grouping is passed via simple text files, which are the main contract between user input and the pipeline.

3. **Two-phase processing model**
   - **Prep phase:** processes each BAM independently and writes per-BAM `.rmats` summaries into `--tmp`.
   - **Post phase:** reads all `.rmats` files, merges them, detects/counts splicing events, and produces final outputs in `--od`.
   - This split is the key scale-out boundary: prep can run on multiple machines, while post needs the full intermediate set.

4. **Core counting engine**
   - The performance-critical logic lives in `rMATS_pipeline/rmatspipeline.pyx` and related Cython declarations.
   - This layer interfaces with compiled code for read traversal, splice junction counting, and event bookkeeping.
   - It is the main reason the project is much faster than the older Python implementation.

5. **BAM and SAM infrastructure**
   - The repo vendors `bamtools`, which provides low-level BAM/SAM reading, writing, indexing, and random access.
   - Internal `bamtools/src/api/internal/...` code handles file/device access, BGZF streams, HTTP/FTP/local I/O abstractions, and BAM indexing.
   - This is the storage-access boundary for all alignment data.

6. **Splicing event model**
   - The analysis is centered on five event types: SE, MXE, A5SS, A3SS, and RI.
   - The pipeline first derives event definitions from the GTF (`fromGTF.*.txt`) and then counts evidence for each event.
   - Output is produced in two counting regimes: JC (junction counts only) and JCEC (junction plus exon-body counts).

7. **Statistical analysis layer**
   - The default differential splicing model is implemented in `rMATS_C` with C code and a Fortran L-BFGS/LAPACK-style optimization dependency chain.
   - `--paired-stats` switches to an R-based paired model that integrates with PAIRADISE.
   - `--darts-model` enables an alternative R-based DARTS model.
   - This is a clear separation between counting and inference.

8. **Intermediate and final data products**
   - Prep writes `.rmats` files and read-outcome summaries into `--tmp`.
   - Post writes event files, raw count tables, p-values/FDR, inclusion levels, and `summary.txt` into `--od`.
   - The final outputs are structured per event type and per counting method, which is important for any diagram of downstream consumers.

9. **Python utilities around the core**
   - `rMATS_P` contains helper scripts for postprocessing and multi-group workflows, such as `prepare_stat_inputs.py`, `summary.py`, and `FDR.py`.
   - `utils/` contains parsing, filtering, FASTA, pileup, and CLI option helpers.
   - These modules are supporting services around the core engine rather than the main compute path.

10. **Workflow packaging and deployment**
   - The repo includes Nextflow (`main.nf`, `nextflow/`), WDL (`wdl/rmatsTurbo.wdl`), and CWL (`cwl/rMATS_bam_packed.cwl`) wrappers.
   - `Dockerfile`, `.dockstore.yml`, and conda requirement files indicate containerized and workflow-manager deployment as first-class use cases.
   - This matters architecturally because the tool is designed to run both locally and in batch/workflow environments.

11. **Testing and validation**
   - `tests/` contains scenario-based integration tests for many splicing cases and edge conditions.
   - `test_rmats` is the test runner entry point.
   - The tests validate end-to-end behavior across input parsing, counting, statistical output, and edge cases like strandedness and variable read length.

12. **Practical architectural summary**
   - The repository is organized around a pipeline: input grouping → optional alignment → per-BAM preprocessing → merged event counting → statistical inference → text outputs.
   - The major boundaries are Python orchestration vs compiled counting/statistics, and per-sample prep vs global post/stat.
   - External dependencies that materially shape the architecture are STAR, PAIRADISE, DARTS, and the C/Fortran scientific stack used by the statistical core.

Main entry points: [`rMATS_pipeline/rmatspipeline/rmatspipeline.pyx`](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_pipeline/rmatspipeline/rmatspipeline.pyx) (Post stage), [`rmats.py`](https://github.com/xinglab/rmats-turbo/blob/master/rmats.py) (rmats.py), [`rMATS_C/src/main.c`](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_C/src/main.c) (rMATS_C), [`tests/runner.py`](https://github.com/xinglab/rmats-turbo/blob/master/tests/runner.py) (Tests)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g1["Orchestration"]
  node_n1["rmats.py<br/>CLI entrypoint<br/>[rmats.py]"]
  node_n2["run_rmats<br/>wrapper script"]
  node_n3["build_rmats<br/>build script"]
end

subgraph group_g2["Core analysis"]
  node_n4["Prep stage<br/>pipeline stage<br/>[rmatspipeline.pyx]"]
  node_n5["Post stage<br/>pipeline stage<br/>[rmatspipeline.pyx]"]
  node_n6["rMATS pipeline<br/>cython engine<br/>[rmatspipeline.pyx]"]
  node_n10["rMATS_C<br/>stat core<br/>[main.c]"]
  node_n11["L-BFGS<br/>optimizer<br/>[lbfgsb.f]"]
  node_n12["PAIRED model<br/>R model<br/>[paired_model.R]"]
  node_n13["DARTS model<br/>R model<br/>[darts_model.R]"]
  node_n14["rMATS_P<br/>postprocess utils"]
  node_n15["utils<br/>support modules"]
end

subgraph group_g3["BAM access"]
  node_n7["BAM tools<br/>I/O library<br/>[BamReader.cpp]"]
  node_n8["I/O internals<br/>storage layer<br/>[BgzfStream_p.cpp]"]
  node_n9["Indexing<br/>index layer"]
end

subgraph group_g4["Workflow packaging"]
  node_n16["Nextflow<br/>workflow wrapper<br/>[main.nf]"]
  node_n17["WDL/CWL<br/>workflow wrapper<br/>[rmatsTurbo.wdl]"]
end

subgraph group_g5["Validation"]
  node_n18["Tests<br/>[runner.py]"]
end

node_n2 -->|"wraps"| node_n1
node_n3 -->|"prepares"| node_n2
node_n1 -->|"dispatches prep"| node_n4
node_n1 -->|"dispatches post"| node_n5
node_n4 -->|"uses"| node_n6
node_n5 -->|"uses"| node_n6
node_n6 -->|"reads alignments"| node_n7
node_n7 -->|"delegates I/O"| node_n8
node_n7 -->|"random access"| node_n9
node_n5 -->|"feeds counts"| node_n10
node_n10 -->|"optimizes with"| node_n11
node_n10 -->|"optional paired"| node_n12
node_n10 -->|"optional darts"| node_n13
node_n5 -->|"postprocesses"| node_n14
node_n5 -->|"supports"| node_n15
node_n16 -->|"invokes"| node_n1
node_n17 -->|"invokes"| node_n1
node_n18 -->|"validates"| node_n1
node_n18 -->|"covers"| node_n4
node_n18 -->|"covers"| node_n5

click node_n1 "https://github.com/xinglab/rmats-turbo/blob/master/rmats.py"
click node_n2 "https://github.com/xinglab/rmats-turbo/tree/master/run_rmats"
click node_n3 "https://github.com/xinglab/rmats-turbo/tree/master/build_rmats"
click node_n4 "https://github.com/xinglab/rmats-turbo/blob/master/rMATS_pipeline/rmatspipeline/rmatspipeline.pyx"
click node_n5 "https://github.com/xinglab/rmats-turbo/blob/master/rMATS_pipeline/rmatspipeline/rmatspipeline.pyx"
click node_n6 "https://github.com/xinglab/rmats-turbo/blob/master/rMATS_pipeline/rmatspipeline/rmatspipeline.pyx"
click node_n7 "https://github.com/xinglab/rmats-turbo/blob/master/bamtools/src/api/BamReader.cpp"
click node_n8 "https://github.com/xinglab/rmats-turbo/blob/master/bamtools/src/api/internal/io/BgzfStream_p.cpp"
click node_n9 "https://github.com/xinglab/rmats-turbo/blob/master/bamtools/src/api/internal/index/BamStandardIndex_p.cpp"
click node_n10 "https://github.com/xinglab/rmats-turbo/blob/master/rMATS_C/src/main.c"
click node_n11 "https://github.com/xinglab/rmats-turbo/blob/master/rMATS_C/lbfgs_scipy/lbfgsb.f"
click node_n12 "https://github.com/xinglab/rmats-turbo/blob/master/rMATS_R/paired_model.R"
click node_n13 "https://github.com/xinglab/rmats-turbo/blob/master/rMATS_R/darts_model.R"
click node_n14 "https://github.com/xinglab/rmats-turbo/blob/master/rMATS_P/prepare_stat_inputs.py"
click node_n15 "https://github.com/xinglab/rmats-turbo/blob/master/bamtools/src/utils/bamtools_utilities.cpp"
click node_n16 "https://github.com/xinglab/rmats-turbo/blob/master/main.nf"
click node_n17 "https://github.com/xinglab/rmats-turbo/blob/master/wdl/rmatsTurbo.wdl"
click node_n18 "https://github.com/xinglab/rmats-turbo/blob/master/tests/runner.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n1,node_n2,node_n3 toneBlue
class node_n4,node_n5,node_n6,node_n10,node_n11,node_n12,node_n13,node_n14,node_n15 toneAmber
class node_n7,node_n8,node_n9 toneMint
class node_n16,node_n17 toneRose
class node_n18 toneIndigo
```

## Components

### Orchestration

CLI entrypoints, staged execution, and packaging wrappers.

- [rmats.py](https://github.com/xinglab/rmats-turbo/blob/master/rmats.py): CLI entrypoint · Main command-line interface and mode dispatcher. · `rmats.py`
- [run_rmats](https://github.com/xinglab/rmats-turbo/tree/master/run_rmats): wrapper script · Launches rMATS in the prepared runtime environment. · `run_rmats`
- [build_rmats](https://github.com/xinglab/rmats-turbo/tree/master/build_rmats): build script · Prepares toolchain and optional dependencies. · `build_rmats`

### Core analysis

Per-sample preprocessing, event counting, and statistical inference.

- [Prep stage](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_pipeline/rmatspipeline/rmatspipeline.pyx): pipeline stage · Per-BAM preprocessing that produces intermediate summaries. · `rMATS_pipeline/rmatspipeline/rmatspipeline.pyx`
- [Post stage](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_pipeline/rmatspipeline/rmatspipeline.pyx): pipeline stage · Merges intermediates, counts events, and writes final outputs. · `rMATS_pipeline/rmatspipeline/rmatspipeline.pyx`
- [rMATS pipeline](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_pipeline/rmatspipeline/rmatspipeline.pyx): cython engine · Performance-critical read traversal, junction counting, and event bookkeeping. · `rMATS_pipeline/rmatspipeline/rmatspipeline.pyx`
- [rMATS_C](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_C/src/main.c): stat core · Compiled statistical inference core for differential splicing. · `rMATS_C/src/main.c`
- [L-BFGS](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_C/lbfgs_scipy/lbfgsb.f): optimizer · Fortran/C optimization backend used by the statistical core. · `rMATS_C/lbfgs_scipy/lbfgsb.f`
- [PAIRED model](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_R/paired_model.R): R model · Optional paired statistical model integration. · `rMATS_R/paired_model.R`
- [DARTS model](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_R/darts_model.R): R model · Optional alternative statistical model integration. · `rMATS_R/darts_model.R`
- [rMATS_P](https://github.com/xinglab/rmats-turbo/blob/master/rMATS_P/prepare_stat_inputs.py): postprocess utils · Helpers for statistical inputs, summaries, and FDR handling. · `rMATS_P/prepare_stat_inputs.py`
- [utils](https://github.com/xinglab/rmats-turbo/blob/master/bamtools/src/utils/bamtools_utilities.cpp): support modules · Parsing, filtering, FASTA, pileup, and CLI helper code. · `bamtools/src/utils/bamtools_utilities.cpp`

### BAM access

Vendored BAM/SAM I/O and indexing library.

- [BAM tools](https://github.com/xinglab/rmats-turbo/blob/master/bamtools/src/api/BamReader.cpp): I/O library · Vendored low-level BAM/SAM access, indexing, and random access. · `bamtools/src/api/BamReader.cpp`
- [I/O internals](https://github.com/xinglab/rmats-turbo/blob/master/bamtools/src/api/internal/io/BgzfStream_p.cpp): storage layer · Local, remote, and BGZF stream abstractions for BAM devices. · `bamtools/src/api/internal/io/BgzfStream_p.cpp`
- [Indexing](https://github.com/xinglab/rmats-turbo/blob/master/bamtools/src/api/internal/index/BamStandardIndex_p.cpp): index layer · BAM index creation and lookup for random access. · `bamtools/src/api/internal/index/BamStandardIndex_p.cpp`

### Workflow packaging

Workflow and container entrypoints for external runtimes.

- [Nextflow](https://github.com/xinglab/rmats-turbo/blob/master/main.nf): workflow wrapper · Nextflow orchestration for local and batch execution. · `main.nf`
- [WDL/CWL](https://github.com/xinglab/rmats-turbo/blob/master/wdl/rmatsTurbo.wdl): workflow wrapper · Workflow definitions and packed CWL wrapper for external engines. · `wdl/rmatsTurbo.wdl`

### Validation

Scenario-based end-to-end tests.

- [Tests](https://github.com/xinglab/rmats-turbo/blob/master/tests/runner.py): integration tests · End-to-end scenario coverage for splicing cases and edge conditions. · `tests/runner.py`

## Connections

- run_rmats → rmats.py: wraps · Runs the CLI through the prepared environment.
- build_rmats → run_rmats: prepares · Builds dependencies and runtime prerequisites for execution.
- rmats.py → Prep stage: dispatches prep · CLI mode routes work into the preprocessing phase.
- rmats.py → Post stage: dispatches post · CLI mode routes work into the merge-and-statistics phase.
- Prep stage → rMATS pipeline: uses · Prep phase relies on the counting engine.
- Post stage → rMATS pipeline: uses · Post phase relies on the counting engine.
- rMATS pipeline → BAM tools: reads alignments · Core engine traverses BAM/SAM records through the vendored library.
- BAM tools → I/O internals: delegates I/O · Library uses internal device and stream abstractions.
- BAM tools → Indexing: random access · Library relies on index support for region queries.
- Post stage → rMATS_C: feeds counts · Merged event counts flow into statistical inference.
- rMATS_C → L-BFGS: optimizes with · C statistical core links to the L-BFGS/Fortran solver stack.
- rMATS_C → PAIRED model: optional paired · Paired inference can switch to the R model.
- rMATS_C → DARTS model: optional darts · Alternative inference can switch to the DARTS model.
- Post stage → rMATS_P: postprocesses · Utilities assemble statistical inputs and summarize outputs.
- Post stage → utils: supports · Shared helpers support parsing and data preparation.
- Nextflow → rmats.py: invokes · Nextflow wrapper drives the main CLI.
- WDL/CWL → rmats.py: invokes · WDL/CWL wrappers execute the same core CLI.
- Tests → rmats.py: validates · Integration tests exercise the full pipeline.
- Tests → Prep stage: covers · Tests verify preprocessing behavior.
- Tests → Post stage: covers · Tests verify postprocessing and outputs.
