How asifsiddque/prion_rnaseq_meta_analysis works
1. Repository purpose
- This is a bulk RNA-seq meta-analysis pipeline for prion disease, centered on discovering and validating the Mouse Conserved Prion Signature (MCPS) across mouse CNS studies, human sCJD, peripheral longitudinal tissue, and cell-type composition analysis.
2. Primary project boundary
- The repo is organized as a workflow from raw public sequencing data to publication-ready results:
- GEO/SRA inputs and curated sample metadata in config/
- transient FASTQs in data/raw/
- reference assets in resources/
- analysis outputs in results/
- figures in reports/figures/
- The core runtime is split between shell scripts for acquisition/QC/quantification and R notebooks/scripts for statistical analysis.
3. Ingestion and sample registry
- config/sample_metadata.tsv is the master sample registry linking SRR accessions to GSE studies, layout, organism, and study-specific annotations.
- scripts/00_download_fastq.sh uses SRA Toolkit to fetch raw FASTQs.
- scripts/04_build_coldata_v2.ipynb/the documented coldata step builds per-study metadata tables from GEO and caches them under config/coldata/.
4. Preprocessing boundary: QC and quantification
- scripts/01_qc.sh runs FastQC and MultiQC on the downloaded FASTQs, producing study-level QC reports.
- scripts/02_build_salmon_index.sh builds Salmon indices for both mouse and human references.
- scripts/03_reads_quant.sh performs transcript quantification with Salmon and writes per-sample quant.sf files.
- FASTQs are explicitly transient: they are removed after quantification, so the persistent boundary is the Salmon quant output.
5. Reference and cache layer
- resources/genome_indices/ holds the built Salmon indices, separating heavy reference generation from downstream analysis.
- resources/cache/ stores reusable metadata caches for tximeta, BioMart, and GEO, which reduces repeated network calls and keeps downstream analysis deterministic after the first run.
- resources/gene_sets/ stores MSigDB-derived gene sets used in enrichment analysis.
6. Stream 1: CNS discovery and meta-analysis
- scripts/05_stream1_cns_meta_analysis_v2.ipynb is the main discovery engine.
- It runs study-specific DESeq2 models for five CNS discovery studies, then combines them with a random-effects meta-analysis to define MCPS.
- It also performs held-out replication on the Stratum D studies and generates pathway enrichment outputs.
- Key artifact: results/stream1/MCPS_genes.csv, which becomes the main dependency for later streams.
7. Stream 2: human validation
- scripts/06_stream2_human_validation_v2.ipynb analyzes the human sCJD dataset (GSE214373) with DESeq2.
- This stream adds a distinct boundary: human ortholog mapping from human Ensembl IDs to mouse gene symbols using cached BioMart lookups.
- It produces human validation tables, MCPS concordance results, GSEA outputs, and power analysis.
- Its outputs feed the integration layer, especially results/stream2/GSE214373_GSEA_MCPS.csv.
8. Stream 3: peripheral longitudinal analysis
- scripts/07_stream3_peripheral_analysis_v2.ipynb analyzes the peripheral longitudinal study (GSE210128) across whole blood, spleen, and skeletal muscle.
- It combines multiple statistical views: longitudinal interaction tests, terminal-timepoint differential expression, concordance with MCPS, and first-detection analysis over time.
- It also cross-references Stream 2 human results for a later “triple-validated” candidate set.
9. Integration layer across streams
- scripts/08_integrate_streams_v2.ipynb is the synthesis step.
- It joins outputs from Streams 1–3 into a single master integration table and computes:
- evidence tiers
- cross-stream log2FC correlations
- replication cascade summaries
- pathway convergence summaries
- This script is intentionally tolerant of missing inputs, so it can still build a partial master table if one upstream stream has not completed.
10. Cell-type deconvolution as an orthogonal validation
- scripts/09_cell_type_deconvolution_v1.ipynb uses VST-normalized matrices from Stream 1 to estimate cell-type signal with marker-gene z-scores.
- It explicitly excludes MCPS genes from marker sets to avoid circularity.
- The result is a separate validation axis: whether MCPS tracks with microglial or other cell-type composition changes.
11. Data model and file dependencies
- The architecture is strongly file-driven:
- raw accession list → FASTQs → Salmon quantifications
- GEO-derived metadata → per-study coldata
- quantifications + coldata → DESeq2 outputs
- Stream 1 MCPS gene list → Streams 2, 3, 9
- Streams 2 and 3 → integration table
- This makes the repo easy to diagram as a pipeline with explicit handoff files between stages.
12. Reproducibility and tooling
- The project relies on a conda environment (scripts/environment.yml) for system-level tools and R runtime support.
- The README also documents R package pinning with renv, plus cached reference metadata to stabilize results across machines.
- Outputs are split cleanly between results/ and reports/figures/, reflecting a clear separation between computation and presentation.
13. Important architectural takeaway
- The repo is not a general-purpose app; it is a reproducible scientific pipeline with three major analysis branches after a shared preprocessing backbone.
- For diagramming, the critical nodes are:
- raw data acquisition
- QC/quantification
- metadata assembly
- Stream 1 discovery
- Stream 2 human validation
- Stream 3 peripheral validation
- integration
- deconvolution
- figure/report outputs
Main entry points: 05_stream1_cns_meta_analysis_v2.ipynb Stream 1 · 08_integrate_streams_v2.ipynb Integrate streams · environment.yml Runtime env · 06_stream2_human_validation_v2.ipynb Stream 2
Components 21
Inputs & metadata
Raw accessions, GEO-derived study metadata, and cached sample tables that seed the workflow.
- Sample registry · Master mapping of SRR accessions to studies, layouts, organisms, and study annotations.
config/sample_metadata.tsv - Script metadata · Script-local sample metadata copy used by the pipeline.
scripts/sample_metadata.tsv - GEO soft files · Downloaded GEO SOFT files used to build study-level coldata tables.
data/raw/geo_soft - Build coldata · Parses GEO metadata into per-study coldata tables and caches them.
scripts/04_build_coldata_v2.ipynb - Coldata cache · Persisted per-study sample tables for downstream DESeq2 models.
config/coldata
Acquisition & preprocessing
Download, QC, reference setup, and quantification backbone shared by all analyses.
- Download FASTQ · Fetches raw sequencing reads from SRA using the sample registry.
scripts/00_download_fastq.sh - QC reports · Runs FastQC and MultiQC on downloaded FASTQs.
scripts/01_qc.sh - Build index · Builds Salmon transcriptome indices for mouse and human references.
scripts/02_build_salmon_index.sh - Quantify reads · Runs Salmon quantification and produces per-sample quant.sf files.
scripts/03_reads_quant.sh
Analysis streams
Three main statistical branches plus synthesis and orthogonal validation.
- Stream 1 · Discovery engine for mouse CNS studies and MCPS definition via DESeq2 and random-effects meta-analysis.
scripts/05_stream1_cns_meta_analysis_v2.ipynb - MCPS genes · Core Mouse Conserved Prion Signature gene list used by later streams.
- Stream 2 · Human sCJD validation with ortholog mapping, concordance analysis, and GSEA.
scripts/06_stream2_human_validation_v2.ipynb - Human GSEA · Human validation enrichment and MCPS concordance results.
- Stream 3 · Longitudinal peripheral analysis across blood, spleen, and skeletal muscle.
scripts/07_stream3_peripheral_analysis_v2.ipynb - Integrate streams · Combines all stream outputs into a master evidence table and summaries.
scripts/08_integrate_streams_v2.ipynb - Cell deconvolution · Estimates cell-type signal from Stream 1 matrices while excluding MCPS genes.
scripts/09_cell_type_deconvolution_v1.ipynb
Runtime & resources
Reproducibility tooling and reusable caches referenced across the pipeline.
- Reference cache · Reusable tximeta, BioMart, and GEO caches that stabilize repeated lookups.
- Gene sets · MSigDB-derived gene sets used for pathway analysis and enrichment testing.
- Runtime env · Pinned system tools and R package environment for reproducibility.
scripts/environment.yml - Results · Persistent computed outputs organized by stream and synthesis stage.
- Figures · Publication figures generated from analysis outputs.
Connections 31
- Sample registry → Download FASTQ: accessions · Sample registry drives SRA download targets.
- Script metadata → Download FASTQ: targets · Script-local accession metadata supports download orchestration.
- GEO soft files → Build coldata: study metadata · GEO soft files are parsed into per-study coldata tables.
- Build coldata → Coldata cache: cache · Built coldata are cached for reuse.
- Download FASTQ → QC reports: FASTQs · Downloaded reads flow into QC.
- QC reports → Quantify reads: clean reads · QC-checked reads proceed to quantification.
- Build index → Quantify reads: index · Salmon quantification uses prebuilt transcriptome indices.
- Quantify reads → Stream 1: quant.sf · Per-sample quantifications feed the discovery analysis.
- Coldata cache → Stream 1: design · Study-level metadata parameterize DESeq2 models.
- Reference cache → Stream 1: lookups · Cached metadata reduce repeated network calls.
- Stream 1 → MCPS genes: signature · Stream 1 defines the MCPS gene set.
- MCPS genes → Stream 2: genes · MCPS genes are mapped into the human validation analysis.
- Reference cache → Stream 2: orthologs · BioMart cache supports human-to-mouse ortholog mapping.
- Coldata cache → Stream 2: design · Human study metadata drive DESeq2 modeling.
- MCPS genes → Stream 3: signature · Peripheral analysis tests MCPS concordance over time.
- Coldata cache → Stream 3: design · Peripheral study metadata support longitudinal models.
- Stream 1 → Cell deconvolution: matrices · VST matrices from Stream 1 support deconvolution.
- MCPS genes → Cell deconvolution: exclude · MCPS genes are removed from marker sets to avoid circularity.
- Stream 1 → Integrate streams: evidence · Discovery outputs feed the synthesis layer.
- Stream 2 → Integrate streams: human results · Human validation results are merged into the master table.
- Stream 3 → Integrate streams: peripheral results · Peripheral longitudinal outputs are merged into the master table.
- Human GSEA → Integrate streams: concordance · Human GSEA and concordance tables are consumed by synthesis.
- Integrate streams → Results: writes · Integrated outputs are stored under the results tree.
- Integrate streams → Figures: plots · Publication figures are generated from the integrated analyses.
- Runtime env → Download FASTQ: runtime · Shared environment provides command-line and R tooling.
- Runtime env → Stream 1: runtime · Analysis notebooks run in the pinned environment.
- Runtime env → Stream 2: runtime · Analysis notebooks run in the pinned environment.
- Runtime env → Stream 3: runtime · Analysis notebooks run in the pinned environment.
- Runtime env → Integrate streams: runtime · Synthesis step uses the same reproducible environment.
- Runtime env → Cell deconvolution: runtime · Orthogonal validation runs in the same pinned environment.
- Gene sets → Stream 1: enrichment · Gene sets support pathway enrichment in the discovery stream.