How 10xgenomics/cellranger works
Main Components and Their Roles
• Input & Preprocessing: Represent components responsible for ingesting raw data, demultiplexing samples, and processing barcodes. This is evidenced by directories like “bin/rna”, “bin/sc_rna”, and “mro/rna”, which house scripts for different stages (e.g., cellranger demultiplexing, basic cell counting, barcode filtering).
• Analysis Pipelines: Identify distinct pipelines or modules for various analytical steps:
– Gene Counting & Expression Analysis (located in lib/python/cellranger/rna and corresponding mro pipelines)
– V(D)J Transcript Assembly (modules under lib/python/cellranger/vdj and mro/rna/stages/vdj)
– Feature Barcode Analysis
– CRISPR or Antibody specific tasks (see “lib/python/cellranger/feature”, “mro/rna/_crispr_analyzer.mro”, etc.)
• Core Libraries and Shared Utilities: Capture shared libraries in “lib” (with subfolders in Python, Go, and Rust) which implement common functionality like barcode correction, file I/O, telemetry (under lib/go/telemetry), and data conversions.
• Pipeline Orchestration: Highlight the “mro” directories that define the high-level stages in the pipeline. These “.mro” files contain the workflow definitions and stage orchestration logic used in analysis.
• External Dependencies & Third-Party Tools: Represent integration with external dependencies (contained in “third-party”) and build/deployment systems (.github workflows, configuration files like conda specs, etc.).
Relationships and Interactions
• Orchestration: Show how the “mro” stage files coordinate the execution between these components, initiating preflight checks, followed by sequential and parallel processing stages, and eventually generating summary reports.
Architectural Patterns and Design Principles
• Modular Design: Emphasize that the system is split into discrete pipelines—for example, separate modules for RNA counting and VDJ analysis—that likely follow a plug-and-play or microservice-like style within a single executable suite.
• Pipeline Staging: Indicate that stages are clearly demarcated (from preprocessing through analysis to postprocessing) using the “mro” definitions.