How morris-lab/celloracle works
Project type and purpose
• CellOracle is an open-source Python library (a toolkit) for in silico gene perturbation analyses using single-cell omics data and Gene Regulatory Network (GRN) models. It provides a modular pipeline from raw data ingestion through network inference and simulation to downstream analysis and visualization.
File-structure analysis
• Top-level packaging and CI:
• setup.py, requirements.txt, Dockerfile, Makefile, GitHub Actions workflows (build_check.yml, build_weekly_check.yml)
• Core code (celloracle/):
• data_conversion: R/Python scripts to convert Seurat or R objects into matrix formats
• data: loading reference genomes, motif and TSS annotation, example Anndata files
• motif_analysis: scanning peaks to TF binding information, processing Cicero and BED files
• network: core classes and utilities for building GRN scaffolds and regression models
• network_analysis: R‐script wrappers and utilities for computing network scores and cartography
• trajectory: oracle core, Markov simulation, pseudotime, Sankey transitions
• oracle_utility & applications: higher-level workflows (pseudotime, perturbation simulation, systematic analysis)
• visualizations: plotting modules for Oracle objects and development modules
• utility: generic helpers (I/O, HDF5, logging, package version checks)
• Test suite under celloracle/tests covers data loading, motif analysis, network construction, tutorial pipelines.
README insights
• Confirms supported reference genomes, external dependencies (Python ≥3.7, Scanpy, pandas, R packages via script wrappers), and use cases (ATAC-seq, scRNA-seq, simulation, pseudotime).
• Points to extensive documentation and tutorials, demonstrating Jupyter notebook–driven user interaction.
a. Main components (layers and modules)
• Input/External Data: raw single-cell omics (h5ad, Seurat), TF motif databases, reference genome annotations, downloaded via utility/data_download_from_web or prepackaged in celloracle/data
• Data Conversion Layer: process_seurat_object.py, seurat_to_mtx.R
• Data Loading Layer: load_data.py, load_promoter_base_GRN.py, config.py
• Motif Analysis: motif_data.py, process_bed_file.py, process_cicero_data.py, tfinfo_core.py
• GRN Scaffold/Construction: net_core.py, regression_models.py, network_construction.py
• Network Analysis: goatools_wrapper.py, use_r_scripts.py, gene_analysis, cartography
• Simulation & Trajectory Engine: oracle_core.py, markov_simulation.py, pseudotime modules, sankey
• Application Workflows: applications/*.py (pseudotime, differentiation_flow, systematic_analysis_helper)
• Visualization: oracle_object_visualization.py, development_module_visualization.py
• Utilities: logging, HDF5 handling, package checker
• Packaging/Deployment: setup.py, Dockerfile, GitHub Actions CI/CD
b. Relationships and data flow
1. Raw input → Data Conversion → standardized matrices
2. Standardized matrices + motif/reference files → Data Loading → in-memory Anndata or Pandas structures
3. Loaded data → Motif Analysis → TF-by-target info (parquet, pickle)
4. Motif results + promoter GRN data → GRN Scaffold → regression models infer edge weights
5. Inferred network → Network Analysis (R wrappers) → network metrics, cartography
6. Inferred network + cell embeddings → Simulation/Trajectory Engine → transition probabilities, pseudotime
7. Simulation outputs → Application Workflows → aggregate metrics, systematic perturbation results
8. Application outputs → Visualization → figures, interactive plots
9. Utility modules support logging, caching, downloads at each stage
10. Tests validate correctness at each step
c. Architectural patterns and design principles
• Layered/pipeline architecture: clearly separated data ingestion, processing, core logic, analysis, and visualization
• Modular design: each subpackage encapsulates a cohesive responsibility
• Extensibility via R script wrappers for network analysis and via application helper modules
• Data formats standardized (Anndata, parquet, pickle) for interoperability
d. Key technologies and frameworks
• Python ecosystem: pandas, numpy, scanpy/anndata, scikit-learn
• R ecosystem: igraph, goatools, linkcomm, newtork packages invoked via subprocess
• File formats: HDF5 (.h5ad), parquet, BED, text, pickle
• CI/CD: GitHub Actions, Docker, PyPI packaging