How ngirmay/tissue-of-origin works
Project Type and Purpose
• This is a data-analysis pipeline implemented in a Jupyter notebook (“tissue_oo.ipynb”) that ingests RNA-Seq gene-expression data, preprocesses it, trains a gradient-boosting model to classify tumor tissue of origin, evaluates performance, and generates visualizations and notes.
Main Components
a. External Data Source
• ICGC DCC (projects/releases) – contains compressed TSV files (e.g., exp_seq.KIRP-US.tsv.gz)
b. Local Data Store
• genes.txt – list of the 400 signature genes
c. Processing & Modeling
• Data Ingestion & Decompression module (Python/pandas)
• Preprocessing module (gene filtering using genes.txt, normalization)
• Model Training module (Gradient Boosting via scikit-learn/XGBoost)
d. Evaluation & Visualization
• Performance Evaluation module (computes F1, precision, recall per cancer type)
• Visualization module (matplotlib/seaborn plots embedded in Jupyter)
e. Reporting & Notes
• tissue_oo.ipynb – orchestrates the pipeline end-to-end, displays results
• TOO_paper_notes, TMB_classification_notes – markdown/text notes documenting methods and insights
f. Versioning & Config
• .gitattributes – ensures consistent EOLs in Git
Relationships and Data Flow
• External Data Source → (download) → Local RawData
• Local RawData + genes.txt → Data Ingestion & Decompression → Preprocessing
• Preprocessing → Model Training → Evaluation
• Evaluation → Visualization → Outputs back into tissue_oo.ipynb
• Notes files (TOO_paper_notes, TMB_classification_notes) ← inform parameters & interpretation of results
Architectural Patterns & Principles
• Linear, directed pipeline (DAG) with clear stage separation
• Single-entry orchestrator (the notebook) that ties together all stages
• Modular code cells or scripts for each stage (ingest, preprocess, train, evaluate, plot)
• Reproducibility emphasis: versioned data, fixed gene list, documented pipeline
Key Technologies & Frameworks
• Python ecosystem: pandas, numpy, scikit-learn / XGBoost, matplotlib/seaborn
• Jupyter Notebook as both orchestration and reporting layer
• Data source: ICGC DCC (TSV data files)
Essential Elements
• Clear, unambiguous labels on all components
• Optional: “Future Directions” extension path (e.g., add “Pathway Analysis” module)