How yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq works
Project type and purpose
• This is a research‐oriented data analysis pipeline rather than a traditional web or microservices application. Its goal is to link COPD clinical metrics to bulk RNA-Seq expression levels by systematically engineering covariates and fitting linear regression models (and a separate QTL‐association analysis).
• Key indicators: presence of Jupyter notebooks under qtl-association and rna-seq-regression, output folders with matrices and result tables, and a PDF presentation summarizing findings.
File structure
• Top‐level directories:
• presentation/ – final PDF summarizing results
• qtl-association/ – notebook(s) for variant matching analysis
• rna-seq-regression/ – notebooks, output covariate/target matrices, and regression results
• Within rna-seq-regression/:
• notebooks/join_and_matrixize_data.ipynb – data ingestion, feature engineering, joining heterogeneous sources
• notebooks/regression.ipynb – fitting linear models per gene/cell type
• output/covariates/ & output/target/ – intermediate design matrices
• output/results/ – final association results (all_genes_analysis.tsv)
• No server, API, or deployment scripts; this is a “pipeline‐as‐code” implemented via notebooks.
README for additional insights
• Describes the overall regression formula with covariates (PEER factors, PCs from WGS, phenotypic covariates).
• Mentions a separate QTL matching step (matching.ipynb) with fuzzy matching on genomic variants.
a. Main components
1. Data Sources
• Clinical metrics CSV/TSV (lung volume measures, covariates)
• RNA-Seq expression matrices by cell type
• WGS principal components and PEER factor files
• e/p/mQTL variant lists
2. Data Processing Layer (Notebooks)
• Join & matrixize notebook
• Regression notebook
• QTL matching notebook
3. Storage / Artifacts
• Interim matrices in output/covariates/, output/target/
• Final results in output/results/ and QTL match outputs (not shown in tree)
4. Presentation Layer
• PDF in presentation/
b. Relationships and interactions
• Raw data flows into the “Join & Matrixize” notebook → produces covariate & target matrices → fed into “Regression” notebook → emits result TSV.
• QTL variant lists flow into “matching.ipynb” → produces QTL‐association outputs.
• Both result sets are combined or summarized in the PDF presentation.
c. Architectural patterns and principles
• Pipeline pattern: sequential data transformations.
• Separation of concerns: one notebook for feature engineering, one for statistical modeling, one for QTL matching, one for presentation.
• Reproducibility: all steps are encoded in versioned notebooks with explicit inputs and outputs.
d. Key technologies / frameworks
• Jupyter Notebooks (IPython) for interactive analysis
• Python data stack: pandas, numpy, statsmodels (or scikit-learn), possibly Biopython
• File‐based storage (CSV/TSV, matrix files)
• PDF export via notebook–to–PDF or external tool