# yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq architecture

> Architecture diagram of the GitHub repository yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq
- Repository: https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq

## Overview

##### Project type and purpose
- This is a research‐oriented data analysis pipeline rather than a traditional web or microservices application. Its goal is to link COPD clinical metrics to bulk RNA-Seq expression levels by systematically engineering covariates and fitting linear regression models (and a separate QTL‐association analysis).
- Key indicators: presence of Jupyter notebooks under `qtl-association` and `rna-seq-regression`, output folders with matrices and result tables, and a PDF presentation summarizing findings.

##### File structure
- Top‐level directories:
- `presentation/` – final PDF summarizing results
- `qtl-association/` – notebook(s) for variant matching analysis
- `rna-seq-regression/` – notebooks, output covariate/target matrices, and regression results
- Within `rna-seq-regression/`:
- `notebooks/join_and_matrixize_data.ipynb` – data ingestion, feature engineering, joining heterogeneous sources
- `notebooks/regression.ipynb` – fitting linear models per gene/cell type
- `output/covariates/` & `output/target/` – intermediate design matrices
- `output/results/` – final association results (`all_genes_analysis.tsv`)
- No server, API, or deployment scripts; this is a “pipeline‐as‐code” implemented via notebooks.

##### README for additional insights
- Describes the overall regression formula with covariates (PEER factors, PCs from WGS, phenotypic covariates).
- Mentions a separate QTL matching step (`matching.ipynb`) with fuzzy matching on genomic variants.

a. Main components
1. Data Sources
- Clinical metrics CSV/TSV (lung volume measures, covariates)
- RNA-Seq expression matrices by cell type
- WGS principal components and PEER factor files
- e/p/mQTL variant lists
2. Data Processing Layer (Notebooks)
- Join & matrixize notebook
- Regression notebook
- QTL matching notebook
3. Storage / Artifacts
- Interim matrices in `output/covariates/`, `output/target/`
- Final results in `output/results/` and QTL match outputs (not shown in tree)
4. Presentation Layer
- PDF in `presentation/`
b. Relationships and interactions
- Raw data flows into the “Join & Matrixize” notebook → produces covariate & target matrices → fed into “Regression” notebook → emits result TSV.
- QTL variant lists flow into “matching.ipynb” → produces QTL‐association outputs.
- Both result sets are combined or summarized in the PDF presentation.
c. Architectural patterns and principles
- Pipeline pattern: sequential data transformations.
- Separation of concerns: one notebook for feature engineering, one for statistical modeling, one for QTL matching, one for presentation.
- Reproducibility: all steps are encoded in versioned notebooks with explicit inputs and outputs.
d. Key technologies / frameworks
- Jupyter Notebooks (IPython) for interactive analysis
- Python data stack: pandas, numpy, statsmodels (or scikit-learn), possibly Biopython
- File‐based storage (CSV/TSV, matrix files)
- PDF export via notebook–to–PDF or external tool

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Sources
    subgraph "Data Sources"
        direction TB
        rawClinical["Clinical Metrics CSV/TSV"]:::dataStore
        rawExpression["RNA-Seq Expression Matrices"]:::dataStore
        rawWGS["WGS PCs & PEER Factor Files"]:::dataStore
        rawQTL["e/p/mQTL Variant Lists"]:::dataStore
    end

    %% Processing Notebooks
    subgraph "Processing Notebooks"
        direction TB
        join["Join & Matrixize Data Notebook"]:::notebook
        regression["Regression Notebook"]:::notebook
        qtlMatch["QTL Matching Notebook"]:::notebook
    end

    %% Intermediate Covariate Matrices
    subgraph "Covariate Matrices" 
        direction TB
        covMono["Monocyte_Exam_5-covariates.matrix"]:::dataStore
        covPBMC["PBMC_Exam_5-covariates.matrix"]:::dataStore
        covTcell["T_cell_Exam_5-covariates.matrix"]:::dataStore
    end

    %% Intermediate Target Matrices
    subgraph "Target Feature Matrices"
        direction TB
        tgtMono["Monocyte_Exam_5-target-features.matrix"]:::dataStore
        tgtPBMC["PBMC_Exam_5-target-features.matrix"]:::dataStore
        tgtTcell["T_cell_Exam_5-target-features.matrix"]:::dataStore
    end

    %% Final Artifacts
    subgraph "Final Artifacts"
        direction TB
        resultTSV["all_genes_analysis.tsv"]:::artifact
        qtlResults["QTL Matching Results"]:::artifact
        presentationPDF["Manichaikul_Rotation_Presentation.pdf"]:::artifact
    end

    %% Data Flow
    rawClinical -->|"clinical metrics"| join
    rawExpression -->|"expression matrices"| join
    rawWGS -->|"PCs & PEER factors"| join

    join -->|"covariates.matrix"| covMono
    join --> covPBMC
    join --> covTcell

    join -->|"target-features.matrix"| tgtMono
    join --> tgtPBMC
    join --> tgtTcell

    covMono --> regression
    covPBMC --> regression
    covTcell --> regression

    tgtMono --> regression
    tgtPBMC --> regression
    tgtTcell --> regression

    regression -->|"all_genes_analysis.tsv"| resultTSV

    rawQTL -->|"variant lists"| qtlMatch
    qtlMatch -->|"matching_results.csv"| qtlResults

    resultTSV --> presentationPDF
    qtlResults --> presentationPDF

    %% Click Events
    click join "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/rna-seq-regression/notebooks/join_and_matrixize_data.ipynb"
    click regression "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/rna-seq-regression/notebooks/regression.ipynb"
    click qtlMatch "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/qtl-association/notebooks/matching.ipynb"
    click covMono "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/rna-seq-regression/output/covariates/Monocyte_Exam_5-covariates.matrix"
    click covPBMC "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/rna-seq-regression/output/covariates/PBMC_Exam_5-covariates.matrix"
    click covTcell "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/rna-seq-regression/output/covariates/T_cell_Exam_5-covariates.matrix"
    click tgtMono "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/rna-seq-regression/output/target/Monocyte_Exam_5-target-features.matrix"
    click tgtPBMC "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/rna-seq-regression/output/target/PBMC_Exam_5-target-features.matrix"
    click tgtTcell "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/rna-seq-regression/output/target/T_cell_Exam_5-target-features.matrix"
    click resultTSV "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/rna-seq-regression/output/results/all_genes_analysis.tsv"
    click presentationPDF "https://github.com/yogionbioinformatics/copd-clinical-metric-prediction-from-rna-seq/blob/main/presentation/Manichaikul_Rotation_Presentation.pdf"

    %% Legend
    subgraph Legend
        NB["Notebook"]:::notebook
        DS["Data Store/File"]:::dataStore
        AR["Artifact"]:::artifact
    end

    classDef dataStore fill:#B0E0E6,stroke:#000;
    classDef notebook fill:#98FB98,stroke:#000;
    classDef artifact fill:#FFA500,stroke:#000;
```
