# ejmric/stmf architecture

> Architecture diagram of the GitHub repository ejmric/stmf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ejmric/stmf
- Repository: https://github.com/ejmric/stmf

## Overview

##### Project type and purpose
- The core deliverable is a Python module (`STMF.py` and `STMF_urf_ulf.py`) offering `STMF` class methods (`fit`, `predict_all`). Surrounding scripts/notebooks perform data preprocessing, model initialization, evaluation (distance correlation), permutation experiments, and result plotting.

##### File structure
- Top-level code modules:
- STMF.py, STMF_urf_ulf.py – core algorithms
- utils.py – helper functions
- distance_correlation.py – implementation of evaluation metric
- Data directories:
- multi_omic/ – real TCGA/PAM50/CBIO raw data files (.txt, .csv)
- initializations/ – sample initialization matrices (notebooks + .ipynb)
- synthetic_experiments/ & perm_techn/ & real_data_experiments/ – numerous Jupyter notebooks and associated CSV result folders
- Config/requirements:
- requirements.txt – Python dependencies
- .gitignore – excludes result artifacts and data caches

##### README for additional insights
- The README describes how to import and run the model, points at external data sources (TCGA, CBIO, PAM50) and the academic reference.

##### Main components and interactions (
a. Components:
1. Data Sources (external links → raw CSVs/TXT)
2. Data Preprocessing layer (notebooks or Python scripts in initializations, utils)
3. Initialization module (`initializations/`, random or custom starts)
4. Core STMF Engine (STMF.py, STMF_urf_ulf.py)
5. Evaluation Metric (distance_correlation.py)
6. Experiment Orchestration (Jupyter notebooks under real_data_experiments/, synthetic_experiments/, perm_techn/)
7. Results Storage (CSV outputs under results/ subfolders)
8. Visualization (notebooks that generate violin plots, critical difference charts)
9. External References (academic papers, external data download sites)
b. Relationships & data flow:
- Raw Data → Preprocessing → Initialization → STMF Engine → Approximation
- Approximation + Raw Data → Evaluation Module → Metrics
- Metrics → Visualization → Plots (CSV + embedded graphs)
- Experiment scripts glue these steps: import Data, call STMF, call distance_correlation, write results
c. Architectural patterns & design principles:
- Modular design: Core algorithm separated from evaluation/utilities and orchestration notebooks.
- Reusability: STMF class exposes a simple API (`fit`, `predict_all`), pluggable “initialization” strategies.
d. Key technologies:
- Python 3.x, NumPy, SciPy, Pandas (in requirements.txt)
- Jupyter notebooks for interactive orchestration
- CSV/TXT for data interchange
- Git for version control

##### Detailed capture of essential elements
- “Experiment Notebooks” to “STMF Core”, “utils.py”, “distance_correlation.py”.
- “Results CSV Folders” as output sinks from notebooks.
- External data URLs feeding into “multi_omic” folder.
- “requirements.txt” as dependency loader for Python environment.

## Diagram (Mermaid)

```mermaid
flowchart TB
  %% Tier 1: Data Acquisition & Preprocessing
  subgraph "Tier 1: Data Acquisition & Preprocessing"
    subgraph "Data Sources"
      BRCA["BRCA_molecularCBIO_patient_strat.txt"]:::data
      BIC["bic_subtypes_pam.csv"]:::data
      PAM50["pam50_annotation.txt"]:::data
    end
    subgraph "Preprocessing"
      Utils["utils.py"]:::code
      DataPrep["data_preparation_bic_subtypes.ipynb"]:::notebook
      Datasets["datasets.ipynb"]:::notebook
    end
  end

  %% Tier 2: Core Algorithm & Evaluation
  subgraph "Tier 2: Core Algorithm & Evaluation"
    subgraph "Core Library"
      Init["initializations/"]:::code
      STMF1["STMF.py"]:::code
      STMF2["STMF_urf_ulf.py"]:::code
      DistCorr["distance_correlation.py"]:::code
    end
    subgraph "Experiment Orchestration"
      Perm["perm_techn.ipynb"]:::notebook
      SynNB["perm_techn_*.ipynb"]:::notebook
      RealNB["real_data_experiments/*.ipynb"]:::notebook
    end
  end

  %% Tier 3: Results Storage & Visualization
  subgraph "Tier 3: Results Storage & Visualization"
    subgraph "Results Storage"
      RealRes["real_data_experiments/results/"]:::data
      SynRes1["synthetic_experiments/results/rank_4/"]:::data
      SynRes2["synthetic_experiments/results/urf_ulf/"]:::data
      PermCSV["perm_techn/violin_plots_*.csv"]:::data
    end
    subgraph "Visualization"
      ViolinAnalysis["violin_plots_analysis.ipynb"]:::notebook
      CritDiff["critical_difference.ipynb"]:::notebook
    end
  end

  %% Configuration / Dependencies
  Requirements["requirements.txt"]:::external
  Gitignore[".gitignore"]:::external
  Readme["README.md"]:::external

  %% Data Flows and Dependencies
  BRCA -->|"raw CSV→DataFrame"| Utils
  BIC --> Utils
  PAM50 --> Utils
  Utils -->|processed DataFrame| Init
  Utils -->|helper functions| STMF1
  Utils --> STMF2
  Init -->|"init matrices"| STMF1
  Init --> STMF2
  STMF1 -->|"approximation"| DistCorr
  STMF2 --> DistCorr
  DistCorr -->|metrics| Perm
  DistCorr --> SynNB
  DistCorr --> RealNB
  Utils --> RealNB
  STMF1 --> RealNB
  STMF2 --> RealNB

  Perm -->|"results CSV"| PermCSV
  SynNB -->|"results CSV"| SynRes1
  SynNB --> SynRes2
  RealNB --> RealRes

  PermCSV --> ViolinAnalysis
  SynRes1 --> ViolinAnalysis
  SynRes2 --> ViolinAnalysis
  RealRes --> CritDiff

  Requirements -.->|env| Utils
  Requirements -.-> STMF1
  Requirements -.-> STMF2
  Readme -.-> Utils

  %% Click Events
  click BRCA "https://github.com/ejmric/stmf/blob/master/multi_omic/BRCA_molecularCBIO_patient_strat.txt"
  click BIC "https://github.com/ejmric/stmf/blob/master/multi_omic/bic_subtypes_pam.csv"
  click PAM50 "https://github.com/ejmric/stmf/blob/master/multi_omic/pam50_annotation.txt"
  click Utils "https://github.com/ejmric/stmf/blob/master/utils.py"
  click DataPrep "https://github.com/ejmric/stmf/blob/master/real_data_experiments/data_preparation_bic_subtypes.ipynb"
  click Datasets "https://github.com/ejmric/stmf/blob/master/real_data_experiments/datasets.ipynb"
  click Init "https://github.com/ejmric/stmf/tree/master/initializations/"
  click STMF1 "https://github.com/ejmric/stmf/blob/master/STMF.py"
  click STMF2 "https://github.com/ejmric/stmf/blob/master/STMF_urf_ulf.py"
  click DistCorr "https://github.com/ejmric/stmf/blob/master/distance_correlation.py"
  click Perm "https://github.com/ejmric/stmf/blob/master/perm_techn.ipynb"
  click PermCSV "https://github.com/ejmric/stmf/blob/master/perm_techn/violin_plots_*.csv"
  click SynNB "https://github.com/ejmric/stmf/blob/master/synthetic_experiments/perm_techn_*.ipynb"
  click SynRes1 "https://github.com/ejmric/stmf/tree/master/synthetic_experiments/results/rank_4/"
  click SynRes2 "https://github.com/ejmric/stmf/tree/master/synthetic_experiments/results/urf_ulf/"
  click RealNB "https://github.com/ejmric/stmf/blob/master/real_data_experiments/*.ipynb"
  click RealRes "https://github.com/ejmric/stmf/tree/master/real_data_experiments/results/"
  click ViolinAnalysis "https://github.com/ejmric/stmf/blob/master/violin_plots_analysis.ipynb"
  click CritDiff "https://github.com/ejmric/stmf/blob/master/critical_difference.ipynb"
  click Requirements "https://github.com/ejmric/stmf/blob/master/requirements.txt"
  click Gitignore "https://github.com/ejmric/stmf/blob/master/.gitignore"
  click Readme "https://github.com/ejmric/stmf/blob/master/README.md"

  %% Styles
  classDef code fill:#D0E1F9,stroke:#3A70A1,color:#0D2A54
  classDef data fill:#E2F0D9,stroke:#5A8F3B,color:#2E4D21
  classDef notebook fill:#FFE4C4,stroke:#D9913E,color:#7A3E00
  classDef external fill:#F0F0F0,stroke:#999999,color:#333333
```
