# dootmaan/m2ost architecture

> Architecture diagram of the GitHub repository dootmaan/m2ost, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dootmaan/m2ost
- Repository: https://github.com/dootmaan/m2ost

## Overview

##### Project Type and Purpose
- This is a research‐oriented deep learning pipeline (not a web app or microservice) for “many-to-one” regression: predicting spatial transcriptomics profiles from digital pathology images.
- It consists of data ingestion/preparation, model definition (M2OST), training orchestration, utility scripts for metrics/visualization, and weight storage.

##### File Structure Analysis
- Top‐level directories:
- dataset/ – Python Dataset classes for HBC, HER2+, cSCC, plus pre-selected gene arrays (.npy)
- model/ – implementation of the M2OST network (M2OST.py)
- utils/ – helper scripts: FLOPs/parameter counting, feature-selection, visualization (npy→mat, spatial maps)
- weights/ – pretrained model checkpoints (.pth)
- Training entrypoint: train_m2ost_example.py (for HBC; to be adapted per dataset)
- Config/deployment: CUDA_VISIBLE_DEVICES environment variable; log redirection.

##### Key README Insights
- Describes end-to-end workflow: clone → download/prepare datasets → run training script
- External dependencies: PyTorch, NumPy, Matplotlib (implicit in utils), 7z for encrypted data
- Mentions M2ORT predecessor; weights may be interchangeable after verification.

a. Components
1. External Data Sources
- HBC, HER2+, cSCC repositories (Mendeley, Zenodo, NCBI GEO)
2. Data Preparation Layer
- dataset/*.py classes (HBCDataset, HER2Dataset, cSCCDataset)
- Pre‐selected genes (.npy)
- 7z decryption step (external tool)
3. Feature Selection
- utils/feature_selection_*.py
4. Training Orchestration
- train_m2ost_example.py (scripts)
- Environment: CUDA devices, Python interpreter
5. Model Definition
- model/M2OST.py (PyTorch nn.Module)
6. Utilities
- utils/calculateParamsFLOPs.py
- utils/visualization_*.py
7. Artifact Storage
- weights/*.pth
- Training logs (.log)
- Generated .mat or spatial visualization outputs
b. Relationships & Data Flow
5. “Utilities” can attach to multiple stages:
– After model instantiation: calculateParamsFLOPs
– After training or inference: visualization_npy2mat, visualization_st
c. Architectural Patterns & Principles
- Modular separation: data / model / training / utilities
- Single‐responsibility: each dataset class handles one dataset, each util script one function
- Script‐driven orchestration rather than heavy config frameworks
- Extensibility: add new Dataset subclass or new feature-selection utility without touching core
d. Key Technologies
- PyTorch (training, model)
- NumPy (.npy arrays, pre-selected genes)
- 7z (data decryption)
- Python logging (via STDOUT/redirect in shell)
- Potential plotting libs for visualization (Matplotlib, SciPy I/O for .mat)

## Diagram (Mermaid)

```mermaid
graph TB
    %% External Systems
    subgraph "External Data Sources"
        style External fill:#ddd,stroke:#333,stroke-width:1px
        HBCRepo["HBC Repository"]:::external
        HER2Repo["HER2+ Repository"]:::external
        cSCCRepo["cSCC Repository"]:::external
    end

    %% Data Layer
    subgraph "Data Preparation Layer"
        style DataPrep fill:#cfe2f3,stroke:#333,stroke-width:1px
        HBCDataset["HBCDataset.py"]:::data
        HBCGenes["HBC_Selected_Genes.npy"]:::storage
        HER2Dataset["HER2Dataset.py"]:::data
        HER2Genes["HER2_Selected_Genes.npy"]:::storage
        cSCCDataset["cSCCDataset.py"]:::data
        cSCCGenes["cSCC_Selected_Genes.npy"]:::storage
        Decrypt["7z Decryption"]:::external
    end

    %% Feature Selection
    subgraph "Feature Selection"
        style FeatureSel fill:#cfe2f3,stroke:#333,stroke-width:1px
        FS_HBC["feature_selection_HBC.py"]:::utils
        FS_HER2["feature_selection_HER2.py"]:::utils
        FS_cSCC["feature_selection_cSCC.py"]:::utils
    end

    %% Model and Training
    subgraph "Model & Training Orchestration"
        style ModelTrain fill:#d9ead3,stroke:#333,stroke-width:1px
        Model["M2OST.py<br/>(PyTorch nn.Module)"]:::model
        TrainScript["train_m2ost_example.py"]:::training
    end

    %% Utilities
    subgraph "Utilities"
        style Utils fill:#f9cb9c,stroke:#333,stroke-width:1px
        FLOPs["calculateParamsFLOPs.py"]:::utils
        VisNpy["visualization_npy2mat.py"]:::utils
        VisST["visualization_st.py"]:::utils
    end

    %% Artifact Storage
    subgraph "Artifact Storage"
        style Storage fill:#eee,stroke:#333,stroke-width:1px
        Check1["MSViT_Small_GSE144240_best.pth"]:::storage
        Check2["MSViT_Small_HER2_best.pth"]:::storage
        Check3["MSViT_Small_best.pth"]:::storage
        Check4["MSViT_small_all_best.pth"]:::storage
        Check5["MSViT_small_best.pth"]:::storage
    end

    %% Data Flows
    HBCRepo -->|raw images + counts| HBCDataset
    HER2Repo -->|raw images + counts| HER2Dataset
    cSCCRepo -->|raw images + counts| cSCCDataset
    Decrypt --> HBCDataset
    Decrypt --> HER2Dataset
    Decrypt --> cSCCDataset

    HBCDataset -->|cleaned arrays| HBCGenes
    HER2Dataset -->|cleaned arrays| HER2Genes
    cSCCDataset -->|cleaned arrays| cSCCGenes

    HBCGenes -->|gene vectors| FS_HBC
    HER2Genes -->|gene vectors| FS_HER2
    cSCCGenes -->|gene vectors| FS_cSCC

    FS_HBC -->|filtered genes| TrainScript
    FS_HER2 -->|filtered genes| TrainScript
    FS_cSCC -->|filtered genes| TrainScript

    Model -->|"model class"| TrainScript
    TrainScript -->|.pth checkpoint| Check1
    TrainScript -->|.pth checkpoint| Check2
    TrainScript -->|.pth checkpoint| Check3
    TrainScript -->|.pth checkpoint| Check4
    TrainScript -->|.pth checkpoint| Check5

    Model -->|model instantiation| FLOPs
    TrainScript -->|inference outputs| VisNpy
    TrainScript -->|inference outputs| VisST

    %% Click Events
    click HBCDataset "https://github.com/dootmaan/m2ost/blob/main/dataset/HBCDataset.py"
    click HBCGenes "https://github.com/dootmaan/m2ost/blob/main/dataset/HBC_Selected_Genes.npy"
    click HER2Dataset "https://github.com/dootmaan/m2ost/blob/main/dataset/HER2Dataset.py"
    click HER2Genes "https://github.com/dootmaan/m2ost/blob/main/dataset/HER2_Selected_Genes.npy"
    click cSCCDataset "https://github.com/dootmaan/m2ost/blob/main/dataset/cSCCDataset.py"
    click cSCCGenes "https://github.com/dootmaan/m2ost/blob/main/dataset/cSCC_Selected_Genes.npy"
    click FS_HBC "https://github.com/dootmaan/m2ost/blob/main/utils/feature_selection_HBC.py"
    click FS_HER2 "https://github.com/dootmaan/m2ost/blob/main/utils/feature_selection_HER2.py"
    click FS_cSCC "https://github.com/dootmaan/m2ost/blob/main/utils/feature_selection_cSCC.py"
    click Model "https://github.com/dootmaan/m2ost/blob/main/model/M2OST.py"
    click TrainScript "https://github.com/dootmaan/m2ost/blob/main/train_m2ost_example.py"
    click FLOPs "https://github.com/dootmaan/m2ost/blob/main/utils/calculateParamsFLOPs.py"
    click VisNpy "https://github.com/dootmaan/m2ost/blob/main/utils/visualization_npy2mat.py"
    click VisST "https://github.com/dootmaan/m2ost/blob/main/utils/visualization_st.py"
    click Check1 "https://github.com/dootmaan/m2ost/blob/main/weights/MSViT_Small_GSE144240_best.pth"
    click Check2 "https://github.com/dootmaan/m2ost/blob/main/weights/MSViT_Small_HER2_best.pth"
    click Check3 "https://github.com/dootmaan/m2ost/blob/main/weights/MSViT_Small_best.pth"
    click Check4 "https://github.com/dootmaan/m2ost/blob/main/weights/MSViT_small_all_best.pth"
    click Check5 "https://github.com/dootmaan/m2ost/blob/main/weights/MSViT_small_best.pth"

    %% Styles
    classDef data fill:#cfe2f3,stroke:#0366d6,color:#0366d6;
    classDef model fill:#d9ead3,stroke:#38761d,color:#38761d;
    classDef training fill:#d9ead3,stroke:#38761d,color:#38761d;
    classDef utils fill:#f9cb9c,stroke:#c55a11,color:#c55a11;
    classDef storage fill:#eee,stroke:#666,color:#666;
    classDef external fill:#ddd,stroke:#999,color:#999;
```
