# lauffenburger-lab/lembas architecture

> Architecture diagram of the GitHub repository lauffenburger-lab/lembas, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lauffenburger-lab/lembas
- Repository: https://github.com/lauffenburger-lab/lembas

## Overview

##### Project type and purpose
- Type: A scientific software pipeline / open‐source research tool for genome‐scale simulation of intracellular signaling.
- Purpose: Ingest signaling data, reconstruct prior‐knowledge networks, train recurrent neural nets constrained by that network, simulate perturbations (e.g., knockouts), and generate publication‐quality figures.

##### File structure
- Top‐level boundaries:
– Demo/: Minimal example pipeline.
– Model/: Core Python modeling code, data, pretrained models, scripts to run cross‐validation, ODE simulation, toy networks, etc.
– “Network Construction”/: Scripts (Python & MATLAB) to build the knowledge‐embedded network from OmniPath.
– “TF activities…” and “TF activities Ligand Screen”/: Pre‐ and postprocessing code (MATLAB, R, Python) to estimate transcription‐factor activities and convert into ASN format.
– “Viability data”/: Extraction scripts for CCLE viability data.
– “complexity tests”/: MATLAB performance profiling tests.
- Configuration & orchestration:
– downloadLiteratureData.py: data‐acquisition entry point.
– runSlurm*.sh: cluster orchestration on SLURM for large‐scale training.
- Supporting files: requirements implied in README for Python (PyTorch, numpy, pandas, scipy, matplotlib, seaborn), MATLAB, R (dorothea, DESeq2, limma).

##### README for architecture insights
- Four main stages described: data download, TF activity estimation, network reconstruction, model training & simulation to produce figures.
- Dependencies are clearly partitioned (Python vs. MATLAB vs. R).
- Figure‐by‐figure breakdown reveals which scripts belong in which stage.

##### Main components & their interactions
a. Data Ingestion
– downloadLiteratureData.py
– External sources: OmniPath API, public transcriptomics repositories, CCLE.
b. TF Activity Estimation
– MATLAB scripts: parseCondition.m, joinData.m
– R scripts: macrophage.R, run.R (DESeq2, limma, dorothea)
– Python converters: convertToASNFormat.py, constructASNLigandMatrix.py
c. Network Reconstruction
– Python: extractRL.py, extractPKN.py, trimKeggModel.py, trimLigandModel.py, trimMacrophageModel.py
– MATLAB: drawNetwork.m, drawSubNetwork.m
– Outputs: TSV network model files in “Network Construction/model/”
d. Model Implementation & Core Library
– Python modules: Model/bionetwork.py, activationFunctions.py, calculateODEvalues.py, ODEsimulation.py
– Utility: plotting.py
e. Model Training & Evaluation
– Python training scripts: ligandScreenCrossValidation.py, macrophageNetCrossValidation.py, viabilityNet.py, synthNet…
– Cross‐validation & sensitivity: ligandScreenCrossValidationResults.py, macrophageNetCrossValidationResults.py, viabilityNetResults.py
– Slurm scripts: runSlurmMacrophage.sh, runSlurmSynthScreen.sh, etc.
– Pretrained model artifacts: .pt files under Model/CVligandScreen, Model/CVmacrophage, Model/synthNetScreen, etc.
f. Simulation & Analysis
– Toy network experiments: toyNetRecurrent.py, toyNetNFKB.py, pipeNet.py
– ODE fits & forward/backward solves: fitODE.py, displayODEvalues.py, showActivationFunction.py
– Knockout & sensitivity analyses: ligandScreenKO.py, macrophageNetKO.py, synthNetKO.py
g. Visualization & Figure Generation
– Scripts in Model/figures and Demo/src/plotting.py
– Demo: Demo/main.py uses a small network example to illustrate pipeline end‐to‐end.
h. Execution Environment & Orchestration
– Python 3.7 + listed libraries
– MATLAB R2020a for network viz & complexity tests
– R 4.0.3 + dorothea, DESeq2, limma
– SLURM for cluster batch runs

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Ingestion"
        DL["downloadLiteratureData.py (Python)"]:::python
        OmniPath["OmniPath API"]:::external
        CCLE["CCLE"]:::external
        DL --> OmniPath
        DL --> CCLE
    end

    subgraph "Viability Data Extraction"
        ECL["extractCellLineData.py (Python)"]:::python
        ED["extractData.m (MATLAB)"]:::matlab
        CCLE --> ECL
        ECL --> ED
    end

    subgraph "TF Activity Estimation"
        subgraph "MATLAB Estimation"
            PC["parseCondition.m (MATLAB)"]:::matlab
            JD["joinData.m (MATLAB)"]:::matlab
            PC --> JD
        end
        subgraph "R Estimation"
            MACR["macrophage.R (R)"]:::r
            RUNR["run.R (R)"]:::r
            MACR --> RUNR
        end
        subgraph "ASN Conversion"
            C2A1["convertToASNFormat.py (Python)"]:::python
            CAM1["constructASNLigandMatrix.py (Python)"]:::python
            C2A2["convertToASNFormat.py (Python)"]:::python
            CAM2["constructASNLigandMatrix.py (Python)"]:::python
            PC -.-> C2A1
            JD -.-> C2A1
            RUNR -.-> C2A2
            C2A1 --> CAM1
            C2A2 --> CAM2
        end
        OmniPath --> PC
        CCLE --> MACR
    end

    subgraph "Network Reconstruction"
        ERL["extractRL.py (Python)"]:::python
        EPKN["extractPKN.py (Python)"]:::python
        TK["trimKeggModel.py (Python)"]:::python
        TL["trimLigandModel.py (Python)"]:::python
        TM["trimMacrophageModel.py (Python)"]:::python
        DN["drawNetwork.m (MATLAB)"]:::matlab
        DSN["drawSubNetwork.m (MATLAB)"]:::matlab
        ERL --> EPKN --> TK --> TL --> TM --> DN --> DSN
        subgraph "Network TSV Models"
            TSV["model/ (TSV files)"]:::datastore
        end
        DSN --> TSV
        CAM1 --> ERL
        CAM2 --> ERL
    end

    subgraph "Core Library & Model Implementation"
        BN["bionetwork.py (Python)"]:::python
        AF["activationFunctions.py (Python)"]:::python
        CO["calculateODEvalues.py (Python)"]:::python
        OS["ODEsimulation.py (Python)"]:::python
        DO["displayODEvalues.py (Python)"]:::python
        FI["fitODE.py (Python)"]:::python
        SA["showActivationFunction.py (Python)"]:::python
        PLT["plotting.py (Python)"]:::python
    end

    TSV --> BN
    BN --> AF --> CO --> OS --> DO --> FI --> SA
    PLT --> SA

    subgraph "Model Training & Evaluation"
        LSCV["ligandScreenCrossValidation.py (Python/PyTorch)"]:::python
        LSCVR["ligandScreenCrossValidationResults.py (Python)"]:::python
        MCV["macrophageNetCrossValidation.py (Python/PyTorch)"]:::python
        MCVR["macrophageNetCrossValidationResults.py (Python)"]:::python
        VNet["viabilityNet.py (Python/PyTorch)"]:::python
        VNTR["viabilityNetResults.py (Python)"]:::python
        GSN["generateSynthNet.py (Python)"]:::python
        SMM["runSlurmMacrophage.sh (Cluster)"]:::cluster
        SSS["runSlurmSynthScreen.sh (Cluster)"]:::cluster
        SMS["runSlurmMacrophageScramble.sh (Cluster)"]:::cluster
        CSVS["CVligandScreen/ (.pt)"]:::datastore
        CVM["CVmacrophage/ (.pt)"]:::datastore
        SSScr["synthNetScreen/ (.pt)"]:::datastore
        SST["synthNetTime/ (.pt)"]:::datastore
        VMCV["viabilityModel/CV/ (.pt)"]:::datastore

        BN --> LSCV
        BN --> MCV
        BN --> VNet
        BN --> GSN
        LSCV --> LSCVR --> CSVS
        MCV --> MCVR --> CVM
        VNet --> VNTR --> VMCV
        GSN --> SSScr
        GSN --> SST
        LSCV --> SMM
        MCV --> SMS
        GSN --> SSS
    end

    subgraph "Simulation & Sensitivity Analysis"
        TNR["toyNetRecurrent.py (Python)"]:::python
        TNK["toyNetNFKB.py (Python)"]:::python
        PN["pipeNet.py (Python)"]:::python
        LKO["ligandScreenKO.py (Python)"]:::python
        MKO["macrophageNetKO.py (Python)"]:::python
        SKO["synthNetKO.py (Python)"]:::python
        LSens["ligandScreenSensitivity.py (Python)"]:::python

        CSVS --> TNR --> TNK --> PN
        CSVS --> LKO --> LSens
        CVM --> MKO
        SSScr --> SKO
    end

    subgraph "Visualization & Demo"
        Figs["figures/"]:::datastore
        DPlot["plotting.py (Demo/Python)"]:::python
        DMain["main.py (Demo/Python)"]:::python

        PN --> DPlot --> DMain
        LSens --> DPlot
    end

    classDef python fill:#abf0c9,stroke:#036c32,stroke-width:1px;
    classDef matlab fill:#f0b97a,stroke:#b35f00,stroke-width:1px;
    classDef r fill:#a0cfff,stroke:#004aab,stroke-width:1px;
    classDef external fill:#e2e2e2,stroke:#7a7a7a,stroke-width:1px,stroke-dasharray: 5 5;
    classDef datastore fill:#ffd27f,stroke:#b37400,stroke-width:1px,stroke-dasharray: 2 2;
    classDef cluster fill:#d3d3d3,stroke:#666,stroke-width:1px,stroke-dasharray: 3 3;

    click DL "https://github.com/lauffenburger-lab/lembas/blob/main/downloadLiteratureData.py"
    click PC "https://github.com/lauffenburger-lab/lembas/blob/main/TF activities/parseCondition.m"
    click JD "https://github.com/lauffenburger-lab/lembas/blob/main/TF activities/joinData.m"
    click MACR "https://github.com/lauffenburger-lab/lembas/blob/main/TF activities/macrophage.R"
    click RUNR "https://github.com/lauffenburger-lab/lembas/blob/main/TF activities Ligand Screen/run.R"
    click C2A1 "https://github.com/lauffenburger-lab/lembas/blob/main/TF activities/convertToASNFormat.py"
    click CAM1 "https://github.com/lauffenburger-lab/lembas/blob/main/TF activities/constructASNLigandMatrix.py"
    click C2A2 "https://github.com/lauffenburger-lab/lembas/blob/main/TF activities Ligand Screen/convertToASNFormat.py"
    click CAM2 "https://github.com/lauffenburger-lab/lembas/blob/main/TF activities Ligand Screen/constructASNLigandMatrix.py"
    click ERL "https://github.com/lauffenburger-lab/lembas/blob/main/Network Construction/extractRL.py"
    click EPKN "https://github.com/lauffenburger-lab/lembas/blob/main/Network Construction/extractPKN.py"
    click TK "https://github.com/lauffenburger-lab/lembas/blob/main/Network Construction/trimKeggModel.py"
    click TL "https://github.com/lauffenburger-lab/lembas/blob/main/Network Construction/trimLigandModel.py"
    click TM "https://github.com/lauffenburger-lab/lembas/blob/main/Network Construction/trimMacrophageModel.py"
    click DN "https://github.com/lauffenburger-lab/lembas/blob/main/Network Construction/drawNetwork.m"
    click DSN "https://github.com/lauffenburger-lab/lembas/blob/main/Network Construction/drawSubNetwork.m"
    click BN "https://github.com/lauffenburger-lab/lembas/blob/main/Model/bionetwork.py"
    click AF "https://github.com/lauffenburger-lab/lembas/blob/main/Model/activationFunctions.py"
    click CO "https://github.com/lauffenburger-lab/lembas/blob/main/Model/calculateODEvalues.py"
    click OS "https://github.com/lauffenburger-lab/lembas/blob/main/Model/ODEsimulation.py"
    click DO "https://github.com/lauffenburger-lab/lembas/blob/main/Model/displayODEvalues.py"
    click FI "https://github.com/lauffenburger-lab/lembas/blob/main/Model/fitODE.py"
    click SA "https://github.com/lauffenburger-lab/lembas/blob/main/Model/showActivationFunction.py"
    click PLT "https://github.com/lauffenburger-lab/lembas/blob/main/Model/plotting.py"
    click LSCV "https://github.com/lauffenburger-lab/lembas/blob/main/Model/ligandScreenCrossValidation.py"
    click LSCVR "https://github.com/lauffenburger-lab/lembas/blob/main/Model/ligandScreenCrossValidationResults.py"
    click MCV "https://github.com/lauffenburger-lab/lembas/blob/main/Model/macrophageNetCrossValidation.py"
    click MCVR "https://github.com/lauffenburger-lab/lembas/blob/main/Model/macrophageNetCrossValidationResults.py"
    click VNet "https://github.com/lauffenburger-lab/lembas/blob/main/Model/viabilityNet.py"
    click VNTR "https://github.com/lauffenburger-lab/lembas/blob/main/Model/viabilityNetResults.py"
    click GSN "https://github.com/lauffenburger-lab/lembas/blob/main/Model/generateSynthNet.py"
    click SMM "https://github.com/lauffenburger-lab/lembas/blob/main/Model/runSlurmMacrophage.sh"
    click SSS "https://github.com/lauffenburger-lab/lembas/blob/main/Model/runSlurmSynthScreen.sh"
    click SMS "https://github.com/lauffenburger-lab/lembas/blob/main/Model/runSlurmMacrophageScramble.sh"
    click ECL "https://github.com/lauffenburger-lab/lembas/blob/main/Viability data/extractCellLineData.py"
    click ED "https://github.com/lauffenburger-lab/lembas/blob/main/Viability data/extractData.m"
    click TNR "https://github.com/lauffenburger-lab/lembas/blob/main/Model/toyNetRecurrent.py"
    click TNK "https://github.com/lauffenburger-lab/lembas/blob/main/Model/toyNetNFKB.py"
    click PN "https://github.com/lauffenburger-lab/lembas/blob/main/Model/pipeNet.py"
    click LKO "https://github.com/lauffenburger-lab/lembas/blob/main/Model/ligandScreenKO.py"
    click MKO "https://github.com/lauffenburger-lab/lembas/blob/main/Model/macrophageNetKO.py"
    click SKO "https://github.com/lauffenburger-lab/lembas/blob/main/Model/synthNetKO.py"
    click LSens "https://github.com/lauffenburger-lab/lembas/blob/main/Model/ligandScreenSensitivity.py"
    click Figs "https://github.com/lauffenburger-lab/lembas/tree/main/Model/figures/"
    click DPlot "https://github.com/lauffenburger-lab/lembas/blob/main/Demo/src/plotting.py"
    click DMain "https://github.com/lauffenburger-lab/lembas/blob/main/Demo/main.py"
```
