# saezlab/pypath architecture

> Architecture diagram of the GitHub repository saezlab/pypath, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/saezlab/pypath
- Repository: https://github.com/saezlab/pypath

## Overview

##### Project type and overall purpose
- **Type**: Open-source Python library + CLI + optional web service.
- **Purpose**: Download, preprocess, integrate, and expose molecular biology resources (OmniPath databases, ID translation, homology, etc.) to Python programs, scripts, web clients, and external tools (R client, Cytoscape app, BEL exporter).

##### Break down the main architectural layers/components
- **Inputs Layer**
– Package: `pypath.inputs`
– Contains ~150 resource-specific modules (e.g. `biogrid.py`, `chembl`, `disgenet/_api`, etc.)
– Responsibilities: Download raw data from external sources (HTTP, FTP, REST), parse, clean, normalize, output standardized records.

- **Data & Cache Layer**
– Packages: `pypath/share/cache.py`, `pypath/share/session.py`, `~/.pypath/pickles/`
– Responsibilities: Persist downloaded raw data, intermediate pickles, sqlite/SQL dumps; avoid repeated downloads; manage singleton instances (`get_db`).

- **Core Database Layer**
– Package: `pypath.core`
– Key classes: `network`, `enz_sub`, `complex`, `annot`, `intercell`
– Responsibilities: Take cleaned records, build in-memory data structures (igraph graphs, pandas DataFrames); provide API to query integrated databases.

- **Utility Layer**
– Package: `pypath/utils` and `pypath/formats`, `pypath/reader`
– Responsibilities: Standalone utilities (ID mapping, homology translation, GO processing, format I/O: OBO, XML, SDF, SQL dump, SQLite), readers for generic field and network operations.

- **OmniPath Application Layer**
– Package: `pypath.omnipath`
- `omnipath.db`: database manager — definitions of pickles, singleton pattern, building/loading logic.
- `omnipath/app.py`, `omnipath/param.py`, `omnipath/export.py`, `bel.py`, `cellphonedb.py`, etc.
– Subfolder `omnipath/server`: Sphinx-documented server generator & runner (`run.py`), HTML templates
– Responsibilities: Higher-level workflows:
- Build complete OmniPath databases
- Expose web service API
- Export to BEL or specialized formats
- Provide CLI scripts (`scripts/server-runner.py`, `scripts/static-tables.py`, etc.)

- **Resources & Documentation Layer**
– Folders: `docs/`, `pypath/resources/`, `README.rst`, Sphinx config
– Responsibilities: Build reference docs, tutorials (Jupyter notebooks), API reference, manual.

- **External Clients & Integrations**
– R: OmnipathR
– Cytoscape app
– BEL
– Direct Python import
– REST clients to `pypath.omnipath.server`

- **Test & CI/CD**
– Folder: `test/`
– GitHub Actions: `.github/workflows/sphinx-build.yml`
– Responsibilities: Unit tests for inputs, settings; doc build; version bump via `.bumpversion.cfg`.

##### Component relationships and data flows
1. **External Resources** → **Inputs Layer** (download & parse)
2. **Inputs Layer** → **Data & Cache Layer** (cache raw/intermediate files)
3. **Data & Cache Layer** → **Core Database Layer** (load/preprocess into graphs/DataFrames)
4. **Core Database Layer** → **OmniPath Application Layer**  (DB manager picks classes from core)
5. **OmniPath Application Layer** →
a. **Python API / CLI Scripts** (user scripts, `scripts/*.py`)
b. **Web Service** (REST endpoints) → **External Clients**
c. **Exports** (BEL, Cytoscape, static tables)

– `pypath.utils` is called by **Inputs**, **Core**, and **OmniPath** layers wherever mapping, homology, or format conversions are needed.
– `pypath.reader` assists **Core** in graph/field abstraction.

##### Architectural patterns & design principles
- **Modular Monolith**: Well-isolated subpackages (`core`, `inputs`, `utils`, `omnipath`) but single codebase.
- **Plugin-style Inputs**: Each data source is a self-contained module following a common interface (`input_module.py`), enabling extensibility.
- **Singleton / Factory**: `get_db` methods and `omnipath.db` manage single instances of large database objects cached on disk.
- **Cache-as-Database**: Raw downloads and built pickles form a local data store to speed up rebuilds.
- **Separation of Concerns**:
- I/O & parsing in `inputs`
- Integration & in-memory models in `core`
- High-level workflows & APIs in `omnipath`
- Utilities in `utils`

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Resources
    ER["External Resources"]:::externalStyle

    %% Inputs Layer
    subgraph "Inputs Layer" 
        direction TB
        Inputs["pypath.inputs"]:::ioStyle
    end

    %% Data & Cache Layer
    subgraph "Data & Cache Layer"
        direction TB
        CachePy["cache.py"]:::ioStyle
        SessionPy["session.py"]:::ioStyle
        Pickles["~/.pypath/pickles/"]:::storageStyle
    end

    %% Core Database Layer
    subgraph "Core Database Layer"
        direction TB
        CorePkg["pypath.core"]:::coreStyle
    end

    %% Utility Layer
    subgraph "Utility Layer"
        direction TB
        UtilsPkg["pypath/utils"]:::utilStyle
        FormatsPkg["pypath/formats"]:::utilStyle
        ReaderPkg["pypath/reader"]:::utilStyle
    end

    %% OmniPath Application Layer
    subgraph "OmniPath Application Layer"
        direction TB
        OmniPkg["pypath/omnipath"]:::coreStyle
        subgraph "Server"
            direction TB
            ServerPkg["pypath/omnipath/server"]:::coreStyle
        end
        ScriptsRunner["scripts/server-runner.py"]:::coreStyle
        StaticTables["scripts/static-tables.py"]:::coreStyle
    end

    %% Resources & Documentation Layer
    subgraph "Resources & Documentation"
        direction TB
        Docs["docs/"]:::docStyle
        Resources["pypath/resources"]:::docStyle
        Readme["README.rst"]:::docStyle
    end

    %% Test & CI/CD
    subgraph "Test & CI/CD"
        direction TB
        Tests["test/"]:::docStyle
        CI[".github/workflows/sphinx-build.yml"]:::docStyle
    end

    %% External Clients
    subgraph "External Clients"
        direction TB
        RClient["OmniPathR"]:::externalStyle
        Cytoscape["Cytoscape App"]:::externalStyle
        BELExp["BEL Exporter"]:::externalStyle
        PyImport["Python Import"]:::externalStyle
        RESTClient["REST Clients"]:::externalStyle
    end

    %% Data Flows
    ER -->|"HTTP/FTP/REST downloads"| Inputs
    Inputs -->|"caches raw/intermediate"| CachePy
    CachePy -->|"stores pickles"| Pickles
    CachePy -->|"stores session"| SessionPy
    CachePy -->|"loads data"| CorePkg
    CorePkg -->|"builds DB"| OmniPkg

    %% OmniPath outputs
    OmniPkg -->|"CLI & Python API"| PyImport
    OmniPkg -->|"runs server"| ServerPkg
    ServerPkg -->|"REST API"| RESTClient
    OmniPkg -->|"exports BEL"| BELExp
    OmniPkg -->|"exports for Cytoscape"| Cytoscape

    %% Utility interactions
    UtilsPkg -.->|"util functions"| Inputs
    UtilsPkg -.->|"util functions"| CorePkg
    UtilsPkg -.->|"util functions"| OmniPkg
    FormatsPkg -.->|"format I/O"| Inputs
    FormatsPkg -.->|"format I/O"| CorePkg
    ReaderPkg -.->|"field/network readers"| CorePkg

    %% Resources and docs
    Readme -.->|"documentation"| Docs
    Resources -.->|"resources"| Docs

    %% Test & CI/CD interactions
    Tests -.->|"unit tests"| Inputs
    Tests -.->|"unit tests"| CorePkg
    CI -.->|"doc build"| Docs

    %% External extensibility
    subgraph "Plugin-style Inputs"
        direction TB
        NewInput["pypath.inputs.my_resource"]:::ioStyle
    end
    NewInput -->|"plugs into"| Inputs

    %% Styling
    classDef ioStyle fill:#a8ddb5,stroke:#333,stroke-width:1px
    classDef coreStyle fill:#74c476,stroke:#333,stroke-width:1px
    classDef utilStyle fill:#fd8d3c,stroke:#333,stroke-width:1px
    classDef storageStyle fill:#31a354,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
    classDef docStyle fill:#bdbdbd,stroke:#333,stroke-width:1px
    classDef externalStyle fill:#6a51a3,stroke:#333,stroke-width:1px

    %% Click Events
    click Inputs "https://github.com/saezlab/pypath/tree/master/pypath/inputs/"
    click CachePy "https://github.com/saezlab/pypath/blob/master/pypath/share/cache.py"
    click SessionPy "https://github.com/saezlab/pypath/blob/master/pypath/share/session.py"
    click CorePkg "https://github.com/saezlab/pypath/tree/master/pypath/core/"
    click UtilsPkg "https://github.com/saezlab/pypath/tree/master/pypath/utils/"
    click FormatsPkg "https://github.com/saezlab/pypath/tree/master/pypath/formats/"
    click ReaderPkg "https://github.com/saezlab/pypath/tree/master/pypath/reader/"
    click OmniPkg "https://github.com/saezlab/pypath/tree/master/pypath/omnipath/"
    click ServerPkg "https://github.com/saezlab/pypath/tree/master/pypath/omnipath/server/"
    click ScriptsRunner "https://github.com/saezlab/pypath/blob/master/scripts/server-runner.py"
    click StaticTables "https://github.com/saezlab/pypath/blob/master/scripts/static-tables.py"
    click Docs "https://github.com/saezlab/pypath/tree/master/docs/"
    click Resources "https://github.com/saezlab/pypath/tree/master/pypath/resources/"
    click Readme "https://github.com/saezlab/pypath/blob/master/README.rst"
    click Tests "https://github.com/saezlab/pypath/tree/master/test/"
    click CI "https://github.com/saezlab/pypath/blob/master/.github/workflows/sphinx-build.yml"
```
