# icb-dcm/pypesto architecture

> Architecture diagram of the GitHub repository icb-dcm/pypesto, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/icb-dcm/pypesto
- Repository: https://github.com/icb-dcm/pypesto

## Overview

##### Main Components
- Core Engine & Task Management:
– Represent the “engine” module which includes different processing strategies (single core, multi-thread, multi-process, MPI pool).
– Include task management for simulations and optimizations.
- Objective & Simulator Interfaces:
– Show the “objective” layer that abstracts simulation models. Indicate the plug-in interfaces for different simulators (AMICI, RoadRunner, Jax, Julia).
- Optimization & Sampling:
– Present the “optimize” module with the various optimization algorithms.
– Include the “sample” submodule, noting the different sampling techniques (MCMC, parallel tempering, etc.).
- Hierarchical & Ensemble Methods:
– Represent the “hierarchical” module which handles specialized parameter estimation (e.g., ordinal, relative, semiquantitative data).
- Result Processing & Visualization:
- Input/Configuration & External Integration:
– Show a component for input data and configuration (supporting SBML/PEtab integration as outlined in the README).
– Include external dependencies and integrations (links to external libraries such as AMICI, RoadRunner, Jax, Julia, and interfaces to external samplers such as emcee, pymc, dynesty).

##### Relationships and Data Flow
– Next, it flows into the objective layer where simulation and sensitivity analysis are defined.
– Results and gradient information are passed on to the core engine and optimization modules.
– The optimization module sends tasks through the engine’s strategies (single core, multi-threaded, etc.), which then update the history/tracking module.
– The outputs are then fed into both the result processing and visualization modules.
- Dependency or invocation relationships between modules (e.g., the “engine” calling methods in “objective”; the “optimize” component utilizing functionality from “startpoint” or “ensemble”).

##### Important Architectural Patterns and Design Principles
- Modular Design: Stress how each module (engine, objective, optimization, etc.) is largely self-contained with clearly defined interfaces.
- Extensibility: Highlight the plug-in interfaces for various simulation backends and sampling methods.
- Scalability: Note the multi-processing/multi-threading and MPI approaches for handling different computational loads.
- Separation of concerns: Data ingestion, simulation, optimization, uncertainty quantification, and visualization are distinct layers with dedicated modules.

##### Relevant Technologies and Frameworks
- Python as the core language.
- External libraries such as AMICI, RoadRunner, Jax, and Julia are critical parts integrated in the objective and simulation components.
- Optimization libraries and sampling methodologies (emcee, pymc, dynesty) should be mentioned where they interface.
- Documentation and testing showcase robust development practices (using tools like PyTest, tox, and continuous integration workflows as seen in the .github directory).

##### Elements
- Annotations for external integrations (AMICI, RoadRunner, Jax, Julia, external samplers).
– Core modules (processing/simulation),
– Auxiliary modules (configuration, logging, visualization),
– External integrations.
- Detailed labels to capture the underlying design intent and dependencies.

Happy diagramming!

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Input & Configuration"
        PETAB["SBML/PEtab Integration (petab)"]:::preprocessing
    end

    subgraph "Simulation & Objective"
        OBJECTIVE["Objective Interface (AMICI, Jax, Julia, RoadRunner)"]:::simulation
    end

    subgraph "Core Engine & Task Management"
        ENGINE["Computational Engine (single/multi/MPI)"]:::core
    end

    subgraph "Optimization & Sampling"
        OPT["Optimization (algorithms & tasks)"]:::optimization
        SAMPLING["Sampling (MCMC, parallel-tempering, etc.)"]:::optimization
        HIER["Hierarchical Methods (ordinal/relative/semiquantitative)"]:::optimization
        ENSEMBLE["Ensemble Methods (covariance, dimension reduction)"]:::optimization
    end

    subgraph "Result Processing & Visualization"
        RESULT["Result Processing (optimize, predict, profile, sample)"]:::result
        VIS["Visualization (optimize_history, profiles, sampling)"]:::result
        PREDICT["Prediction (e.g., Amici predictor)"]:::result
    end

    %% Data Flow Connections
    PETAB -->|"preprocess"| OBJECTIVE
    OBJECTIVE -->|"simulation_call"| ENGINE
    ENGINE -->|"dispatch_opt"| OPT
    ENGINE -->|"dispatch_sample"| SAMPLING
    ENGINE -->|"dispatch_hier"| HIER
    ENGINE -->|"dispatch_ensemble"| ENSEMBLE
    OPT -->|"post_process"| RESULT
    SAMPLING -->|"post_process"| RESULT
    HIER -->|"post_process"| RESULT
    ENSEMBLE -->|"post_process"| RESULT
    RESULT -->|"visualize_results"| VIS
    RESULT -->|"predict_outcomes"| PREDICT

    %% Click Events for Components
    click PETAB "https://github.com/icb-dcm/pypesto/blob/main/pypesto/petab/importer.py"
    click OBJECTIVE "https://github.com/icb-dcm/pypesto/blob/main/pypesto/objective/base.py"
    click ENGINE "https://github.com/icb-dcm/pypesto/tree/main/pypesto/engine/"
    click OPT "https://github.com/icb-dcm/pypesto/blob/main/pypesto/optimize/optimize.py"
    click SAMPLING "https://github.com/icb-dcm/pypesto/blob/main/pypesto/sample/sample.py"
    click HIER "https://github.com/icb-dcm/pypesto/tree/main/pypesto/hierarchical/"
    click ENSEMBLE "https://github.com/icb-dcm/pypesto/tree/main/pypesto/ensemble/"
    click RESULT "https://github.com/icb-dcm/pypesto/blob/main/pypesto/result/result.py"
    click VIS "https://github.com/icb-dcm/pypesto/blob/main/pypesto/visualize/optimize_history.py"
    click PREDICT "https://github.com/icb-dcm/pypesto/blob/main/pypesto/predict/amici_predictor.py"

    %% Styling for different layers
    classDef preprocessing fill:#cfc,stroke:#333,stroke-width:2px;
    classDef simulation fill:#fcf,stroke:#333,stroke-width:2px;
    classDef core fill:#fdd,stroke:#333,stroke-width:2px;
    classDef optimization fill:#cff,stroke:#333,stroke-width:2px;
    classDef result fill:#ddf,stroke:#333,stroke-width:2px;
```
