# selfexplainml/piml-toolbox architecture

> Architecture diagram of the GitHub repository selfexplainml/piml-toolbox, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/selfexplainml/piml-toolbox
- Repository: https://github.com/selfexplainml/piml-toolbox

## Overview

##### Main Components
- Experiment Controller/Orchestrator – the part that initializes experiments, loads data, directs the flow from data preprocessing to modeling, explanation, diagnosis, and comparison.
- Data Management Layer – includes data loader, data summary, data preparation, data quality assessment, feature selection, and exploratory data analysis (EDA). This is usually invoked as a sequence within an “exp.data_loader()”, “exp.data_prepare()”, etc.
- Model Explanation and Interpretation Components – incorporate global and local interpretability tools (PFI, PDP, ALE, LIME, SHAP) to elucidate model behavior.
- Model Diagnostics & Comparison Module – includes mechanisms to measure accuracy, assess fairness, detect overfitting, and evaluate robustness, reliability, and resilience of the models.
- User Interface Layer – leveraging notebooks and a low-code interface coupled with a high-code API, plus an interactive dashboard for dynamic visualizations (evidenced by the variety of example notebooks and plot images in the docs).

##### Relationships and Data Flow
- The Data Management Layer passes clean, preprocessed data to the Model Training Module.
- Post training, the results flow into the Model Explanation/Interpretation and then into the Diagnostics/Comparison modules.
- The outputs of each stage (such as plots, summaries, diagnostic reports) are displayed in the user interface (often via notebooks or interactive visualizations), thereby completing the pipeline.

##### Architectural Patterns and Design Principles
- Pipeline Architecture: This project follows a pipeline structure where each stage (data handling, model training, interpretation, diagnosis) feeds into the next.
- Modular Design: Each major function (data handling, training, explanation, diagnostics) is separated into dedicated modules making the system extensible and scalable.
- Dual API Paradigm: With both low-code and high-code operating models, the design strategy supports different user levels.

##### Relevant Technologies and Artifacts
- Python as the core language.
- Jupyter Notebooks serving as both use case examples and interactive demonstration environments.
- Sphinx and related documentation generation (visible in the many docs/_build/ directories).
- Pip for installation and package management.
- Visualization packages (as indicated by the variety of plot images and interactive visualizations).

##### Elements
- Annotations that reference integration points with external libraries and documentation assets (e.g., “Sphinx Docs”, “Jupyter Notebooks”, “Colab” links).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "ML Pipeline"
        EC["Experiment Controller"]:::controller
        DM["Data Management Layer"]:::data
        MT["Model Training Module"]:::training
        ME["Model Explanation/Interpretation"]:::explain
        MD["Diagnostics & Comparison"]:::diagnostics
        UI["User Interface Layer"]:::ui
    end

    EC -->|"initiates"| DM
    DM -->|"prepares_for"| MT
    MT -->|"triggers"| ME
    ME -->|"feeds_into"| MD
    MD -->|"renders_on"| UI
    MT -->|"wraps_external"| MD

    click EC "https://github.com/selfexplainml/piml-toolbox/blob/main/PiML Low-code Example Run.ipynb"
    click EC "https://github.com/selfexplainml/piml-toolbox/blob/main/docs/_build/doctrees/modules/generated/piml.Experiment.doctree"
    
    click DM "https://github.com/selfexplainml/piml-toolbox/tree/main/datasets"
    click DM "https://github.com/selfexplainml/piml-toolbox/tree/main/docs/_build/doctrees/auto_examples/0_data"
    
    click MT "https://github.com/selfexplainml/piml-toolbox/tree/main/docs/_build/doctrees/auto_examples/1_model_train"
    click MT "https://github.com/selfexplainml/piml-toolbox/tree/main/docs/_build/doctrees/auto_examples/1_train"
    click MT "https://github.com/selfexplainml/piml-toolbox/tree/main/examples"
    
    click ME "https://github.com/selfexplainml/piml-toolbox/tree/main/docs/_build/doctrees/auto_examples/2_explain"
    click ME "https://github.com/selfexplainml/piml-toolbox/tree/main/docs/Workshop"
    
    click MD "https://github.com/selfexplainml/piml-toolbox/tree/main/docs/_build/doctrees/auto_examples/4_testing"
    click MD "https://github.com/selfexplainml/piml-toolbox/tree/main/docs/_build/doctrees/auto_examples/5_compare"
    click MD "https://github.com/selfexplainml/piml-toolbox/tree/main/examples"
    
    click UI "https://github.com/selfexplainml/piml-toolbox/tree/main/docs/Workshop"
    click UI "https://github.com/selfexplainml/piml-toolbox/tree/main/examples"
    click UI "https://github.com/selfexplainml/piml-toolbox/blob/main/PiML Low-code Example Run.ipynb"
    
    classDef controller fill:#FFD700,stroke:#333,stroke-width:2px;
    classDef data fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef training fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef explain fill:#FFA07A,stroke:#333,stroke-width:2px;
    classDef diagnostics fill:#DDA0DD,stroke:#333,stroke-width:2px;
    classDef ui fill:#F08080,stroke:#333,stroke-width:2px;
```
