# pytorch/captum architecture

> Architecture diagram of the GitHub repository pytorch/captum, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pytorch/captum
- Repository: https://github.com/pytorch/captum

## Overview

##### Project Type and Purpose
- Captum is an open-source Python library for model interpretability on top of PyTorch.
- It provides a modular collection of attribution methods (IntegratedGradients, Saliency, etc.), influence-based methods (TracIn), concept analysis (TCAV), adversarial robustness tools, and a web/UI visualization layer called Captum Insights.

##### High-Level Components
A. Core Python Package (captum/)
- attr – all attribution algorithms; subfolders for _core implementations (including saliency), _models, and _utils.
- concept – concept activation vector algorithms (TCAV).
- influence – influence functions and related utilities.
- metrics – infidelity and sensitivity metrics.
- module – stochastic gate modules.
- robust – adversarial attack and perturbation algorithms.
- insights – server and Jupyter widget; includes a Flask/FastAPI-style Python server and React frontend under attr_vis/frontend.
B. Tutorials & Examples (tutorials/) – Jupyter notebooks and data for hands-on demos.
C. Documentation (docs/, sphinx/) – static site generation source.
D. Website (website/) – Docusaurus/React site for marketing and docs hosting.
E. CI/CD & Build Scripts (scripts/, .github/) – test pipelines, linting, packaging, documentation deployment.
F. External Dependencies – PyTorch core, TorchVision/TorchText adapters, Node.js/Yarn for building Insights frontend.

##### Saliency Focus
- The Saliency method lives in captum/attr/_core/saliency.py and uses gradient utilities in captum/_utils/gradient.py and sample_gradient.py.
- It depends on the common framework in captum/attr/_core/common.py to handle inputs, baselines, target selection, and convergence deltas.

##### Relationships & Data Flow
1. User model (PyTorch nn.Module) + input tensor
↓
2. Captum Saliency component (captum.attr._core.saliency)
- Queries gradient utilities → computes gradient of output w.r.t. inputs
- Returns attribution map & optional convergence metrics
↓
3. Higher-level utilities (e.g., NoiseTunnel, LayerSaliency) can wrap Saliency for smoothing or internal layer attribution
↓
4. Optional visualization:
- Captum Insights server → REST or WebSocket API → React frontend widget
- Jupyter widget embedding via captum.insights.attr_vis.widget

##### Architectural Patterns & Design Principles
- Modular plugin-style: Each algorithm in its own module under attr/_core.
- Separation of concerns: Core compute layer (algorithms) decoupled from visualization/UI layer.
- Reusability: Common utilities for batching, gradient sampling, typing.
- Extensibility: Adding a new algorithm involves creating a new _core/<algo>.py and registering it with the top-level captum.attr namespace.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Inputs
    UserModel["User Model\n(PyTorch nn.Module)"]:::external
    InputTensor["Input Tensor"]:::external

    %% Core Package
    subgraph "Core (captum/)"
        subgraph "Attribution Algorithms\n(captum/attr/)"
            SG["Saliency\n(captum.attr._core.saliency)"]:::saliency
            CM["Common Core\n(captum.attr._core.common)"]:::core
            NoiseTunnel["NoiseTunnel Wrapper"]:::core
        end
        CAV["TCAV\n(captum.concept/)"]:::core
        Infl["Influence\n(captum.influence/)"]:::core
        Metrics["Metrics\n(captum.metrics/)"]:::core
        Modl["Module\n(captum.module/)"]:::core
        Robust["Robust\n(captum.robust/)"]:::core
    end

    %% Utilities
    subgraph "Utilities (captum/_utils/)"
        Grad["Gradient Utils\n(gradient.py)"]:::core
        SGrad["Sample Gradient\n(sample_gradient.py)"]:::core
    end

    %% Attribution Output
    Attribution["Attribution Map"]:::external

    %% Insights Subsystem
    subgraph "Insights (captum/insights)"
        Server["Python Server\n(app.py)"]:::ui
        Widget["Jupyter Widget\n(widget/)"]:::ui
        Frontend["React Frontend\n(frontend/)"]:::ui
    end

    %% Dependencies
    subgraph "External Dependencies"
        PyTorch["PyTorch"]:::external
        Torch["TorchVision/Text"]:::external
        Node["Node.js / Yarn"]:::external
    end

    %% Tutorials, Docs & CI
    subgraph "Docs & Others"
        Tutorials["Tutorials\n(tutorials/)"]:::aux
        DocsM["Docs\n(docs/)"]:::aux
        Sphinx["Sphinx\n(sphinx/source/)"]:::aux
        Website["Website\n(website/)"]:::aux
        CI["CI Workflows\n(.github/workflows/)"]:::aux
        Scripts["Scripts\n(scripts/)"]:::aux
    end

    %% Data Flows
    UserModel -->|"model & config"| SG
    InputTensor -->|"input tensor"| SG
    SG -->|"uses"| CM
    SG -->|"calls"| Grad
    SG -->|"calls"| SGrad
    SG -->|"wrapped by"| NoiseTunnel
    SG -->|"computes"| Attribution
    Attribution -->|"REST/WebSocket API"| Server
    Server -->|"serves data"| Frontend
    Widget -->|"embeds"| Frontend

    PyTorch -->|"provides autograd"| SG
    Torch -->|"adapters"| SG
    Node -->|"builds frontend"| Frontend

    %% Click Events
    click SG "https://github.com/pytorch/captum/blob/master/captum/attr/_core/saliency.py"
    click CM "https://github.com/pytorch/captum/blob/master/captum/attr/_core/__init__.py"
    click Grad "https://github.com/pytorch/captum/blob/master/captum/_utils/gradient.py"
    click SGrad "https://github.com/pytorch/captum/blob/master/captum/_utils/sample_gradient.py"
    click CAV "https://github.com/pytorch/captum/tree/master/captum/concept/"
    click Infl "https://github.com/pytorch/captum/tree/master/captum/influence/"
    click Metrics "https://github.com/pytorch/captum/tree/master/captum/metrics/"
    click Modl "https://github.com/pytorch/captum/tree/master/captum/module/"
    click Robust "https://github.com/pytorch/captum/tree/master/captum/robust/"
    click Server "https://github.com/pytorch/captum/blob/master/captum/insights/attr_vis/app.py"
    click Frontend "https://github.com/pytorch/captum/tree/master/captum/insights/attr_vis/frontend/"
    click Widget "https://github.com/pytorch/captum/tree/master/captum/insights/attr_vis/widget/"
    click Tutorials "https://github.com/pytorch/captum/tree/master/tutorials/"
    click DocsM "https://github.com/pytorch/captum/tree/master/docs/"
    click Sphinx "https://github.com/pytorch/captum/tree/master/sphinx/source/"
    click Website "https://github.com/pytorch/captum/tree/master/website/"
    click CI "https://github.com/pytorch/captum/tree/master/.github/workflows/"
    click Scripts "https://github.com/pytorch/captum/tree/master/scripts/"

    %% Styles
    classDef core fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef saliency fill:#FFA500,stroke:#333,stroke-width:1px
    classDef ui fill:#98FB98,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef aux fill:#EEEEEE,stroke:#333,stroke-width:1px
```
