# inikishev/torchzero architecture

> Architecture diagram of the GitHub repository inikishev/torchzero, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/inikishev/torchzero
- Repository: https://github.com/inikishev/torchzero

## Overview

##### Main Components Structure

a) Core Layer:
- `module.py` and `tensorlist_optimizer.py` as fundamental components

b) Optimization Modules Layer:
- First Order Optimizers (SGD, Adam, etc.)
- Second Order Optimizers (Newton methods)
- Zeroth Order Optimizers (FDM-based)
- Quasi-Newton Methods
- Line Search Algorithms
- Regularization Components
- Momentum Components
- Gradient Approximation Methods

c) Integration Layer:
- External Wrappers (scipy, nevergrad)
- Meta Components (chains, grafting)
- Experimental Features

##### Key Architectural Elements

Crucial aspects:
- Modular composition system (central to the design)
- Plugin architecture for optimizers
- Gradient flow pathways
- Extension points for new optimizers

##### Specific Implementation Details

- Core interfaces and abstract classes
- Key method signatures (step, closure)
- Critical configuration points
- Performance optimization points (_foreach operations)

##### Performance optimization pathways

Keep the abstraction level focused on the architectural components while showing enough detail to understand the modular composition system, which is the key feature of this library.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Style Definitions
    classDef core fill:#2b4b6b,color:white
    classDef firstOrder fill:#4a90e2,color:white
    classDef secondOrder fill:#50b36e,color:white
    classDef zerothOrder fill:#c77eb5,color:white
    classDef quasiNewton fill:#d4a35d,color:white
    classDef integration fill:#967bb6,color:white
    classDef module fill:#7c94a7,color:white
    classDef experimental fill:#b55353,stroke-dasharray: 5 5,color:white

    subgraph CoreLayer["Core Layer"]
        CoreModule["Core Module"]:::core
        TensorListOpt["TensorList Optimizer"]:::core
    end

    subgraph OptimizationModules["Optimization Modules"]
        subgraph FirstOrder["First Order Methods"]
            SGD["SGD"]:::firstOrder
            Adam["Adam"]:::firstOrder
            RMSprop["RMSprop"]:::firstOrder
            Adagrad["Adagrad"]:::firstOrder
        end

        subgraph SecondOrder["Second Order Methods"]
            Newton["Newton Methods"]:::secondOrder
        end

        subgraph ZerothOrder["Zeroth Order Methods"]
            FDM["FDM-based Methods"]:::zerothOrder
        end

        subgraph QuasiNewton["Quasi-Newton Methods"]
            BFGS["BFGS"]:::quasiNewton
        end
    end

    subgraph Modules["Module Components"]
        LineSearch["Line Search"]:::module
        Regularization["Regularization"]:::module
        Momentum["Momentum"]:::module
        GradApprox["Gradient Approximation"]:::module
        Smoothing["Smoothing Methods"]:::module
        Adaptive["Adaptive Methods"]:::module
    end

    subgraph IntegrationLayer["Integration Layer"]
        ExternalWrappers["External Wrappers"]:::integration
        MetaComponents["Meta Components"]:::integration
        Chain["Chain Implementation"]:::integration
        Grafting["Grafting"]:::integration
        OptimizerWrapper["Optimizer Wrapper"]:::integration
    end

    subgraph Utilities["Utility Components"]
        RandomUtils["Random Utilities"]:::module
        GradUtils["Gradient Utilities"]:::module
    end

    subgraph Experimental["Experimental Features"]
        ExpFeatures["Experimental Modules"]:::experimental
    end

    %% Relationships
    CoreModule --> TensorListOpt
    TensorListOpt --> OptimizationModules
    OptimizationModules --> Modules
    Modules --> IntegrationLayer
    IntegrationLayer --> ExpFeatures
    
    %% Click Events
    click CoreModule "https://github.com/inikishev/torchzero/blob/main/src/torchzero/core/module.py"
    click TensorListOpt "https://github.com/inikishev/torchzero/blob/main/src/torchzero/core/tensorlist_optimizer.py"
    click FirstOrder "https://github.com/inikishev/torchzero/tree/main/src/torchzero/optim/first_order/"
    click SecondOrder "https://github.com/inikishev/torchzero/tree/main/src/torchzero/optim/second_order/"
    click ZerothOrder "https://github.com/inikishev/torchzero/tree/main/src/torchzero/optim/zeroth_order/"
    click QuasiNewton "https://github.com/inikishev/torchzero/tree/main/src/torchzero/optim/quasi_newton/"
    click LineSearch "https://github.com/inikishev/torchzero/tree/main/src/torchzero/modules/line_search/"
    click Regularization "https://github.com/inikishev/torchzero/tree/main/src/torchzero/modules/regularization/"
    click Momentum "https://github.com/inikishev/torchzero/tree/main/src/torchzero/modules/momentum/"
    click GradApprox "https://github.com/inikishev/torchzero/tree/main/src/torchzero/modules/gradient_approximation/"
    click SGD "https://github.com/inikishev/torchzero/blob/main/src/torchzero/modules/optimizers/sgd.py"
    click Adam "https://github.com/inikishev/torchzero/blob/main/src/torchzero/modules/optimizers/adam.py"
    click RMSprop "https://github.com/inikishev/torchzero/blob/main/src/torchzero/modules/optimizers/rmsprop.py"
    click Adagrad "https://github.com/inikishev/torchzero/blob/main/src/torchzero/modules/optimizers/adagrad.py"
    click ExternalWrappers "https://github.com/inikishev/torchzero/tree/main/src/torchzero/optim/wrappers/"
    click MetaComponents "https://github.com/inikishev/torchzero/tree/main/src/torchzero/modules/meta/"
    click ExpFeatures "https://github.com/inikishev/torchzero/tree/main/src/torchzero/modules/experimental/"
    click Smoothing "https://github.com/inikishev/torchzero/tree/main/src/torchzero/modules/smoothing/"
    click Adaptive "https://github.com/inikishev/torchzero/tree/main/src/torchzero/modules/adaptive/"
    click RandomUtils "https://github.com/inikishev/torchzero/tree/main/src/torchzero/random/"
    click GradUtils "https://github.com/inikishev/torchzero/tree/main/src/torchzero/grad/"
    click Chain "https://github.com/inikishev/torchzero/blob/main/src/torchzero/modules/meta/chain.py"
    click Grafting "https://github.com/inikishev/torchzero/blob/main/src/torchzero/modules/meta/grafting.py"
    click OptimizerWrapper "https://github.com/inikishev/torchzero/blob/main/src/torchzero/modules/meta/optimizer_wrapper.py"
```
