# fchaubard/gradient_agreement_filtering architecture

> Architecture diagram of the GitHub repository fchaubard/gradient_agreement_filtering, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fchaubard/gradient_agreement_filtering
- Repository: https://github.com/fchaubard/gradient_agreement_filtering

## Overview

##### Main Components

a) Core GAF Component:
- Two primary interfaces: step_GAF() and train_GAF()
- Gradient filtering mechanism using cosine distance

b) Integration Layer:
- How GAF interfaces with existing optimizers (SGD, Adam, AdamW, RMSProp)
- Demonstrate the connection to PyTorch's optimization framework

c) Data Flow Components:
- Input processing (model, optimizer, criterion, microbatches)
- Gradient computation and agreement filtering
- Output handling (filtered gradients, optimization steps)

##### Specific Design Elements

**┌─────────────────────────────────────────────────────────┐**
- External Interface                      │
- ┌─────────────┐                     ┌──────────────┐   │
- step_GAF() │                     │  train_GAF() │   │
- ┘                     └──────────────┘   │
- ▼                                  ▼           │
- ┌─────────────────────────────────────────────────┐   │
- GAF Core Engine                     │   │
- ┌─────────────┐        ┌────────────────────┐  │   │
- Gradient  │        │ Cosine Distance    │  │   │
- Computing  │◄─────► │    Calculator      │  │   │
- ┘        └────────────────────┘  │   │
- ▼                       ▼              │   │
- ┌─────────────────────────────────────────┐    │   │
- Agreement Filtering Logic         │    │   │
- ┘    │   │
- ┘   │
- ▼                                             │
- ┌─────────────────────────────────────────────────┐   │
- Optimizer Interface                 │   │
- ┌────────┐  ┌─────┐  ┌───────┐  ┌────────┐    │   │
- SGD   │  │Adam │  │AdamW  │  │RMSProp │    │   │
- ┘  └─────┘  └───────┘  └────────┘    │   │
- ┘   │
- ▼                                             │
- ┌─────────────────────────────────────────────────┐   │
- Monitoring & Logging                │   │
- ┌────────────────┐    ┌─────────────────┐     │   │
- Weights & Biases│    │ Verbose Logging │     │   │
- ┘    └─────────────────┘     │   │
- ┘   │
- ┘

##### Important Relationships

**Input Data → Microbatches → Gradient Computation →**
**Agreement Filtering → Optimizer Update → Model Update**

##### Notes for Implementation Details
- Annotations showing key parameters (cos_distance_thresh, num_batches_to_force_agreement)
- Version compatibility information
- Supported frameworks (PyTorch)

Remember to include a small textbox explaining that GAF is designed to bolt on top of existing optimization procedures, as this is a key feature of the system.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph External_Interface
        step["step_GAF()"]
        train["train_GAF()"]
    end

    subgraph GAF_Core["GAF Core Engine"]
        gradient["Gradient Computing"]:::core
        cosine["Cosine Distance Calculator"]:::core
        filter["Agreement Filtering Logic"]:::core
        gradient <--> cosine
        gradient --> filter
        cosine --> filter
    end

    subgraph Optimizer_Interface["Optimizer Interface"]
        sgd["SGD"]:::optimizer
        adam["Adam"]:::optimizer
        adamw["AdamW"]:::optimizer
        rmsprop["RMSProp"]:::optimizer
    end

    subgraph Monitoring["Monitoring & Logging"]
        wandb["Weights & Biases"]:::monitoring
        logging["Verbose Logging"]:::monitoring
    end

    step --> gradient
    train --> gradient
    filter --> sgd
    filter --> adam
    filter --> adamw
    filter --> rmsprop
    
    sgd --> wandb
    adam --> wandb
    adamw --> wandb
    rmsprop --> wandb
    
    sgd --> logging
    adam --> logging
    adamw --> logging
    rmsprop --> logging

    subgraph Data_Flow["Data Flow"]
        direction TB
        input["Input Data"]:::flow
        micro["Microbatches"]:::flow
        grad_comp["Gradient Computation"]:::flow
        agreement["Agreement Filtering"]:::flow
        opt_update["Optimizer Update"]:::flow
        model_update["Model Update"]:::flow
        
        input --> micro
        micro --> grad_comp
        grad_comp --> agreement
        agreement --> opt_update
        opt_update --> model_update
    end

    subgraph Legend["Legend"]
        l1["Core Components"]:::core
        l2["Optimizer Components"]:::optimizer
        l3["Monitoring Components"]:::monitoring
        l4["Data Flow"]:::flow
    end

    %% Component Mappings
    click gradient "https://github.com/Fchaubard/gradient_agreement_filtering/blob/main/src/gradient_agreement_filtering/gaf.py"
    click step "https://github.com/Fchaubard/gradient_agreement_filtering/blob/main/examples/1_cifar_100_train_loop_exposed.py"
    click train "https://github.com/Fchaubard/gradient_agreement_filtering/blob/main/examples/2_cifar_100_trainer.py"
    click model_update "https://github.com/Fchaubard/gradient_agreement_filtering/blob/main/examples/CIFAR-100_human.pt"
    click logging "https://github.com/Fchaubard/gradient_agreement_filtering/blob/main/test/test_gaf.py"

    classDef core fill:#3498db,stroke:#2980b9,color:white
    classDef optimizer fill:#e67e22,stroke:#d35400,color:white
    classDef monitoring fill:#9b59b6,stroke:#8e44ad,color:white
    classDef flow fill:#2ecc71,stroke:#27ae60,color:white

    %% Note Box
    note["GAF is designed to bolt on top of<br/>existing optimization procedures"]
    style note fill:#f1c40f,stroke:#f39c12,color:black
```
