# rfarell/guided-dropout-experiments architecture

> Architecture diagram of the GitHub repository rfarell/guided-dropout-experiments, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rfarell/guided-dropout-experiments
- Repository: https://github.com/rfarell/guided-dropout-experiments

## Overview

##### Main Components

A. Data Pipeline
- CIFAR-10 Dataset Input (3x32x32 images)
- Data Preprocessing/Flattening (3072 features)

B. Model Architecture (Three Variants)
1. Guided Dropout Model
2. Regular Dropout Model
3. No Dropout Model

C. Training System
- Loss Calculation
- Optimization
- Metrics Collection

D. Evaluation System
- Test Accuracy Measurement
- Signal Strength Analysis
- Visualization Generation

**[Data Layer]**

**[Model Layer]**

**[Training Layer]**

**[Evaluation Layer]**

##### Specific Component Details

For each model variant, show:
- Input Layer (3072 nodes)
- Hidden Layer 1 (8192 nodes)
- Hidden Layer 2 (8192 nodes)
- Output Layer (10 nodes)

- Regular connections
- Dropout connections (for Regular Dropout)
- Guided Dropout connections (with strength parameter t_j)

##### Key Flows

A. Training Flow:
**Input → Flatten → Hidden Layers → Output → Loss → Backprop**

B. Strength Calculation Flow:
**Activations → Mean/Variance/SNR Calculations → Visualization**

##### Implementation Details

- ReLU Activation: f(z) = max(0, z)
- Guided Dropout: a_j^(l) = t_j^(l) * f(z_j^(l))
- Signal Strength Metrics:
- Mean Activation
- Variance
- SNR

##### Additional Elements

- Strength parameter distribution
- Dropout mechanism comparison
- Metrics visualization pipeline

##### The relationship between components

Keep the following files in mind when showing component relationships:
- models/guided_dropout_model.py
- models/regular_dropout_model.py
- models/no_dropout_model.py
- train.py
- visualize_activation_strengths.py

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Data
        CIFAR[CIFAR-10 Dataset]
        Preprocess[Data Preprocessing]
        CIFAR --> Preprocess
    end

    subgraph Models
        direction TB
        subgraph GuidedModel["Guided Dropout Model"]
            GInput["Input Layer (3072)"]
            GH1["Hidden Layer 1 (8192)"]
            GH2["Hidden Layer 2 (8192)"]
            GOutput["Output Layer (10)"]
            GInput --> GH1
            GH1 --> GH2
            GH2 --> GOutput
        end

        subgraph RegularModel["Regular Dropout Model"]
            RInput["Input Layer (3072)"]
            RH1["Hidden Layer 1 (8192)"]
            RH2["Hidden Layer 2 (8192)"]
            ROutput["Output Layer (10)"]
            RInput --> RH1
            RH1 --> RH2
            RH2 --> ROutput
        end

        subgraph NoDropModel["No Dropout Model"]
            NInput["Input Layer (3072)"]
            NH1["Hidden Layer 1 (8192)"]
            NH2["Hidden Layer 2 (8192)"]
            NOutput["Output Layer (10)"]
            NInput --> NH1
            NH1 --> NH2
            NH2 --> NOutput
        end
    end

    subgraph Training
        Loss["Loss Calculation"]
        Optimizer["Optimization"]
        Metrics["Metrics Collection"]
        Loss --> Optimizer
        Optimizer --> Metrics
    end

    subgraph Evaluation
        TestAcc["Test Accuracy"]
        SignalStr["Signal Strength Analysis"]
        Viz["Visualization Generation"]
        TestAcc --> SignalStr
        SignalStr --> Viz
    end

    Preprocess --> GInput
    Preprocess --> RInput
    Preprocess --> NInput
    
    GOutput --> Loss
    ROutput --> Loss
    NOutput --> Loss
    
    Metrics --> TestAcc

    %% Component Mappings
    click GuidedModel "https://github.com/rfarell/guided-dropout-experiments/blob/main/models/guided_dropout_model.py"
    click RegularModel "https://github.com/rfarell/guided-dropout-experiments/blob/main/models/regular_dropout_model.py"
    click NoDropModel "https://github.com/rfarell/guided-dropout-experiments/blob/main/models/no_dropout_model.py"
    click Training "https://github.com/rfarell/guided-dropout-experiments/blob/main/train.py"
    click Viz "https://github.com/rfarell/guided-dropout-experiments/blob/main/visualize_activation_strengths.py"
    click Evaluation "https://github.com/rfarell/guided-dropout-experiments/blob/main/visualize_activation_strengths.py"
    click Models "https://github.com/rfarell/guided-dropout-experiments/blob/main/models/__init__.py"

    classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px
    classDef model fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
    classDef training fill:#e8f5e9,stroke:#388e3c,stroke-width:2px
    classDef evaluation fill:#fff3e0,stroke:#f57c00,stroke-width:2px
    classDef data fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px

    class GuidedModel,RegularModel,NoDropModel model
    class Training,Loss,Optimizer,Metrics training
    class Evaluation,TestAcc,SignalStr,Viz evaluation
    class Data,CIFAR,Preprocess data
```
