# tomouellette/herb architecture

> Architecture diagram of the GitHub repository tomouellette/herb, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tomouellette/herb
- Repository: https://github.com/tomouellette/herb

## Overview

##### Component Details

Training Infrastructure (Left):
- FolderDataset
- Optional WebDataset support
- To data/ directory

- Below that, create a "Training Models" section containing:
- DINO (Self-supervised learning)
- MAE (Masked Autoencoder)
- MBT (Masked Barlow Twins)

Model Architectures (Center):
- ConvNext
- MLP
- MLP Mixer
- NAViT
- ViT
- Parameter scaling variants:
- nano (2.5M params)
- micro (5M params)
- tiny (10M params)
- small (20M params)
- base (80M params)
- large (200M+ params)

Inference Infrastructure (Right):
- Candle ConvNext
- Candle MLP Mixer
- Candle ViT
- "Model Format" component showing:
- PyTorch (.pth)
- SafeTensors format

##### Connections and Data Flow

- Data flow from input to training models
- Model architecture selection for training
- Model saving paths to both PyTorch and SafeTensors formats
- SafeTensors loading path to Rust inference components

##### Additional Elements

- Configuration parameters
- Resource requirements
- Input/Output specifications
- Image size requirements (224x224)

##### Technical Details Layer

- Training hyperparameters
- Model-specific parameters
- I/O specifications
- GPU/CPU requirements

##### The dual-language implementation (Python/Rust)

Remember to maintain a balance between showing the technical depth of the ML components while keeping the overall system architecture clear and understandable.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Training
        DataInput[Data Input]
        FolderDS[FolderDataset]
        WebDS[WebDataset]
        
        subgraph TrainingModels[Training Models]
            DINO[DINO]
            MAE[MAE]
            MBT[MBT]
        end
        
        Config[Configuration Parameters]
    end
    
    subgraph Architectures[Model Architectures]
        subgraph Backbones
            ConvNext[ConvNext]
            MLP[MLP]
            MLPMixer[MLP Mixer]
            NAViT[NAViT]
            ViT[ViT]
        end
        
        subgraph Scaling[Parameter Scaling]
            Nano[nano - 2.5M]
            Micro[micro - 5M]
            Tiny[tiny - 10M]
            Small[small - 20M]
            Base[base - 80M]
            Large[large - 200M+]
        end
        
        ModelZoo[Model Registry]
    end
    
    subgraph Inference[Inference Infrastructure]
        subgraph RustInference[Rust Inference]
            CandleConv[Candle ConvNext]
            CandleMLP[Candle MLP Mixer]
            CandleViT[Candle ViT]
        end
        
        subgraph ModelFormats[Model Formats]
            PyTorch[PyTorch .pth]
            SafeTensors[SafeTensors]
        end
    end
    
    %% Connections
    DataInput --> FolderDS
    DataInput --> WebDS
    FolderDS & WebDS --> TrainingModels
    Config --> TrainingModels
    TrainingModels --> Backbones
    Backbones --> Scaling
    Backbones --> ModelFormats
    ModelFormats --> RustInference
    Backbones --> ModelZoo
    
    %% Styling
    classDef python fill:#3776AB
    classDef rust fill:#DEA584
    classDef storage fill:#90EE90
    classDef config fill:#D3D3D3
    
    class DINO,MAE,MBT,ConvNext,MLP,MLPMixer,NAViT,ViT python
    class CandleConv,CandleMLP,CandleViT rust
    class DataInput,FolderDS,WebDS,SafeTensors,PyTorch storage
    class Config,Scaling config
    
    %% Click Events
    click DINO "https://github.com/tomouellette/herb/blob/main/models/dino.py"
    click MAE "https://github.com/tomouellette/herb/blob/main/models/mae.py"
    click MBT "https://github.com/tomouellette/herb/blob/main/models/mbt.py"
    click ConvNext "https://github.com/tomouellette/herb/blob/main/backbones/convnext.py"
    click MLP "https://github.com/tomouellette/herb/blob/main/backbones/mlp.py"
    click MLPMixer "https://github.com/tomouellette/herb/blob/main/backbones/mlp_mixer.py"
    click NAViT "https://github.com/tomouellette/herb/blob/main/backbones/navit.py"
    click ViT "https://github.com/tomouellette/herb/blob/main/backbones/vit.py"
    click CandleConv "https://github.com/tomouellette/herb/tree/main/backbones/candle_convnext/"
    click CandleMLP "https://github.com/tomouellette/herb/tree/main/backbones/candle_mlp_mixer/"
    click CandleViT "https://github.com/tomouellette/herb/tree/main/backbones/candle_vit/"
    click DataInput "https://github.com/tomouellette/herb/tree/main/data/"
    click ModelZoo "https://github.com/tomouellette/herb/blob/main/backbones/zoo.py"
```
