# nvlabs/mambavision architecture

> Architecture diagram of the GitHub repository nvlabs/mambavision, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvlabs/mambavision
- Repository: https://github.com/nvlabs/mambavision

## Overview

##### Project Type and Purpose
- MambaVision is an open-source deep learning library (PyTorch) providing implementations of Mamba-Transformer vision backbones, training & evaluation scripts, configuration management, and utilities for image classification and feature extraction.
- It is not a web app or microservice system; rather it is a research‐oriented ML framework/toolkit with core functionality exposed as a Python package and CLI scripts.

##### File Structure
- Top-level:
- Dockerfile, requirements.txt, setup.py/cfg – packaging & environment
- README.md, LICENSE – documentation & licensing
- mambavision/ – source code
- mambavision/ subpackages:
- models/ – model definitions and registry (mamba_vision.py, registry.py)
- scheduler/ – learning‐rate schedulers with factory pattern
- utils/ – dataset loaders, other helper functions
- configs/ – YAML configs for different model variants
- train.py, validate.py + shell wrappers (.sh) – entry points for training/validation
- tensorboard.py – logging support

##### Extract Additional Insights from README
- Core features:
- create_model factory
- Hugging Face integration (AutoModelForImageClassification.from_pretrained)
- Support arbitrary input resolutions
- Workflow:
- Data ingestion via timm transforms
- Forward passes through hierarchical self-attention & mixer blocks
- Logging & checkpointing
- Evaluation on ImageNet
- Packaging:
- pip install mambavision
- Docker container

##### Key Components
a. Main Components (grouped into layers/modules):
1. Model Layer
– MambaVision backbone (mamba_vision.py)
– Model registry & factory
2. Configuration Layer
– YAML files → SchedulerFactory, ModelFactory
3. Data Layer
– Dataset utils, timm.transforms
4. Training Pipeline
– train.py, train.sh
– Scheduler (cosine, step, poly, etc.)
– TensorBoard logger
5. Evaluation Pipeline
– validate.py, validate.sh, validate_pip_model.py
6. Packaging & Deployment
– setup.py / setup.cfg → pip package
– Dockerfile → Docker image
– Hugging Face integration
b. Relationships & Interactions
- Config YAML → parsed by train.py/validate.py → instantiate model & scheduler via factory
- Dataset utils → load images → apply transforms → batch → feed model
- Model forward/backward → output logits/loss → optimizer step
- Scheduler callbacks adjust learning rate
- TensorBoard logger tracks metrics
- validate scripts load checkpoints → run evaluation dataset → output accuracy
- Packaging layer exposes create_model() API and CLI entry points
- External service: Hugging Face repository for model download
c. Architectural Patterns & Design Principles
- Factory pattern (ModelFactory, SchedulerFactory) for extensibility
- Modular package structure (separate models, schedulers, utils)
- Configuration‐driven execution (YAML + argparse)
- Separation of concerns: training vs. validation scripts
- Logging/monitoring abstraction (tensorboard.py)
d. Technologies & Frameworks
- Python, PyTorch, timm, transformers (for Hugging Face integration)
- Docker
- YAML for configuration
- TensorBoard for metrics
- Shell wrappers for reproducible runs

##### Final Recommendations
- Be exhaustive: represent each subpackage and script as a distinct node
- Data/control flow: show how YAML config drives execution
- Factory and registry patterns that enable extensibility
- Both local components (code) and external dependencies (HF, Docker image)

## Diagram (Mermaid)

```mermaid
graph TB
    %% Configuration Layer
    subgraph "Configuration Layer"
        direction TB
        ConfigFiles["YAML Configs"]:::config
        SchedulerFactory["SchedulerFactory"]:::core
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        DataUtils["Dataset Utils"]:::core
    end

    %% Model Layer
    subgraph "Model Layer"
        direction TB
        Registry["Model Registry"]:::core
        Backbone["MambaVision Backbone"]:::core
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        direction TB
        TrainSH["train.sh"]:::pipeline
        TrainPy["train.py"]:::pipeline
        SchedulerImpl["Learning-Rate Schedulers"]:::core
        TBLogger["TensorBoard Logger"]:::core
    end

    %% Evaluation Pipeline
    subgraph "Evaluation Pipeline"
        direction TB
        ValidateSH["validate.sh"]:::pipeline
        ValidatePipSH["validate_pip.sh"]:::pipeline
        ValidatePy["validate.py"]:::pipeline
        ValidatePipModel["validate_pip_model.py"]:::pipeline
    end

    %% Packaging & Deployment
    subgraph "Packaging & Deployment"
        direction TB
        Setup["setup.py/cfg"]:::pipeline
        Docker["Dockerfile"]:::external
        HFHub["Hugging Face Model Hub"]:::external
        PipPkg["pip Package API"]:::external
    end

    %% Relationships
    ConfigFiles -->|"parsed by"| SchedulerFactory
    ConfigFiles -->|"parsed by"| Registry
    SchedulerFactory -->|"instantiates"| SchedulerImpl
    Registry -->|"instantiates"| Backbone
    DataUtils -->|"feeds data to"| Backbone
    TrainSH -->|"calls"| TrainPy
    TrainPy -->|"uses"| ConfigFiles
    TrainPy -->|"uses"| SchedulerFactory
    TrainPy -->|"uses"| Registry
    TrainPy -->|"loads data via"| DataUtils
    TrainPy -->|"executes"| Backbone
    TrainPy -->|"logs metrics to"| TBLogger
    TrainPy -->|"saves checkpoints"| PipPkg
    ValidateSH -->|"calls"| ValidatePy
    ValidatePipSH -->|"calls"| ValidatePipModel
    ValidatePy -->|"uses"| PipPkg
    ValidatePy -->|"uses"| DataUtils
    ValidatePy -->|"executes"| Backbone
    ValidatePipModel -->|"executes"| Backbone
    Setup -->|"builds"| PipPkg
    Docker -->|"packages image with"| PipPkg
    PipPkg -->|"downloads model from"| HFHub

    %% Click Events
    click Backbone "https://github.com/nvlabs/mambavision/blob/main/mambavision/models/mamba_vision.py"
    click Registry "https://github.com/nvlabs/mambavision/blob/main/mambavision/models/registry.py"
    click ConfigFiles "https://github.com/nvlabs/mambavision/blob/main/mambavision/configs/*.yaml"
    click SchedulerFactory "https://github.com/nvlabs/mambavision/blob/main/mambavision/scheduler/scheduler_factory.py"
    click DataUtils "https://github.com/nvlabs/mambavision/blob/main/mambavision/utils/datasets.py"
    click TrainPy "https://github.com/nvlabs/mambavision/blob/main/mambavision/train.py"
    click TrainSH "https://github.com/nvlabs/mambavision/blob/main/mambavision/train.sh"
    click SchedulerImpl "https://github.com/nvlabs/mambavision/tree/main/mambavision/scheduler/"
    click TBLogger "https://github.com/nvlabs/mambavision/blob/main/mambavision/tensorboard.py"
    click ValidatePy "https://github.com/nvlabs/mambavision/blob/main/mambavision/validate.py"
    click ValidateSH "https://github.com/nvlabs/mambavision/blob/main/mambavision/validate.sh"
    click ValidatePipSH "https://github.com/nvlabs/mambavision/blob/main/mambavision/validate_pip.sh"
    click ValidatePipModel "https://github.com/nvlabs/mambavision/blob/main/mambavision/validate_pip_model.py"
    click Setup "https://github.com/nvlabs/mambavision/blob/main/setup.py"
    click Setup "https://github.com/nvlabs/mambavision/blob/main/setup.cfg"
    click Docker "https://github.com/nvlabs/mambavision/tree/main/Dockerfile"

    %% Styles
    classDef core fill:#AED6F1,stroke:#1B4F72,color:#000
    classDef pipeline fill:#ABEBC6,stroke:#196F3D,color:#000
    classDef config fill:#F9E79F,stroke:#B9770E,color:#000
    classDef external fill:#D5D8DC,stroke:#566573,color:#000
```
