# ucb-bar/baremetal-nn architecture

> Architecture diagram of the GitHub repository ucb-bar/baremetal-nn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ucb-bar/baremetal-nn
- Repository: https://github.com/ucb-bar/baremetal-nn

## Overview

##### Main Components

a) Model Conversion Pipeline:
- PyTorch Model Input
- Converter Module (in converter/src/torchconverter)
- Generated C Code Output (model.h, weights.h, operators.h)

b) Runtime Components:
- Core Neural Network Library (nn/ directory)
- Device-Specific Backends
- Memory Management System

**[Input Layer]**
**PyTorch Models**
- ↓
**[Conversion Layer]**
**TorchConverter**
- ↓
**[Output Generation Layer]**
**Generated C Files**
- ↓
**[Runtime Layer]**
**Backend Implementation Layer → Device-Specific Optimizations**
- ↓
**[Hardware Layer]**
**Target Platforms (RISC-V, x86, K230, etc.)**

##### Key Elements to Highlight

a) Model Processing Pipeline:
- Flow from PyTorch model → Converter → C code generation
- Three main output files (nn.h, operators.h, weights.h)

b) Backend Support Matrix:
- Datatype support (F16, F32, I32)

c) Memory Management:
- NHWC memory layout system
- Up to 4-dimension tensor support

##### Implementation Details

- Tensor Operations Support
- Supported operators (matmul, linear, relu, etc.)
- Datatype variants
- Backend Implementations
- RISC-V Vector support
- Scalar CPU implementation
- Gemmini support (marked as in-progress)
- Build System Integration
- CMake configuration options
- Toolchain support

##### Specific Technical Notes

- RISC-V Vector backend with FP16 support requirements
- Assembly implementations for GCC<14.0
- Relationship between different device backends
- Tensor operation dispatch system

Remember to include a version number and date, as this is an actively developed project (current version 0.2.0).

## Diagram (Mermaid)

```mermaid
graph TB
    %% Input Layer
    subgraph Input
        PyTorch["PyTorch Models"]:::input
    end

    %% Conversion Layer
    subgraph Converter
        TorchConv["TorchConverter"]:::converter
        Tracer["Model Tracer"]:::converter
    end

    %% Output Generation Layer
    subgraph Generated
        ModelH["model.h"]:::output
        WeightsH["weights.h"]:::output
        OperatorsH["operators.h"]:::output
    end

    %% Runtime Layer
    subgraph Runtime
        NNLib["Neural Network Library"]:::core
        subgraph DataTypes
            F16["F16 Support"]:::dtype
            F32["F32 Support"]:::dtype
            I32["I32 Support"]:::dtype
        end
        subgraph Backends
            RVVector["RISC-V Vector"]:::backend
            ScalarCPU["Scalar CPU"]:::backend
            Gemmini["Gemmini"]:::backend
            Metal["Metal"]:::backend
        end
    end

    %% Hardware Layer
    subgraph Hardware
        RISCV["RISC-V"]:::hw
        x86["x86"]:::hw
        K230["K230"]:::hw
    end

    %% Examples and Testing
    subgraph Examples
        Simple["Simple Example"]:::example
        MNIST["MNIST Example"]:::example
        CharRNN["Char-RNN"]:::example
        LLaMA2["LLaMA2"]:::example
    end

    subgraph Testing
        TestFrame["Test Framework"]:::test
        TestGen["Test Generator"]:::test
    end

    %% Relationships
    PyTorch --> TorchConv
    TorchConv --> Tracer
    Tracer --> ModelH
    Tracer --> WeightsH
    Tracer --> OperatorsH
    ModelH & WeightsH & OperatorsH --> NNLib
    NNLib --> DataTypes
    NNLib --> Backends
    Backends --> Hardware
    NNLib --> Examples
    Examples --> TestFrame
    TestGen --> TestFrame

    %% Click Events
    click TorchConv "https://github.com/ucb-bar/Baremetal-NN/tree/main/converter/src/torchconverter/"
    click Tracer "https://github.com/ucb-bar/Baremetal-NN/blob/main/converter/src/torchconverter/tracer.py"
    click NNLib "https://github.com/ucb-bar/Baremetal-NN/tree/main/nn/"
    click F16 "https://github.com/ucb-bar/Baremetal-NN/blob/main/nn/nn_f16.h"
    click F32 "https://github.com/ucb-bar/Baremetal-NN/blob/main/nn/nn_f32.h"
    click I32 "https://github.com/ucb-bar/Baremetal-NN/blob/main/nn/nn_i32.h"
    click RVVector "https://github.com/ucb-bar/Baremetal-NN/blob/main/device/riscv_vector.h"
    click Gemmini "https://github.com/ucb-bar/Baremetal-NN/blob/main/device/gemmini/gemmini.h"
    click Metal "https://github.com/ucb-bar/Baremetal-NN/blob/main/device/metal.h"
    click Simple "https://github.com/ucb-bar/Baremetal-NN/tree/main/examples/simple/"
    click MNIST "https://github.com/ucb-bar/Baremetal-NN/tree/main/examples/mnist/"
    click CharRNN "https://github.com/ucb-bar/Baremetal-NN/tree/main/examples/char-rnn/"
    click LLaMA2 "https://github.com/ucb-bar/Baremetal-NN/tree/main/examples/llama2/"
    click TestFrame "https://github.com/ucb-bar/Baremetal-NN/blob/main/tests/src/unittest.h"
    click TestGen "https://github.com/ucb-bar/Baremetal-NN/blob/main/tests/generate_test.py"

    %% Styles
    classDef input fill:#90EE90
    classDef converter fill:#6495ED
    classDef output fill:#98FB98
    classDef core fill:#4169E1
    classDef dtype fill:#DDA0DD
    classDef backend fill:#FFA500
    classDef hw fill:#A9A9A9
    classDef example fill:#20B2AA
    classDef test fill:#BA55D3

    %% Legend
    subgraph Legend
        L1["Input Components"]:::input
        L2["Converter Components"]:::converter
        L3["Output Files"]:::output
        L4["Core Components"]:::core
        L5["Data Types"]:::dtype
        L6["Backends"]:::backend
        L7["Hardware"]:::hw
        L8["Examples"]:::example
        L9["Testing"]:::test
    end
```
