# vipul577/ml_framework architecture

> Architecture diagram of the GitHub repository vipul577/ml_framework, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vipul577/ml_framework
- Repository: https://github.com/vipul577/ml_framework

## Overview

1. **Overall shape**
   - Seera is a Python-first deep learning framework with two native acceleration backends: a **C++ CPU extension** and a **CUDA GPU extension**.
   - Python owns the public API, model construction, and autograd graph; native code performs heavy tensor math and kernel execution.

2. **Primary frontend boundary**
   - `Seera.py` is the high-level user API: layers, `Sequential`, losses, optimizers, training, prediction, save/load.
   - This is the main entry point for application code and the layer where users choose `device="cpu"` or `device="cuda"`.

3. **Tensor and graph layer**
   - `Seera_init.py` defines the core `Tensor` abstraction, operator overloading, forward ops, and computation-graph bookkeeping.
   - Tensors can wrap either NumPy arrays or GPU tensor objects, so the same Python-level operations can dispatch across backends.

4. **Autograd engine**
   - `Seera_Engine.py` implements reverse-mode differentiation.
   - It builds a dynamic graph, topologically orders nodes, and then dispatches backward logic per op, with separate CPU and GPU code paths where needed.

5. **Backend selection and fallback chain**
   - Execution is tri-tiered: first GPU via `seera_cuda`, then CPU C++ via `seera_cpp`, then pure NumPy fallback.
   - This makes native acceleration optional while preserving functional correctness on systems without compiled extensions.

6. **GPU tensor runtime**
   - `cuTen.py` is the GPU tensor runtime.
   - It wraps raw `cudaMalloc` device pointers, manages lifetime, supports zero-copy reshape, and exposes tensor-like operations that stay on device.
   - This class is the main bridge between Python autograd objects and CUDA kernel invocations.

7. **CPU acceleration subsystem**
   - `engine/` builds into `seera_cpp` with pybind11 bindings.
   - It contains C++17 implementations for matmul, activations, convolution, pooling, batchnorm, and tensor utilities.
   - OpenBLAS is used for dense linear algebra, and OpenMP is used for parallel loops and reductions.

8. **CUDA acceleration subsystem**
   - `engine_cuda/` builds into `seera_cuda`.
   - It contains handwritten CUDA kernels for GEMM, convolution, transposed convolution, activations, reductions, pooling, broadcasting, upsampling, and tensor utility ops.
   - The README emphasizes Tensor Core WMMA usage, so matrix multiply and convolution are the performance-critical GPU paths.

9. **Python-to-native bridge**
   - Both native engines are exposed through pybind11 wrappers: `engine/src/bindings.cpp` and `engine_cuda/src/cuda_bindings.cpp`.
   - The bindings move NumPy arrays to and from device/host memory, expose kernel entry points, and treat GPU pointers as Python-accessible handles.

10. **Data flow during training**
   - User code creates a model in Python, which instantiates `Tensor` objects and layer state.
   - Forward passes dispatch to CPU or GPU kernels.
   - The loss tensor triggers autograd, which walks the graph backward and accumulates gradients.
   - Optimizers update weights in place, staying on device when running on CUDA.

11. **Model and state persistence**
   - Save/load uses Python `pickle`.
   - When serializing GPU models, parameters are transferred back to host form for persistence, then reconstructed on load.

12. **Validation and tooling**
   - `Testing/` contains diagnostics, unit-style scripts, and MNIST comparison runs against PyTorch.
   - `benchmark.py` provides a performance/convergence reference, while the notebooks are exploratory demos.
   - `build_engine.py` and `build_engine_cuda.py` are the build boundaries for the two native shared libraries.

13. **Useful diagram boundaries**
   - A clear architecture diagram should separate:
     - **Python API and autograd control plane**
     - **CPU native backend**
     - **GPU native backend**
     - **Memory/transport bridge**
   - The key flow is: user API → tensor graph → autograd engine → backend dispatch → native kernels → gradients/optimizer updates.

Main entry points: [`Seera_init.py`](https://github.com/vipul577/ml_framework/blob/main/Seera_init.py) (Tensor core), [`Seera.py`](https://github.com/vipul577/ml_framework/blob/main/Seera.py) (Seera API), [`engine_cuda/src/cuda_bindings.cpp`](https://github.com/vipul577/ml_framework/blob/main/engine_cuda/src/cuda_bindings.cpp) (GPU binding), [`Seera_Engine.py`](https://github.com/vipul577/ml_framework/blob/main/Seera_Engine.py) (Autograd engine)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_python["Python control plane"]
  node_api["Seera API<br/>python API<br/>[Seera.py]"]
  node_tensor(("Tensor core<br/>tensor graph<br/>[Seera_init.py]"))
  node_autograd(("Autograd engine<br/>reverse-mode autodiff<br/>[Seera_Engine.py]"))
  node_cuten(("CUDA tensor<br/>gpu tensor runtime<br/>[cuTen.py]"))
end

subgraph group_cpu["CPU backend"]
  node_bridge_cpu["CPU binding<br/>pybind11 bridge<br/>[bindings.cpp]"]
  node_cpu_tensor["CPU tensor ops<br/>C++ tensor ops<br/>[tensor_ops.cpp]"]
  node_cpu_math["CPU kernels<br/>C++ compute kernels<br/>[activation_ops.cpp]"]
  node_cpu_engine_hdr["CPU engine API<br/>C++ header<br/>[seera_engine.hpp]"]
end

subgraph group_gpu["GPU backend"]
  node_bridge_gpu["GPU binding<br/>pybind11 bridge<br/>[cuda_bindings.cpp]"]
  node_gpu_tensors["GPU runtime<br/>CUDA tensor runtime"]
  node_gpu_gemm["GEMM kernels<br/>CUDA compute kernels<br/>[GEMM.cu]"]
  node_gpu_ops["GPU ops<br/>CUDA kernels<br/>[activations.cu]"]
  node_gpu_engine_hdr["GPU engine API<br/>CUDA header"]
end

subgraph group_tooling["Tooling"]
  node_build_cpu["Build CPU<br/>build script<br/>[build_engine.py]"]
  node_build_gpu["Build GPU<br/>build script"]
  node_bench["Benchmark<br/>perf script<br/>[benchmark.py]"]
  node_tests["Diagnostics<br/>validation suite"]
end

node_api -->|"builds models"| node_tensor
node_tensor -->|"records graph"| node_autograd
node_autograd -->|"CPU backward"| node_cpu_math
node_autograd -->|"GPU backward"| node_gpu_ops
node_tensor -->|"device tensors"| node_cuten
node_tensor -->|"CPU dispatch"| node_bridge_cpu
node_tensor -->|"GPU dispatch"| node_bridge_gpu
node_bridge_cpu -->|"loads ops"| node_cpu_tensor
node_bridge_cpu -->|"calls kernels"| node_cpu_math
node_bridge_gpu -->|"manages memory"| node_gpu_tensors
node_bridge_gpu -->|"launches GEMM"| node_gpu_gemm
node_bridge_gpu -->|"launches ops"| node_gpu_ops
node_cpu_math -->|"implements API"| node_cpu_engine_hdr
node_gpu_ops -->|"implements API"| node_gpu_engine_hdr
node_api -.->|"loads extension"| node_build_cpu
node_api -.->|"loads extension"| node_build_gpu
node_bench -.->|"measures"| node_api
node_tests -.->|"validates"| node_api
node_tests -.->|"checks gradients"| node_autograd

click node_api "https://github.com/vipul577/ml_framework/blob/main/Seera.py"
click node_tensor "https://github.com/vipul577/ml_framework/blob/main/Seera_init.py"
click node_autograd "https://github.com/vipul577/ml_framework/blob/main/Seera_Engine.py"
click node_cuten "https://github.com/vipul577/ml_framework/blob/main/cuTen.py"
click node_bridge_cpu "https://github.com/vipul577/ml_framework/blob/main/engine/src/bindings.cpp"
click node_cpu_tensor "https://github.com/vipul577/ml_framework/blob/main/engine/src/tensor_ops.cpp"
click node_cpu_math "https://github.com/vipul577/ml_framework/blob/main/engine/src/activation_ops.cpp"
click node_cpu_engine_hdr "https://github.com/vipul577/ml_framework/blob/main/engine/include/seera_engine.hpp"
click node_bridge_gpu "https://github.com/vipul577/ml_framework/blob/main/engine_cuda/src/cuda_bindings.cpp"
click node_gpu_tensors "https://github.com/vipul577/ml_framework/blob/main/engine_cuda/src/cuTen_essentails.cu"
click node_gpu_gemm "https://github.com/vipul577/ml_framework/blob/main/engine_cuda/src/GEMM.cu"
click node_gpu_ops "https://github.com/vipul577/ml_framework/blob/main/engine_cuda/src/activations.cu"
click node_gpu_engine_hdr "https://github.com/vipul577/ml_framework/blob/main/engine_cuda/include/seera_engine_cuda.hpp"
click node_build_cpu "https://github.com/vipul577/ml_framework/blob/main/build_engine.py"
click node_build_gpu "https://github.com/vipul577/ml_framework/blob/main/build_engine_cuda.py"
click node_bench "https://github.com/vipul577/ml_framework/blob/main/benchmark.py"
click node_tests "https://github.com/vipul577/ml_framework/blob/main/Testing/mnist_seera_vs_pytorch.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_tensor,node_autograd,node_cuten toneBlue
class node_bridge_cpu,node_cpu_tensor,node_cpu_math,node_cpu_engine_hdr toneAmber
class node_bridge_gpu,node_gpu_tensors,node_gpu_gemm,node_gpu_ops,node_gpu_engine_hdr toneMint
class node_build_cpu,node_build_gpu,node_bench,node_tests toneRose
```

## Components

### Python control plane

Public API, tensor graph, and autograd orchestration

- [Seera API](https://github.com/vipul577/ml_framework/blob/main/Seera.py): python API · High-level layers, Sequential, losses, optimizers, training, prediction, and persistence · `Seera.py`
- [Tensor core](https://github.com/vipul577/ml_framework/blob/main/Seera_init.py): tensor graph · Tensor abstraction, operator overloading, forward ops, and graph bookkeeping · `Seera_init.py`
- [Autograd engine](https://github.com/vipul577/ml_framework/blob/main/Seera_Engine.py): reverse-mode autodiff · Builds the backward graph, topological ordering, and gradient dispatch · `Seera_Engine.py`
- [CUDA tensor](https://github.com/vipul577/ml_framework/blob/main/cuTen.py): gpu tensor runtime · Device-pointer tensor wrapper and on-device tensor operations · `cuTen.py`

### CPU backend

Native C++ execution path

- [CPU binding](https://github.com/vipul577/ml_framework/blob/main/engine/src/bindings.cpp): pybind11 bridge · Python bindings into the C++ backend · `engine/src/bindings.cpp`
- [CPU tensor ops](https://github.com/vipul577/ml_framework/blob/main/engine/src/tensor_ops.cpp): C++ tensor ops · Tensor utilities and host-side tensor mechanics · `engine/src/tensor_ops.cpp`
- [CPU kernels](https://github.com/vipul577/ml_framework/blob/main/engine/src/activation_ops.cpp): C++ compute kernels · Matmul, activations, convolution, pooling, and batchnorm · `engine/src/activation_ops.cpp`
- [CPU engine API](https://github.com/vipul577/ml_framework/blob/main/engine/include/seera_engine.hpp): C++ header · Public native engine interface for the CPU backend · `engine/include/seera_engine.hpp`

### GPU backend

Native CUDA execution path

- [GPU binding](https://github.com/vipul577/ml_framework/blob/main/engine_cuda/src/cuda_bindings.cpp): pybind11 bridge · Python bindings into the CUDA backend · `engine_cuda/src/cuda_bindings.cpp`
- [GPU runtime](https://github.com/vipul577/ml_framework/blob/main/engine_cuda/src/cuTen_essentails.cu): CUDA tensor runtime · Device memory management, pointer handling, and tensor utilities · `engine_cuda/src/cuTen_essentails.cu`
- [GEMM kernels](https://github.com/vipul577/ml_framework/blob/main/engine_cuda/src/GEMM.cu): CUDA compute kernels · Tensor Core-aware matrix multiply and related dense linear algebra · `engine_cuda/src/GEMM.cu`
- [GPU ops](https://github.com/vipul577/ml_framework/blob/main/engine_cuda/src/activations.cu): CUDA kernels · Activations, convolution, reduction, pooling, broadcasting, and resizing ops · `engine_cuda/src/activations.cu`
- [GPU engine API](https://github.com/vipul577/ml_framework/blob/main/engine_cuda/include/seera_engine_cuda.hpp): CUDA header · Public native engine interface for the CUDA backend · `engine_cuda/include/seera_engine_cuda.hpp`

### Tooling

Build, benchmark, and validation assets

- [Build CPU](https://github.com/vipul577/ml_framework/blob/main/build_engine.py): build script · Compiles the C++ extension module · `build_engine.py`
- [Build GPU](https://github.com/vipul577/ml_framework/blob/main/build_engine_cuda.py): build script · Compiles the CUDA extension module · `build_engine_cuda.py`
- [Benchmark](https://github.com/vipul577/ml_framework/blob/main/benchmark.py): perf script · Performance and convergence reference script · `benchmark.py`
- [Diagnostics](https://github.com/vipul577/ml_framework/blob/main/Testing/mnist_seera_vs_pytorch.py): validation suite · MNIST comparison runs, diagnostic scripts, and exploratory checks · `Testing/mnist_seera_vs_pytorch.py`

## Connections

- Seera API → Tensor core: builds models · User-facing layers instantiate and manipulate tensors
- Tensor core → Autograd engine: records graph · Tensor ops create nodes for backward differentiation
- Autograd engine → CPU kernels: CPU backward · Dispatches CPU gradient logic to native kernels
- Autograd engine → GPU ops: GPU backward · Dispatches GPU gradient logic to CUDA kernels
- Tensor core → CUDA tensor: device tensors · GPU-backed tensors wrap device pointers and stay on device
- Tensor core → CPU binding: CPU dispatch · Forwards host execution to the C++ extension
- Tensor core → GPU binding: GPU dispatch · Forwards device execution to the CUDA extension
- CPU binding → CPU tensor ops: loads ops · Binding exposes host tensor utilities and tensor state
- CPU binding → CPU kernels: calls kernels · Binding exposes compute kernels to Python
- GPU binding → GPU runtime: manages memory · Binding exposes device memory and tensor handles
- GPU binding → GEMM kernels: launches GEMM · Binding exposes dense matrix execution paths
- GPU binding → GPU ops: launches ops · Binding exposes the broader CUDA kernel suite
- CPU kernels → CPU engine API: implements API · Native CPU sources conform to the shared engine interface
- GPU ops → GPU engine API: implements API · CUDA sources conform to the shared engine interface
- Seera API → Build CPU: loads extension · CPU backend is built and linked as an optional native module
- Seera API → Build GPU: loads extension · GPU backend is built and linked as an optional native module
- Benchmark → Seera API: measures · Benchmark exercises training and inference paths
- Diagnostics → Seera API: validates · Diagnostics compare behavior against reference runs
- Diagnostics → Autograd engine: checks gradients · Validation scripts probe backward correctness
