How vipul577/ml_framework works
1. Overall shape
- Seera is a Python-first deep learning framework with two native acceleration backends: a C++ CPU extension and a CUDA GPU extension.
- Python owns the public API, model construction, and autograd graph; native code performs heavy tensor math and kernel execution.
2. Primary frontend boundary
- Seera.py is the high-level user API: layers, Sequential, losses, optimizers, training, prediction, save/load.
- This is the main entry point for application code and the layer where users choose device="cpu" or device="cuda".
3. Tensor and graph layer
- Seera_init.py defines the core Tensor abstraction, operator overloading, forward ops, and computation-graph bookkeeping.
- Tensors can wrap either NumPy arrays or GPU tensor objects, so the same Python-level operations can dispatch across backends.
4. Autograd engine
- Seera_Engine.py implements reverse-mode differentiation.
- It builds a dynamic graph, topologically orders nodes, and then dispatches backward logic per op, with separate CPU and GPU code paths where needed.
5. Backend selection and fallback chain
- Execution is tri-tiered: first GPU via seera_cuda, then CPU C++ via seera_cpp, then pure NumPy fallback.
- This makes native acceleration optional while preserving functional correctness on systems without compiled extensions.
6. GPU tensor runtime
- cuTen.py is the GPU tensor runtime.
- It wraps raw cudaMalloc device pointers, manages lifetime, supports zero-copy reshape, and exposes tensor-like operations that stay on device.
- This class is the main bridge between Python autograd objects and CUDA kernel invocations.
7. CPU acceleration subsystem
- engine/ builds into seera_cpp with pybind11 bindings.
- It contains C++17 implementations for matmul, activations, convolution, pooling, batchnorm, and tensor utilities.
- OpenBLAS is used for dense linear algebra, and OpenMP is used for parallel loops and reductions.
8. CUDA acceleration subsystem
- engine_cuda/ builds into seera_cuda.
- It contains handwritten CUDA kernels for GEMM, convolution, transposed convolution, activations, reductions, pooling, broadcasting, upsampling, and tensor utility ops.
- The README emphasizes Tensor Core WMMA usage, so matrix multiply and convolution are the performance-critical GPU paths.
9. Python-to-native bridge
- Both native engines are exposed through pybind11 wrappers: engine/src/bindings.cpp and engine_cuda/src/cuda_bindings.cpp.
- The bindings move NumPy arrays to and from device/host memory, expose kernel entry points, and treat GPU pointers as Python-accessible handles.
10. Data flow during training
- User code creates a model in Python, which instantiates Tensor objects and layer state.
- Forward passes dispatch to CPU or GPU kernels.
- The loss tensor triggers autograd, which walks the graph backward and accumulates gradients.
- Optimizers update weights in place, staying on device when running on CUDA.
11. Model and state persistence
- Save/load uses Python pickle.
- When serializing GPU models, parameters are transferred back to host form for persistence, then reconstructed on load.
12. Validation and tooling
- Testing/ contains diagnostics, unit-style scripts, and MNIST comparison runs against PyTorch.
- benchmark.py provides a performance/convergence reference, while the notebooks are exploratory demos.
- build_engine.py and build_engine_cuda.py are the build boundaries for the two native shared libraries.
13. Useful diagram boundaries
- A clear architecture diagram should separate:
- Python API and autograd control plane
- CPU native backend
- GPU native backend
- Memory/transport bridge
- The key flow is: user API → tensor graph → autograd engine → backend dispatch → native kernels → gradients/optimizer updates.
Main entry points: Seera_init.py Tensor core · Seera.py Seera API · cuda_bindings.cpp GPU binding · Seera_Engine.py Autograd engine
Components 17
Python control plane
Public API, tensor graph, and autograd orchestration
- Seera API · High-level layers, Sequential, losses, optimizers, training, prediction, and persistence
Seera.py - Tensor core · Tensor abstraction, operator overloading, forward ops, and graph bookkeeping
Seera_init.py - Autograd engine · Builds the backward graph, topological ordering, and gradient dispatch
Seera_Engine.py - CUDA tensor · Device-pointer tensor wrapper and on-device tensor operations
cuTen.py
CPU backend
Native C++ execution path
- CPU binding · Python bindings into the C++ backend
engine/src/bindings.cpp - CPU tensor ops · Tensor utilities and host-side tensor mechanics
engine/src/tensor_ops.cpp - CPU kernels · Matmul, activations, convolution, pooling, and batchnorm
engine/src/activation_ops.cpp - CPU engine API · Public native engine interface for the CPU backend
engine/include/seera_engine.hpp
GPU backend
Native CUDA execution path
- GPU binding · Python bindings into the CUDA backend
engine_cuda/src/cuda_bindings.cpp - GPU runtime · Device memory management, pointer handling, and tensor utilities
engine_cuda/src/cuTen_essentails.cu - GEMM kernels · Tensor Core-aware matrix multiply and related dense linear algebra
engine_cuda/src/GEMM.cu - GPU ops · Activations, convolution, reduction, pooling, broadcasting, and resizing ops
engine_cuda/src/activations.cu - GPU engine API · Public native engine interface for the CUDA backend
engine_cuda/include/seera_engine_cuda.hpp
Tooling
Build, benchmark, and validation assets
- Build CPU · Compiles the C++ extension module
build_engine.py - Build GPU · Compiles the CUDA extension module
build_engine_cuda.py - Benchmark · Performance and convergence reference script
benchmark.py - Diagnostics · MNIST comparison runs, diagnostic scripts, and exploratory checks
Testing/mnist_seera_vs_pytorch.py
Connections 19
- Seera API → Tensor core: builds models · User-facing layers instantiate and manipulate tensors
- Tensor core → Autograd engine: records graph · Tensor ops create nodes for backward differentiation
- Autograd engine → CPU kernels: CPU backward · Dispatches CPU gradient logic to native kernels
- Autograd engine → GPU ops: GPU backward · Dispatches GPU gradient logic to CUDA kernels
- Tensor core → CUDA tensor: device tensors · GPU-backed tensors wrap device pointers and stay on device
- Tensor core → CPU binding: CPU dispatch · Forwards host execution to the C++ extension
- Tensor core → GPU binding: GPU dispatch · Forwards device execution to the CUDA extension
- CPU binding → CPU tensor ops: loads ops · Binding exposes host tensor utilities and tensor state
- CPU binding → CPU kernels: calls kernels · Binding exposes compute kernels to Python
- GPU binding → GPU runtime: manages memory · Binding exposes device memory and tensor handles
- GPU binding → GEMM kernels: launches GEMM · Binding exposes dense matrix execution paths
- GPU binding → GPU ops: launches ops · Binding exposes the broader CUDA kernel suite
- CPU kernels → CPU engine API: implements API · Native CPU sources conform to the shared engine interface
- GPU ops → GPU engine API: implements API · CUDA sources conform to the shared engine interface
- Seera API → Build CPU: loads extension · CPU backend is built and linked as an optional native module
- Seera API → Build GPU: loads extension · GPU backend is built and linked as an optional native module
- Benchmark → Seera API: measures · Benchmark exercises training and inference paths
- Diagnostics → Seera API: validates · Diagnostics compare behavior against reference runs
- Diagnostics → Autograd engine: checks gradients · Validation scripts probe backward correctness