# mariosieg/magnetron architecture

> Architecture diagram of the GitHub repository mariosieg/magnetron, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mariosieg/magnetron
- Repository: https://github.com/mariosieg/magnetron

## Overview

##### Project Type & Purpose
- This is a deep learning framework focused on providing core tensor operations, neural network building blocks, and automatic differentiation.
- It is developed as an educational “homemade” alternative to PyTorch, offering both a performance-critical C core and a modern Python API.

##### File Structure
- Core C Library ("magnetron"): Contains source files in C (e.g., magnetron.c, magnetron.h, several CPU-specific and CUDA implementations) that implement the core functionality. The presence of multiple versions (e.g., BLAS for different architectures, CUDA files, internal registry) indicates a design with multiple computational backends (multi-threaded CPU, SIMD, GPU).
- Python Binding/Framework ("python/magnetron_framework"): Provides the Python API via CFFI, including scripts to build and install a wheel, core modules (core.py, model.py), and FFI glue (lib_loader.py, _ffi_cdecl_generated.py). Other Python subdirectories include examples, a viewer, chat server, research experiments, and benchmarks.
- Benchmarks and Testing: The "benchmark" folder (with nanobench integration) and the various C/C++ tests in the "test" directory (including unit tests using Google Test) indicate rigorous performance and correctness testing.
- Build and Configuration: The CMake files in the root, cmake/, and in subdirectories (e.g., for fuzzer and tests) reveal that the project uses CMake with custom modules (accelerate, cuda, blas_tune, etc.) for cross-platform builds.
- Additional Tools: The "fuzzer" directory and the "tools/install_deps_linux.sh" script provide automation and robustness testing support.

##### Key Components & Their Interactions
- Core Engine (C Library):
- Computation engine with modular backend implementations for CPU (multithreading, SIMD optimizations) and GPU (CUDA).
- Implements automatic differentiation and other tensor operations.
- Interaction: Provides the primitives called from higher-level languages.
- Python Binding Layer (magnetron_framework):
- Acts as the bridge between the high-performance C library and a modern Python API.
- Uses CFFI to interface with C code.
- Interaction: Receives calls from Python examples, experiments, and networking (chat) applications.
- Python Viewer & Examples:
- Interaction: Utilize the Python API provided by magnetron_framework.
- Benchmarking & Testing:
- Benchmarking modules (in both C++ and Python directories) track performance and verify that the computational paths (including different backends) perform as expected.
- The fuzzer ensures robustness and helps catch edge-case bugs.
- Interaction: Interconnected with the core engine to validate functionality.
- Build & CI Infrastructure:
- The CMake scripts along with GitHub Actions workflows manage the build process for multiple platforms.
- Interaction: Ensures that the various components (C core, Python wheel, tests) integrate smoothly.
- External Dependencies:
- “extern” contains third-party single-header libraries (like stb_image) used by the core engine.
- “datasets” such as the MNIST dataset support practical experiments and example training scenarios.

##### Specific Elements
- API calls from Python to the C library.
- Data flow and feedback between tensor operations (forward, backward passes).
- Build/configuration flows feeding into the compilation of the core engine and Python components.
- Core modules versus auxiliary tools (benchmarks/tests).
- Different layers (C layer, Python interface, user-level applications).
- Annotations or legends to indicate specific technologies (C99, CUDA, CFFI, CMake).

Happy diagramming!

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Core Layer"
        CoreC["Core C Library"]:::core
    end

    subgraph "Python Binding Layer"
        PyBind["Python Binding Layer"]:::python
    end

    subgraph "Python Applications"
        PyEx["Examples"]:::apps
        PyChat["Chat Application"]:::apps
        PyViewer["Viewer"]:::apps
    end

    subgraph "Benchmarking & Testing"
        Bench["Benchmarks"]:::testing
        PyTest["Python Tests"]:::testing
        CTest["C/C++ Tests"]:::testing
    end

    subgraph "Build & CI"
        BuildCI["Build & CI Infrastructure"]:::build
    end

    subgraph "External Dependencies"
        External["External Dependencies"]:::external
    end

    subgraph "Datasets"
        Datasets["Datasets"]:::dataset
    end

    subgraph "Fuzzer"
        Fuzzer["Fuzzer Support"]:::fuzzer
    end

    %% Relationships
    PyBind -->|"calls"| CoreC
    PyEx   -->|"calls"| PyBind
    PyChat -->|"calls"| PyBind
    PyViewer -->|"calls"| PyBind
    Bench  -->|"validates"| CoreC
    PyTest -->|"tests"| PyBind
    CTest  -->|"tests"| CoreC
    BuildCI -->|"builds"| CoreC
    BuildCI -->|"builds"| PyBind
    CoreC  -->|"uses"| External
    CoreC  -->|"uses"| Datasets
    Fuzzer -->|"fuzzes"| CoreC

    %% Click Events
    click CoreC "https://github.com/mariosieg/magnetron/tree/master/magnetron/"
    click PyBind "https://github.com/mariosieg/magnetron/tree/master/python/magnetron_framework/"
    click PyEx "https://github.com/mariosieg/magnetron/tree/master/python/examples/"
    click PyChat "https://github.com/mariosieg/magnetron/tree/master/python/chat/"
    click PyViewer "https://github.com/mariosieg/magnetron/tree/master/python/magnetron_viewer/"
    click Bench "https://github.com/mariosieg/magnetron/tree/master/benchmark/"
    click PyTest "https://github.com/mariosieg/magnetron/tree/master/python/tests/"
    click CTest "https://github.com/mariosieg/magnetron/tree/master/test/"
    click BuildCI "https://github.com/mariosieg/magnetron/blob/master/CMakeLists.txt"
    click External "https://github.com/mariosieg/magnetron/blob/master/extern/stb_image.h"
    click Datasets "https://github.com/mariosieg/magnetron/tree/master/datasets/mnist/"
    click Fuzzer "https://github.com/mariosieg/magnetron/tree/master/fuzzer/"

    %% Styles
    classDef core fill:#ffcccc,stroke:#cc0000,stroke-width:2px;
    classDef python fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef apps fill:#ccffcc,stroke:#008000,stroke-width:2px;
    classDef testing fill:#ffe5cc,stroke:#ff6600,stroke-width:2px;
    classDef build fill:#e5ccff,stroke:#6600ff,stroke-width:2px;
    classDef external fill:#f0f0f0,stroke:#808080,stroke-width:2px;
    classDef dataset fill:#d1f0d1,stroke:#228b22,stroke-width:2px;
    classDef fuzzer fill:#fadadd,stroke:#ff69b4,stroke-width:2px;
```
