# satyamdixit6/cuddly-octo-spoon architecture

> Architecture diagram of the GitHub repository satyamdixit6/cuddly-octo-spoon, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/satyamdixit6/cuddly-octo-spoon
- Repository: https://github.com/satyamdixit6/cuddly-octo-spoon

## Overview

##### Project Type and Purpose
- This is a CUDA‐based “Function Showcase” repository: a collection of standalone CUDA demos (vector add, matrix multiply, image filters, reduction, plus larger models like transformers and DIT) plus Python wrappers and build scripts.
- Purpose: educational/testbed for CUDA kernels, plus simple Python/C++ integration (e.g., transformer modules).

##### File Structure Analysis
- Top‐level directories/modules:
- “DIT” – custom Deep Inference Transformer kernels in CUDA and Python.
- “flashattention” – a high‐performance attention implementation in CUDA.
- “transformer” & “transformer_cuda” – larger transformer models (MOE, simple LLM) with C++/Python glue.
- Various standalone .cu files (Vector_Add.cu, Vaector_Add.cu, matrix‐mul demos, image‐proc, etc.).
- Python scripts & setup.py for building Python extensions.
- Build/configuration: multiple setup.py files under transformer/simpleLLM and transformer_cuda, and nvcc invocation comments in .cu files.

##### Key README Insights
- Highlights purpose: showcase different parallel techniques.
- Emphasizes per‐directory compilation via nvcc, and Python integration via setup.py.

##### Relationships and Data Flow
1. User runs script →
2. Python loads C++/CUDA extension (via import) →
3. Extension launches nvcc‐compiled GPU kernels →
4. GPU executes kernels on device memory →
5. Results return to host and back to Python/CLI.
- For standalone demos, script/shell invokes nvcc → binary → GPU.

##### Architectural Patterns
- Modular plugins: each CUDA demo isolated in its own directory/file.
- Python‐C++ extension pattern: setup.py + compiled .o files.
- Hierarchical transformer design: separate kernel layers (attention, MLP, norm) then composed in higher‐level blocks.

##### Technologies & Frameworks
- CUDA Toolkit / nvcc
- C++ compiler (g++, MSVC)
- Python 3.12, setuptools (setup.py)
- Ninja build files (in simpleLLM build)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Tier 1: User & Host Environment
    subgraph "Tier 1: User & Host Environment"
        control("control_test.py"):::python
        inference("inference.py"):::python
        trans_orch("transformer.py"):::python
        pipeline("transformer_pipeline.py"):::python
        tcuda("traset.py"):::python
        test_model("test_model.py"):::python
        demo_driver("traset.py (standalone)"):::python
        nb1("inn.ipynb"):::python
        nb2("jupyter_test.ipynb"):::python
        host("Host OS & CUDA Toolkit"):::host
    end

    %% Tier 2: Compiled Extensions & Libraries
    subgraph "Tier 2: Compiled Extensions & Libraries"
        simple_setup("transformer/simpleLLM/setup.py"):::extension
        att_o("attention.o"):::extension
        moe_o("moe.o"):::extension
        trans_o("transformer.o"):::extension

        moekv_setup("kv_moe_transformer setup.py"):::extension
        tcuda_cu("transformer_cuda.cu"):::extension

        demo_bins("Standalone CUDA Binaries"):::extension
    end

    %% Tier 3: GPU Kernels & Hardware
    subgraph "Tier 3: GPU Kernels & Hardware"
        DIT_src("DIT/src/cuda/*.cu"):::cuda
        util_kernels("DIT/src/cuda/cuda_util_kernels.cu & DIT/utils.cu"):::cuda
        orig_dit("DIT original demos (*.cu)"):::cuda

        flash_mod("flashattention/*.cu & .h"):::cuda
        trans_base("transformer/*.cu"):::cuda
        simpleLLM_cuda("transformer/simpleLLM/*.cu & .h"):::cuda

        hardware("NVIDIA GPU Device"):::hardware
    end

    %% Runtime Data Flow
    control -->|runs on| host
    inference -->|runs on| host
    trans_orch -->|runs on| host
    pipeline -->|runs on| host
    tcuda -->|runs on| host
    test_model -->|runs on| host
    demo_driver -->|runs on| host
    nb1 -->|runs on| host
    nb2 -->|runs on| host

    host -->|imports| att_o
    host -->|imports| moe_o
    host -->|imports| trans_o
    host -->|imports| tcuda_cu
    host -->|executes| demo_bins

    att_o -->|launches kernels| hardware
    moe_o -->|launches kernels| hardware
    trans_o -->|launches kernels| hardware
    tcuda_cu -->|launches kernels| hardware
    demo_bins -->|launches kernels| hardware

    hardware -->|results| host

    %% Build/Deploy Flows
    host -.-> simple_setup
    host -.-> moekv_setup
    host -.-> demo_bins

    simple_setup -.-> att_o
    simple_setup -.-> moe_o
    simple_setup -.-> trans_o

    moekv_setup -.-> tcuda_cu

    %% Kernel Modules Dependencies
    att_o --> DIT_src
    moe_o --> DIT_src
    trans_o --> DIT_src

    tcuda_cu --> flash_mod
    trans_base --> simpleLLM_cuda
    simpleLLM_cuda --> util_kernels
    orig_dit --> util_kernels

    %% Click Events
    click control "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/DIT/control_test.py"
    click inference "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/DIT/src/cuda/inference.py"
    click trans_orch "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer.py"
    click pipeline "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer/simpleLLM/transformer_pipeline.py"
    click tcuda "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer_cuda/traset.py"
    click test_model "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer_cuda/moe_kv_transformer/kv_moe_transformer/test_model.py"
    click demo_driver "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/traset.py"
    click nb1 "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/DIT/inn.ipynb"
    click nb2 "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer/simpleLLM/jupyter_test.ipynb"
    click simple_setup "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer/simpleLLM/setup.py"
    click att_o "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer/simpleLLM/build/temp.linux-x86_64-cpython-312/attention.o"
    click moe_o "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer/simpleLLM/build/temp.linux-x86_64-cpython-312/moe.o"
    click trans_o "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer/simpleLLM/build/temp.linux-x86_64-cpython-312/transformer.o"
    click moekv_setup "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer_cuda/moe_kv_transformer/kv_moe_transformer/setup.py"
    click tcuda_cu "https://github.com/satyamdixit6/cuddly-octo-spoon/blob/main/transformer_cuda/moe_kv_transformer/kv_moe_transformer/transformer_cuda.cu"

    %% Styles
    classDef python fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef host fill:#E0E0E0,stroke:#000,stroke-width:1px
    classDef extension fill:#90EE90,stroke:#000,stroke-width:1px
    classDef cuda fill:#FFA500,stroke:#000,stroke-width:1px
    classDef hardware fill:#D3D3D3,stroke:#000,stroke-width:1px
```
