# wangyu-/mlsys20_workshop architecture

> Architecture diagram of the GitHub repository wangyu-/mlsys20_workshop, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wangyu-/mlsys20_workshop
- Repository: https://github.com/wangyu-/mlsys20_workshop

## Overview

##### Project type and purpose
- This is an open-source research tool for “Energy-aware DNN Graph Optimization,” implemented in C++ with CUDA/cuDNN and Intel MKL backends. It provides:
- A core graph-representation and operator library
- A graph-substitution/optimization engine
- Power-measurement scripts and a small database of results
- Export utilities (e.g., to TensorRT)
- Its goal is to analyze, measure, and optimize DNN compute graphs for energy efficiency.

##### File structure
- Top-level:
- README.md, module.txt, qsub.txt (docs, submission hints)
- code/ (all C++ source, CUDA kernels, MKL-backed kernels, operator definitions, Makefile, scripts, data files)
- notused/ (legacy Python executor, install script)
- In code/:
- operator implementations: conv2d.*, pool2d.*, matmul.*, batchnorm.*, activation.*, concat.*
- Three backends: _cudnn.cu, _mkl.cc, pure CC versions
- Graph optimization: substitution.cc/.h, subst_examples.*
- Measurement tools and scripts: manual_measure.sh, measure_power.sh, db.txt
- Export to other frameworks: graph_to_trt.cc, export_graphs.sh
- Build: Makefile
- No classic frontend/backend split—this is a library/CLI tool with multiple modules.

##### README
- Confirms it’s a reproducible code base for the MLSys’20 workshop paper.
- Points to an online Wiki for architecture details.
- Notes it is based on MetaFlow (another graph optimizer).

##### Main components & their relationships
a. Core Graph Library
- Data structures (nodes, edges, operator definitions: code/ops.h, densenet.h, resnet.h, etc.)
b. Operator Backends
- cuDNN kernels (code/*_cudnn.cu)
- MKL kernels (code/*_mkl.cc)
- Reference C++ kernels (code/*.cc)
c. Graph Optimization Engine
- subst_examples.{cc,h}: sample transformations
d. Measurement & Profiling Module
- manual_measure.sh, measure_power.sh: harness to run specific operators or graphs and record power/time
- db.txt: stores measurement results
e. Export & Integration
- graph_to_trt.cc, export_graphs.sh: convert optimized graphs to TensorRT representations
f. Build & Glue
- code/Makefile
- cuda_helper.h/.cu, mkl_helper.h for abstraction

##### Architectural patterns and design principles
- Modular backends: each operator can be plugged in with either a CUDA, MKL, or reference implementation.
- Rule-based graph rewriting: the substitution engine applies transformation rules at the graph level.
- Data-driven tuning: measurement results in db.txt guide which substitutions to apply.
- Scripted orchestration: shell scripts glue together build, measurement, optimization, and export steps.

- Graph definitions → SubstitutionEngine
- SubstitutionEngine → Selected Backend Plugin
- Backend plugin → Measurement Harness → Database
- Optimizer ← Measurement results from Database
- Optimized Graph → Exporter → External Framework (TensorRT)
- Blue: Core library components
- Green: Backends
- Orange: Measurement & storage
- Purple: Export/integration
- Rectangles for modules

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layer 1: Orchestration
    subgraph "1. Orchestration Layer"
        MM[/"manual_measure.sh"/]:::script
        EG[/"export_graphs.sh"/]:::script
    end

    %% Layer 2: Core Library
    subgraph "2. Core Library Layer"
        GR["GraphRepresentation"]:::core
        SE["SubstitutionEngine"]:::core
    end

    %% Layer 3: Backend Plugins
    subgraph "3. Backend Plugins Layer"
        subgraph "cuDNN Plugin"
            C1["conv2d_cudnn.cu"]:::backend
            C2["activation_cudnn.cu"]:::backend
            C3["batchnorm_cudnn.cu"]:::backend
            C4["concat_cudnn.cu"]:::backend
            C5["element_cudnn.cu"]:::backend
            C6["matmul_cudnn.cu"]:::backend
            C7["ops_cudnn.cu"]:::backend
            C8["pool2d_cudnn.cu"]:::backend
        end
        subgraph "MKL Plugin"
            M1["conv2d_mkl.cc"]:::backend
            M2["activation_mkl.cc"]:::backend
            M3["batchnorm_mkl.cc"]:::backend
            M4["element_mkl.cc"]:::backend
            M5["matmul_mkl.cc"]:::backend
            M6["ops_mkl.cc"]:::backend
            M7["pool2d_mkl.cc"]:::backend
        end
        subgraph "Reference CPU Plugin"
            R1["conv2d.cc"]:::backend
            R2["activation.cc"]:::backend
            R3["batchnorm.cc"]:::backend
            R4["concat.cc"]:::backend
            R5["element.cc"]:::backend
            R6["matmul.cc"]:::backend
            R7["ops.cc"]:::backend
            R8["pool2d.cc"]:::backend
            R9["cnn.cc"]:::backend
            R10["noop.cc"]:::backend
            R11["split.cc"]:::backend
        end
    end

    %% Layer 4: Measurement & Database
    subgraph "4. Measurement & Database Layer"
        MP[/"measure_power.sh"/]:::measurement
        DB[(db.txt)]:::measurement
    end

    %% Layer 5: Export
    subgraph "5. Export Layer"
        TRT["TensorRT Exporter\n(graph_to_trt.cc)"]:::export
    end

    %% Connections
    MM -->|"run benchmarks"| MP
    GR -->|"Graph AST"| SE
    SE -->|"invoke kernels"| C1
    SE -->|"invoke kernels"| M1
    SE -->|"invoke kernels"| R1
    C1 -->|"execute"| MP
    C2 -->|"execute"| MP
    C3 -->|"execute"| MP
    C4 -->|"execute"| MP
    C5 -->|"execute"| MP
    C6 -->|"execute"| MP
    C7 -->|"execute"| MP
    C8 -->|"execute"| MP
    M1 -->|"execute"| MP
    M2 -->|"execute"| MP
    M3 -->|"execute"| MP
    M4 -->|"execute"| MP
    M5 -->|"execute"| MP
    M6 -->|"execute"| MP
    M7 -->|"execute"| MP
    R1 -->|"execute"| MP
    R2 -->|"execute"| MP
    R3 -->|"execute"| MP
    R4 -->|"execute"| MP
    R5 -->|"execute"| MP
    R6 -->|"execute"| MP
    R7 -->|"execute"| MP
    R8 -->|"execute"| MP
    R9 -->|"execute"| MP
    R10 -->|"execute"| MP
    R11 -->|"execute"| MP
    MP -->|"Results"| DB
    DB -->|"Measurement Data"| SE
    SE -->|"Optimized Graph"| TRT
    EG -->|"trigger export"| TRT

    %% Click Events
    click GR "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/ops.h"
    click GR "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/ops.cc"
    click SE "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/substitution.h"
    click SE "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/substitution.cc"
    click SE "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/subst_examples.h"
    click SE "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/subst_examples.cc"
    click C1 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/conv2d_cudnn.cu"
    click C2 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/activation_cudnn.cu"
    click C3 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/batchnorm_cudnn.cu"
    click C4 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/concat_cudnn.cu"
    click C5 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/element_cudnn.cu"
    click C6 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/matmul_cudnn.cu"
    click C7 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/ops_cudnn.cu"
    click C8 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/pool2d_cudnn.cu"
    click M1 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/conv2d_mkl.cc"
    click M2 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/activation_mkl.cc"
    click M3 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/batchnorm_mkl.cc"
    click M4 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/element_mkl.cc"
    click M5 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/matmul_mkl.cc"
    click M6 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/ops_mkl.cc"
    click M7 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/pool2d_mkl.cc"
    click R1 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/conv2d.cc"
    click R2 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/activation.cc"
    click R3 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/batchnorm.cc"
    click R4 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/concat.cc"
    click R5 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/element.cc"
    click R6 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/matmul.cc"
    click R7 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/ops.cc"
    click R8 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/pool2d.cc"
    click R9 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/cnn.cc"
    click R10 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/noop.cc"
    click R11 "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/split.cc"
    click MP "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/measure_power.sh"
    click DB "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/db.txt"
    click MM "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/manual_measure.sh"
    click EG "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/export_graphs.sh"
    click TRT "https://github.com/wangyu-/mlsys20_workshop/blob/master/code/graph_to_trt.cc"

    %% Styles
    classDef core fill:#D4E6F1,stroke:#2E86C1,color:#1B4F72;
    classDef backend fill:#D5F5E3,stroke:#27AE60,color:#1E8449;
    classDef measurement fill:#FCF3CF,stroke:#F1C40F,color:#B7950B;
    classDef export fill:#F5EEF8,stroke:#8E44AD,color:#5B2C6F;
    classDef script fill:#F2F3F4,stroke:#ABB2B9,color:#5D6D7E,stroke-dasharray: 5 5;
```
