# pytorch/glow architecture

> Architecture diagram of the GitHub repository pytorch/glow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pytorch/glow
- Repository: https://github.com/pytorch/glow

## Overview

##### Main Components
- Frontend / Model Importers:
– Modules and libraries for importing models from different frameworks such as Caffe2, ONNX, TFLite and PyTorch (as evidenced in directories like “include/glow/Importer”, “lib/Importer” and the torch_glow folder).
- Intermediate Representation (IR):
– The core of the compiler where neural network graphs are first converted into a high-level IR (documented in docs/IR.md) and then lowered to a low-level, instruction-based IR.
– Include components such as the IR builder, IR optimizer (in “lib/IR” and “include/glow/IR”), and various passes (graph optimizers, function passes, lower passes, etc).
- Optimization Pipeline:
– Specialized passes for domain-specific optimizations such as constant folding, node splitting, quantization, and training preparations.
– Separate optimization flows are implemented in “lib/Optimizer” and “include/glow/Optimizer”.
- Code Generation and Backend Components:
– Several distinct backends (CPU, NNPI, Habana, Interpreter, OpenCL, etc.), each of which includes code-generation logic (LLVMIR generation, JIT, ahead-of-time compilation for bundles) and device-specific runtime support.
– These are found in both “include/glow/Backends” and “lib/Backends”.
– Represent the connection from the lower-level IR to the hardware-specific code generation, for example via LLVMIRCodeGen modules.
- Runtime and Execution Engine:
– Execution engine components that run the compiled neural network computation, including device managers, executor threads/pools, error reporters, and trace/statistics exporters.
– See directories like “include/glow/Runtime” and “lib/Runtime”.
- Tools and Utilities:
– A suite of supporting tools such as the network debugger, loader tools, model compilers, AOT bundle creators (“tools/”, “examples/bundles”), and testing/benchmark frameworks (“tests/”, “examples/”).
– Also include code generators (e.g., in “tools/ClassGen”) along with profiling and debugging utilities.

##### Relationships and Interactions
- Two-phase IR lowering process: a high-level IR that feeds the domain-specific optimizer, which then lowers to a machine-level IR.
- That the optimized IR is passed to a backend-specific code generator and then to the runtime/execution engine running on various hardware targets.
- That multiple backends act as interchangeable modules—and have dependencies with shared IR and optimization layers.

##### Architectural Patterns and Major Technologies
- Layered / Pipeline Architecture: Separate the compilation process into discrete stages (import, IR generation, optimization, backend code generation, and runtime execution).
- Modular Components & Plug-in Backends: Emphasize the modularity by showing separate “modules” for different hardware targets (CPU, NNPI, Habana, OpenCL).
- Use of CMake for build orchestration, with third-party libraries and submodules (e.g., googletest, onnx, fmt).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Frontend / Model Importers"
        MI1["Include Importer"]:::frontend
        MI2["Lib Importer"]:::frontend
        MI3["Torch Glow (PyTorchModelLoader.cpp)"]:::frontend
    end

    subgraph "Intermediate Representation (IR)"
        IR1["Include IR"]:::ir
        IR2["Lib IR"]:::ir
        IR3["IR Documentation"]:::ir
    end

    subgraph "Optimization Pipeline"
        OP1["Include Optimizer"]:::optimizer
        OP2["Lib Optimizer"]:::optimizer
    end

    subgraph "Code Generation and Backends"
        CG1["Include Backends"]:::backend
        CG2["Lib Backends"]:::backend
        LLVM1["Include LLVM IR CodeGen"]:::backend
        LLVM2["Lib LLVM IR CodeGen"]:::backend
    end

    subgraph "Runtime and Execution Engine"
        RT1["Include Runtime"]:::runtime
        RT2["Lib Runtime"]:::runtime
    end

    subgraph "Tools and Utilities"
        TL1["Tools"]:::tools
        TL2["Examples"]:::tools
        TL3["Tests"]:::tools
        TL4["Utils"]:::tools
    end

    %% Main Flow Connections
    MI1 --> IR1
    MI2 --> IR1
    MI3 --> IR1

    IR1 --> OP1
    IR1 --> OP2
    IR2 --> OP1
    IR2 --> OP2
    IR3 --> OP1
    IR3 --> OP2

    OP1 --> CG1
    OP2 --> CG2
    OP1 --> LLVM1
    OP2 --> LLVM2

    CG1 --> RT1
    CG2 --> RT1
    LLVM1 --> RT1
    LLVM2 --> RT1

    RT1 --> RT2

    %% Tools Interface (dashed lines)
    TL1 --- MI1
    TL1 --- IR1
    TL1 --- OP1
    TL1 --- CG1
    TL1 --- RT1

    %% Styles
    classDef frontend fill:#f9f5d7,stroke:#333,stroke-width:2px;
    classDef ir fill:#d0e6a5,stroke:#333,stroke-width:2px;
    classDef optimizer fill:#ffcccb,stroke:#333,stroke-width:2px;
    classDef backend fill:#add8e6,stroke:#333,stroke-width:2px;
    classDef runtime fill:#ffd700,stroke:#333,stroke-width:2px;
    classDef tools fill:#e6e6fa,stroke:#333,stroke-width:2px;

    class MI1,MI2,MI3 frontend;
    class IR1,IR2,IR3 ir;
    class OP1,OP2 optimizer;
    class CG1,CG2,LLVM1,LLVM2 backend;
    class RT1,RT2 runtime;
    class TL1,TL2,TL3,TL4 tools;

    %% Click Events
    click MI1 "https://github.com/pytorch/glow/tree/master/include/glow/Importer"
    click MI2 "https://github.com/pytorch/glow/tree/master/lib/Importer"
    click MI3 "https://github.com/pytorch/glow/blob/master/torch_glow/src/PyTorchModelLoader.cpp"
    click IR1 "https://github.com/pytorch/glow/tree/master/include/glow/IR"
    click IR2 "https://github.com/pytorch/glow/tree/master/lib/IR"
    click IR3 "https://github.com/pytorch/glow/blob/master/docs/IR.md"
    click OP1 "https://github.com/pytorch/glow/tree/master/include/glow/Optimizer"
    click OP2 "https://github.com/pytorch/glow/tree/master/lib/Optimizer"
    click CG1 "https://github.com/pytorch/glow/tree/master/include/glow/Backends"
    click CG2 "https://github.com/pytorch/glow/tree/master/lib/Backends"
    click LLVM1 "https://github.com/pytorch/glow/tree/master/include/glow/LLVMIRCodeGen"
    click LLVM2 "https://github.com/pytorch/glow/tree/master/lib/LLVMIRCodeGen"
    click RT1 "https://github.com/pytorch/glow/tree/master/include/glow/Runtime"
    click RT2 "https://github.com/pytorch/glow/tree/master/lib/Runtime"
    click TL1 "https://github.com/pytorch/glow/tree/master/tools/"
    click TL2 "https://github.com/pytorch/glow/tree/master/examples/bundles"
    click TL3 "https://github.com/pytorch/glow/tree/master/tests/"
    click TL4 "https://github.com/pytorch/glow/tree/master/utils/"
```
