How pytorch/glow works
Main Components
• Frontend / Model Importers:
– Modules and libraries for importing models from different frameworks such as Caffe2, ONNX, TFLite and PyTorch (as evidenced in directories like “include/glow/Importer”, “lib/Importer” and the torch_glow folder).
• Intermediate Representation (IR):
– The core of the compiler where neural network graphs are first converted into a high-level IR (documented in docs/IR.md) and then lowered to a low-level, instruction-based IR.
– Include components such as the IR builder, IR optimizer (in “lib/IR” and “include/glow/IR”), and various passes (graph optimizers, function passes, lower passes, etc).
• Optimization Pipeline:
– Specialized passes for domain-specific optimizations such as constant folding, node splitting, quantization, and training preparations.
– Separate optimization flows are implemented in “lib/Optimizer” and “include/glow/Optimizer”.
• Code Generation and Backend Components:
– Several distinct backends (CPU, NNPI, Habana, Interpreter, OpenCL, etc.), each of which includes code-generation logic (LLVMIR generation, JIT, ahead-of-time compilation for bundles) and device-specific runtime support.
– These are found in both “include/glow/Backends” and “lib/Backends”.
– Represent the connection from the lower-level IR to the hardware-specific code generation, for example via LLVMIRCodeGen modules.
• Runtime and Execution Engine:
– Execution engine components that run the compiled neural network computation, including device managers, executor threads/pools, error reporters, and trace/statistics exporters.
– See directories like “include/glow/Runtime” and “lib/Runtime”.
• Tools and Utilities:
– A suite of supporting tools such as the network debugger, loader tools, model compilers, AOT bundle creators (“tools/”, “examples/bundles”), and testing/benchmark frameworks (“tests/”, “examples/”).
– Also include code generators (e.g., in “tools/ClassGen”) along with profiling and debugging utilities.
Relationships and Interactions
• Two-phase IR lowering process: a high-level IR that feeds the domain-specific optimizer, which then lowers to a machine-level IR.
• That the optimized IR is passed to a backend-specific code generator and then to the runtime/execution engine running on various hardware targets.
• That multiple backends act as interchangeable modules—and have dependencies with shared IR and optimization layers.
Architectural Patterns and Major Technologies
• Layered / Pipeline Architecture: Separate the compilation process into discrete stages (import, IR generation, optimization, backend code generation, and runtime execution).
• Modular Components & Plug-in Backends: Emphasize the modularity by showing separate “modules” for different hardware targets (CPU, NNPI, Habana, OpenCL).
• Use of CMake for build orchestration, with third-party libraries and submodules (e.g., googletest, onnx, fmt).