# inducer/loopy architecture

> Architecture diagram of the GitHub repository inducer/loopy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/inducer/loopy
- Repository: https://github.com/inducer/loopy

## Overview

##### Project Type & Purpose
- Loopy is essentially a domain-specific compiler-like tool. It takes a high-level specification of array computations (through a Python interface and even Fortran frontends) and transforms them into optimized low-level code targeting multiple backends (e.g., C, OpenCL, CUDA, ISPC, and pyopencl).
- Its focus is on applying various loop transformations and optimizations (e.g., loop tiling, unrolling, data layout transformation, etc.) to generate high-performance code for both CPUs and GPUs.

##### File Tree — Identify Key Directories/Modules
- /loopy: Contains the core implementation and is subdivided into numerous modules:
- loopy/frontend: Implements the parsing of high-level user specifications, including for Fortran. Contains modules such as translator.py and tree.py that convert source code into an intermediate representation (IR).
- loopy/kernel: Represents the IR for computations. It includes submodules that define arrays, kernel creation, data description, and function interfaces.
- loopy/transform: This is the transformation engine. It contains multiple transformation passes (e.g., add_barrier, arithmetic, fusion, etc.) that modify the IR for optimization.
- loopy/codegen: Serves as the code generator portion. It has codegen tools that convert the transformed IR into target language code, handling bounds, control flow, and instruction-level details.
- loopy/target: Contains target-specific backends (C, CUDA, OpenCL, pyopencl, ispc). These modules are where the generated code is serialized and tailored for each hardware platform.
- loopy/library: Provides utility routines (like random123, reductions, etc.) that are shared across the system.
- Other modules (schedule, diagnostic, options, symbolic, types) complement the design by supporting scheduling, error reporting, configuration management, and type inference.

- Additional Directories:
- /examples: Contains both Python and Fortran examples that illustrate how to interface with Loopy.
- /test & /proto-tests: These directories house unit and integration tests that help ensure the consistency of the transformation and code generation stages.

a. Core Components to Include:
- Frontend Parsing Module (loopy/frontend):
- Input processors that handle array computations from Python/Fortran.
- Conversion of user code into an intermediate tree or IR.
- Intermediate Representation (IR) Module (loopy/kernel):
- Abstract representation of kernels including arrays, instructions, and data flow.
- Transformation Engine (loopy/transform):
- Various passes that apply optimizations and transformations (e.g., loop fusion, tiling, unrolling, etc.).
- Code Generation Module (loopy/codegen):
- Converts the IR into target-specific code fragments.
- Target Abstraction Layers (loopy/target):
- Separate modules for each supported backend (C, CUDA, ISPC, OpenCL, pyopencl), converting the generated code and handling device-specific execution details.
- Supportive Libraries (loopy/library, and other util modules):
- Shared routines and functions which provide math utilities, random number generation, reductions, etc.
- CLI and Execution Entry (loopy/cli.py, __main__.py):
- Interface for users to invoke the whole transformation process.

b. Relationships & Data Flow:
- The transformed IR then passes through multiple transformation passes (each transformation module can be represented as nodes in a layered pipeline).
- After transformations, the IR is fed to the Code Generation component that, based on configuration, selects an appropriate target backend.
- Finally, generated code is sent to the target-specific module for further optimizations and dispatch, eventually culminating in executable code for GPUs/CPUs.
- Feedback loops if any diagnostics or performance statistics (loopy/statistics.py) are collected and routed back to optimize further iterations.

c. Architectural Design Principles:
- Transformation Pipeline: Emphasize that the design recognizes a multi-stage flow—from the high-level specification to multiple transformation passes before final code emission.
- Modular Separation: Each module (frontend, kernel, transform, codegen, target) is relatively self-contained and interacts through clearly defined data structures (e.g., the IR/Kernel representation).
- Extensibility: The distinct directories for targets and transformations indicate the design is open for extension, allowing the addition of new optimization passes or backend support.

d. Technical Details & Technologies:
- Python is the primary language. The system’s design employs object-oriented and functional programming patterns.
- Dependency management configurations (e.g., requirements.txt, pyproject.toml) and CI/CD pipelines (in .ci, .github/workflows, .gitlab-ci.yml) support continuous testing and integration—all of which might be noted in an “Operations/DevOps” block if needed.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input and Frontend Layer
    subgraph "Input and Frontend"
        Input["User High-Level Kernel Specification"]:::input
        Frontend["Frontend Parser (loopy/frontend)"]:::frontend
        Translator["Fortran Translator"]:::frontendDetail
        AST["AST Builder (Fortran Tree)"]:::frontendDetail
    end

    %% Intermediate Representation Layer
    subgraph "Intermediate Representation (IR)"
        IR["Kernel IR (loopy/kernel)"]:::ir
    end

    %% Transformation Engine Layer
    subgraph "Transformation Engine"
        Transform["Transformation Engine (loopy/transform)"]:::transform
        AddBarrier["Transformation: add_barrier"]:::transformDetail
        Fusion["Transformation: fusion"]:::transformDetail
        Tiling["Transformation: tiling"]:::transformDetail
        Unrolling["Transformation: unrolling"]:::transformDetail
    end

    %% Code Generation Layer
    subgraph "Code Generation"
        CodeGen["Code Generation Module (loopy/codegen)"]:::codegen
        Bounds["Codegen: Bounds Analysis"]:::codegenDetail
        Control["Codegen: Control Flow"]:::codegenDetail
    end

    %% Target Abstraction Layers
    subgraph "Target Abstraction Layers"
        Target["Target Abstraction (loopy/target)"]:::target
        CBackend["C Backend"]:::targetDetail
        CUDABackend["CUDA Backend"]:::targetDetail
        OpenCLBackend["OpenCL Backend"]:::targetDetail
        ISPCBackend["ISPC Backend"]:::targetDetail
        PyOpenCLBackend["Pyopencl Backend"]:::targetDetail
    end

    %% Support and Execution Entry
    subgraph "Support & CLI"
        Lib["Supportive Libraries (loopy/library)"]:::library
        CLI["CLI & Execution Entry"]:::cli
    end

    %% Relationships / Data Flow
    Input -->|"parsed_by"| Frontend
    Frontend --> Translator
    Frontend --> AST
    Translator -->|"builds"| IR
    AST --> IR
    IR -->|"optimizes_via"| Transform
    Transform --> AddBarrier
    Transform --> Fusion
    Transform --> Tiling
    Transform --> Unrolling
    IR -->|"prepares_for"| CodeGen
    CodeGen --> Bounds
    CodeGen --> Control
    Bounds -->|"selects_target"| Target
    Control --> Target
    Target --> CBackend
    Target --> CUDABackend
    Target --> OpenCLBackend
    Target --> ISPCBackend
    Target --> PyOpenCLBackend
    CLI -->|"invokes"| Input
    Lib --- IR
    Lib --- Transform

    %% Click Events
    click Frontend "https://github.com/inducer/loopy/tree/main/loopy/frontend"
    click Translator "https://github.com/inducer/loopy/blob/main/loopy/frontend/fortran/translator.py"
    click AST "https://github.com/inducer/loopy/blob/main/loopy/frontend/fortran/tree.py"
    click IR "https://github.com/inducer/loopy/tree/main/loopy/kernel"
    click AddBarrier "https://github.com/inducer/loopy/blob/main/loopy/transform/add_barrier.py"
    click Fusion "https://github.com/inducer/loopy/blob/main/loopy/transform/fusion.py"
    click CodeGen "https://github.com/inducer/loopy/tree/main/loopy/codegen"
    click Bounds "https://github.com/inducer/loopy/blob/main/loopy/codegen/bounds.py"
    click Control "https://github.com/inducer/loopy/blob/main/loopy/codegen/control.py"
    click Target "https://github.com/inducer/loopy/tree/main/loopy/target"
    click CBackend "https://github.com/inducer/loopy/tree/main/loopy/target/c"
    click CUDABackend "https://github.com/inducer/loopy/blob/main/loopy/target/cuda.py"
    click OpenCLBackend "https://github.com/inducer/loopy/blob/main/loopy/target/opencl.py"
    click ISPCBackend "https://github.com/inducer/loopy/blob/main/loopy/target/ispc.py"
    click PyOpenCLBackend "https://github.com/inducer/loopy/blob/main/loopy/target/pyopencl.py"
    click Lib "https://github.com/inducer/loopy/tree/main/loopy/library"
    click CLI "https://github.com/inducer/loopy/blob/main/loopy/cli.py"
    click CLI "https://github.com/inducer/loopy/blob/main/loopy/__main__.py"

    %% Styling Classes
    classDef input fill:#f9e79f,stroke:#333,stroke-width:2px;
    classDef frontend fill:#aed6f1,stroke:#333,stroke-width:2px;
    classDef frontendDetail fill:#d6eaf8,stroke:#333,stroke-width:1px;
    classDef ir fill:#a9dfbf,stroke:#333,stroke-width:2px;
    classDef transform fill:#f9e79f,stroke:#333,stroke-width:2px;
    classDef transformDetail fill:#fcf3cf,stroke:#333,stroke-width:1px;
    classDef codegen fill:#f5cba7,stroke:#333,stroke-width:2px;
    classDef codegenDetail fill:#fdebd0,stroke:#333,stroke-width:1px;
    classDef target fill:#d2b4de,stroke:#333,stroke-width:2px;
    classDef targetDetail fill:#e8daef,stroke:#333,stroke-width:1px;
    classDef library fill:#d6eaf8,stroke:#333,stroke-width:2px;
    classDef cli fill:#f9e79f,stroke:#333,stroke-width:2px;
```
