# jittor/jittor architecture

> Architecture diagram of the GitHub repository jittor/jittor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jittor/jittor
- Repository: https://github.com/jittor/jittor

## Overview

##### Main components
- Front-End (Python API and Interface):
- This layer is built in Python (see the “python/jittor” directory). It exposes the user-facing APIs including model definition, autograd (through dynamic graph execution), and common deep learning constructs (e.g., neural network modules, dataset utilities, optimizers).
- Notebooks and tutorials (in “python/jittor/notebook”) illustrate usage and serve as an educational/test interface.

- Intermediate Engine and Execution Core:
- This includes core modules for dynamic graph construction, operation scheduling and execution, and the JIT compiling functionalities (files under “python/jittor/src” such as executor.cc, fused_op.cc, and jit_compiler.cc).
- The core also embeds key components like the operation registry, auto-differentiation mechanisms (grad related modules) and meta-operator handling.

- Op Compiler and Tuner:
- A specialized component that “compiles” high-level operations into high-performance code for a given hardware target.
- Files like “python/jittor/compiler.py”, “python/jittor/pyjt_compiler.py”, and the “extern” directories show integrations with CUDA, CUBLAS, and even other backends (such as MKL or ROCm).
- This component is responsible for generating optimized backend code on the fly.

- Backend and Platform-Specific Modules:
- Implemented in C++, CUDA, and other high-performance languages. Under “python/jittor/src” you see low-level code for memory allocators, fusion of operations, parallel execution (e.g., “src/mem”, “src/op*”) and device-specific integrations.
- The “extern” folder contains integrations with external libraries for GPU computations, including CUDA, cuDNN, cuBLAS, MKL, and others.

- Testing and Validation:
- A robust suite of tests (see “python/jittor/test” and “python/jittor/test/system”) covering unit tests, performance benchmarks, and system tests.

- Auxiliary Components:
- Documentation (in the “doc” folder) and helper scripts (in “python/jittor/script”) for building, installing, and debugging the framework.
- Other utilities such as notebooks, configuration and deployment files (e.g., Dockerfile, CI/CD workflows).

##### Relationships and Interactions
- The Front-End API layer (Python) collects model definitions, operations, and triggers the dynamic graph construction.
- From this layer, calls are forwarded to the Core Execution Engine, which uses the operation registry and graph executor.
- The JIT Compiler & Op Tuner analyze operations in the intermediate representation and interface with low-level back-end modules (via C++/CUDA glue code and pybind interfaces).
- Memory management modules, device-specific libraries, and runtime optimizers handle the actual execution on CPU and GPU, with feedback loops helping to auto-tune performance (e.g., tuner modules, “python/jittor/src/opt/tuner”).
- Testing and profiling components interact with the engine to ensure correctness and performance.

##### Architectural Patterns & Design Principles
- Layered Architecture: Clear separation between the Python front-end, intermediate dynamic graph engine, and low-level, hardware-optimized back-end.
- Just-in-Time (JIT) Compilation: Code generation based on the runtime graph, which allows for optimization tailored to the specific model.
- Modular Design: Each functional area (graph execution, memory management, op tuning, external library bindings) is organized in separate directories and modules.
- Integration of External Libraries: Directories under “extern” show how Jittor integrates with CUDA, cuDNN, MKL, and more.
- Emphasis on Performance and Tunability: Use of meta-operators and integrated tuners as key design principles.

##### Elements
- Clear labels for each component (e.g., “Python Front-End”, “JIT Compiler & Op Tuner”, “Dynamic Graph Executor”, “Memory Manager”, “CUDA/External Libraries”).

##### Additional Detailing
- That the framework is designed for high-performance deep learning tasks, and thus special attention should be given to components responsible for dynamic execution and JIT code generation.
- Sub-components for memory allocation (multiple allocators in “python/jittor/src/mem”), operation fusion (found in “src/fused_op”), and GPUs operations (under “extern/cuda/…”).
- Error-handling and testing hooks as part of the core operational flow.

- A Python front-end for defining and managing models.
- A high-performance core engine for dynamic graph execution and just-in-time compilation.
- A back-end that leverages C++/CUDA and external libraries to perform efficient computations.
- Dedicated modules for memory management, operation tuning, and extensive testing.

This approach should cover all essential architectural elements for the Jittor project.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Python Front-End Layer
    subgraph "Python Front-End"
        FE_API["Python API"]
        FE_Notebooks["Notebooks & Tutorials"]
    end

    %% Intermediate Core Engine Layer
    subgraph "Intermediate Core Engine"
        Core["Dynamic Graph Executor/AutoDiff"]
    end

    %% JIT Compiler & Op Tuner Layer
    subgraph "JIT Compiler & Op Tuner"
        Compiler["Op Compiler (compiler.py)"]
        PyJT["Op Compiler (pyjt_compiler.py)"]
        Tuner["Performance Tuner"]
    end

    %% Backend & Accelerators Layer
    subgraph "Backend & Accelerators"
        BackendOps["Low-level Operations (C++/CUDA)"]
        ExternLib["External Libraries Integration"]
    end

    %% Testing & DevOps Layer
    subgraph "Testing & DevOps"
        PyTest["Python Test Suite"]
        CTest["C++ Test Suite"]
    end

    %% Auxiliary Components Layer
    subgraph "Auxiliary Components"
        Documentation["Documentation"]
        Scripts["Helper Scripts"]
    end

    %% Connections between layers
    FE_Notebooks -->|"uses"| FE_API
    FE_API -->|"model_definition"| Core
    Core -->|"jit_compile"| Compiler
    Core -->|"jit_compile"| PyJT
    Core -->|"tuner_trigger"| Tuner
    Compiler -->|"code_generation"| BackendOps
    PyJT -->|"backend_interface"| BackendOps
    Tuner -->|"performance_opt"| BackendOps
    BackendOps -->|"external_calls"| ExternLib
    BackendOps -->|"feedback_loop"| Core
    Core -->|"monitored_by"| PyTest
    BackendOps -->|"monitored_by"| CTest
    Documentation -->|"supports"| FE_API
    Scripts -->|"supports"| FE_API

    %% Click Events
    click FE_API "https://github.com/jittor/jittor/tree/master/python/jittor"
    click FE_Notebooks "https://github.com/jittor/jittor/tree/master/python/jittor/notebook"
    click Core "https://github.com/jittor/jittor/tree/master/python/jittor/src"
    click Compiler "https://github.com/jittor/jittor/blob/master/python/jittor/compiler.py"
    click PyJT "https://github.com/jittor/jittor/blob/master/python/jittor/pyjt_compiler.py"
    click Tuner "https://github.com/jittor/jittor/tree/master/python/jittor/src/opt/tuner"
    click BackendOps "https://github.com/jittor/jittor/tree/master/python/jittor/src"
    click ExternLib "https://github.com/jittor/jittor/tree/master/python/jittor/extern"
    click PyTest "https://github.com/jittor/jittor/tree/master/python/jittor/test"
    click CTest "https://github.com/jittor/jittor/tree/master/python/jittor/src/test"
    click Documentation "https://github.com/jittor/jittor/tree/master/doc"
    click Scripts "https://github.com/jittor/jittor/tree/master/python/jittor/script"

    %% Styles
    classDef frontend fill:#add8e6,stroke:#000,stroke-width:2px;
    classDef intermediate fill:#90ee90,stroke:#000,stroke-width:2px;
    classDef jit fill:#ffb6c1,stroke:#000,stroke-width:2px;
    classDef backend fill:#ffe4b5,stroke:#000,stroke-width:2px;
    classDef testing fill:#ffcccb,stroke:#000,stroke-width:2px;
    classDef auxiliary fill:#d3d3d3,stroke:#000,stroke-width:2px;

    class FE_API,FE_Notebooks frontend;
    class Core intermediate;
    class Compiler,PyJT,Tuner jit;
    class BackendOps,ExternLib backend;
    class PyTest,CTest testing;
    class Documentation,Scripts auxiliary;
```
