# google/gematria architecture

> Architecture diagram of the GitHub repository google/gematria, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/google/gematria
- Repository: https://github.com/google/gematria

## Overview

##### Project type and purpose
– Gematria is an open-source framework for machine learning on machine code. It provides:
- Data ingestion and feature extraction for basic blocks of machine code
- Two ML models (GRANITE GNN and hierarchical LSTM) for predicting basic-block throughput
- Training and inference pipelines in Python and C++ (via pybind11)
- Optional integration into LLVM as an external project (the `llvm-cm` tool)

##### Key directories and architectural patterns
Top‐level build & config
- WORKSPACE, BUILD.bazel, CMakeLists.txt, Dockerfile, Bazel/CMake build scripts
- `.github/workflows` for CI

gematria/… - modular C++ + Python code
- basic_block – core basic-block abstraction & protos + Python wrappers
- datasets – importers, converters, TFRecord pipelines (C++ + Python)
- granite – graph builder & GRANITE model (C++ + Python)
- sequence – Ithemal LSTM model (Python only)
- model – shared model utilities (inference, training scaffolding)
- io – file‐system and TFRecord utilities (Python)
- llvm – parsers/canonicalizers/disassemblers leveraging LLVM
- tflite – custom ops for TensorFlow Lite
- proto – .proto definitions for annotations, throughput, blocks
- utils – general C++/Python helpers (flags, timer, string)
- testing – shared test harness code and lit integration

llvm_cm/… – standalone C++ tool (`llvm-cm`) that uses gematria libraries inside LLVM

##### System components & data flow
– Data Ingestion & Preprocessing (gematria/datasets + io)
- Reads binaries or IR → extracts basic-blocks → serializes to TFRecord

– Feature Extraction & Representation
- basic_block module defines canonical block representation via protos
- llvm module parses/disassembles code and normalizes instructions

– Model Training (Python driver in `model/python` + C++ heavy lifting)
- GRANITE: graph‐builder (C++), GNN training in TensorFlow 2.x
- Sequence model: hierarchical LSTM implemented in Python/TensorFlow

– Model Artifacts
- Saved TensorFlow checkpoints,/or exported TFLite models (`tflite` module)

– Inference APIs
- Python inference wrapper (pybind11) calling into C++ graph builder + TF runtime
- C++ inference API (gematria/granite-inference-api.md)

– Optional integration into LLVM as an external project tool (`llvm-cm`)

##### Architectural patterns & design principles
– Layered modular design: separate data, representation, model, utils, and CLI/tools
– Cross‐language bindings: pybind11 exposes performance‐critical C++ to Python drivers
– Protocol buffers for stable on‐disk interchange between stages
– Build‐system abstraction: Bazel for full monorepo, CMake for LLVM subproject

##### Putting it all together
[Binary/IR sources] → (llvm parser, basic_block) → [Proto TFRecords] → (training driver) → [TF Checkpoints/TFLite] → (inference API) → [Predictions]
Underneath, annotate build/integration layers: Bazel and pybind11 connecting C++ ↔ Python, and optional LLVM external project path (`llvm-cm`).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    Source["Binary/IR Sources"]:::ext

    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing"
        Preprocess["TFRecord Pipeline\n(gematria/datasets)"]:::cpp
        IO["I/O Utilities\n(gematria/io)"]:::python
        ProtoDefs["Protocol Buffers\n(gematria/proto)"]:::cpp
        BasicBlock["Basic-Block Protocs\n(gematria/basic_block)"]:::cpp
        LLVMParser["LLVM Parser\n(gematria/llvm)"]:::cpp
    end

    %% Intermediate Data Format
    TFRec["Proto TFRecords"]:::ext

    %% Model Training
    subgraph "Model Training"
        GraphBuilder["GRANITE GNN Graph Builder\n(gematria/granite)"]:::cpp
        PythonGNN["GRANITE Python Bindings\n(gematria/granite/python)"]:::python
        SequenceModel["Hierarchical LSTM Model\n(gematria/sequence/python)"]:::python
        TrainingDriver["Training Driver\n(gematria/model/python)"]:::python
        Utils["Model Utilities\n(gematria/model)"]:::python
        TFLiteOps["TFLite Custom Ops\n(gematria/tflite)"]:::cpp
    end

    %% Model Artifacts
    Artifacts["Model Artifacts\n(TF Checkpoints / TFLite)"]:::ext

    %% Inference
    subgraph "Inference APIs"
        PyInfer["Python Inference\n(pybind11)"]:::python
        CPPInfer["C++ Inference API\n(g3doc/granite-inference-api.md)"]:::cpp
    end

    %% Optional LLVM Integration
    subgraph "LLVM Integration Tool"
        LLVMCm["llvm-cm Tool\n(llvm_cm/tools/llvm-cm)"]:::cpp
    end

    %% Build & CI
    subgraph "Build & CI"
        Bazel["Bazel/CMake Build"]:::ext
        CI["CI Workflows\n(.github/workflows)"]:::ext
    end

    %% Data Flow
    Source -->|input| LLVMParser
    LLVMParser -->|normalizes| BasicBlock
    BasicBlock -->|feeds| Preprocess
    IO -->|handles| Preprocess
    ProtoDefs -->|defines| Preprocess
    Preprocess -->|outputs| TFRec

    TFRec -->|trains| GraphBuilder
    TFRec -->|trains| SequenceModel
    GraphBuilder -->|uses| PythonGNN
    GraphBuilder -->|uses| Utils
    SequenceModel -->|uses| Utils
    GraphBuilder -->|builds| Artifacts
    SequenceModel -->|exports| Artifacts
    TFLiteOps -->|generates| Artifacts

    Artifacts -->|loads| PyInfer
    Artifacts -->|loads| CPPInfer
    PyInfer -->|predicts| Prediction["Predictions"]:::ext
    CPPInfer -->|predicts| Prediction

    Bazel -->|builds| GraphBuilder
    Bazel -->|builds| LLVMParser
    Bazel -->|builds| LLVMCm
    CI -->|tests| Preprocess
    CI -->|tests| GraphBuilder
    CI -->|tests| PyInfer

    LLVMCm -->|integrates| LLVMParser
    LLVMCm -->|links| GraphBuilder

    %% Click Events
    click Preprocess "https://github.com/google/gematria/tree/main/gematria/datasets/"
    click IO "https://github.com/google/gematria/tree/main/gematria/io/python/"
    click ProtoDefs "https://github.com/google/gematria/tree/main/gematria/proto/"
    click BasicBlock "https://github.com/google/gematria/tree/main/gematria/basic_block/"
    click LLVMParser "https://github.com/google/gematria/tree/main/gematria/llvm/"
    click GraphBuilder "https://github.com/google/gematria/blob/main/gematria/granite/graph_builder.cc"
    click PythonGNN "https://github.com/google/gematria/tree/main/gematria/granite/python/"
    click SequenceModel "https://github.com/google/gematria/tree/main/gematria/sequence/python/"
    click TrainingDriver "https://github.com/google/gematria/tree/main/gematria/model/python/"
    click TFLiteOps "https://github.com/google/gematria/blob/main/gematria/tflite/unsorted_segment_sum_op.cc"
    click Utils "https://github.com/google/gematria/tree/main/gematria/utils/"
    click CPPInfer "https://github.com/google/gematria/blob/main/g3doc/granite-inference-api.md"
    click PyInfer "https://github.com/google/gematria/blob/main/gematria/model/python/inference.py"
    click LLVMCm "https://github.com/google/gematria/blob/main/llvm_cm/tools/llvm-cm/llvm-cm.cpp"

    %% Styles
    classDef cpp fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef python fill:#ffe0b2,stroke:#333,stroke-width:1px
    classDef ext fill:#d4edda,stroke:#333,stroke-width:1px
```
