# quic/aimet architecture

> Architecture diagram of the GitHub repository quic/aimet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/quic/aimet
- Repository: https://github.com/quic/aimet

## Overview

##### Primary purpose and scope
- AIMET (AI Model Efficiency Toolkit) is responsible for advanced model quantization (including simulation, bias correction, adaptive rounding, and quantization-aware training) as well as model compression (using techniques such as spatial SVD and channel pruning).
- It integrates with multiple deep learning frameworks: PyTorch, TensorFlow, and ONNX.

##### Main components
- Core Quantization Engine: This contains the algorithms and modules that manage quantization techniques:
– Quantization Simulation
– Adaptive Rounding (AdaRound)
– Bias Correction and Cross-Layer Equalization
– Quantization-aware Training support (including for recurrent models)
- Compression Module: Represent components for model compression that perform spatial SVD, channel pruning, and per-layer compression strategy selection.
- Optimization/Model Improvement Logic: This includes modules such as connected graph analysis, layer output utilities, and sensitivity analysis (the project also includes visualization tools to inspect weight ranges, per-layer sensitivities, etc.).
- Underlying C/C++ and Performance-Critical Subsystems: Note the presence of ModelOptimizations (with distinct directories for DlQuantization, DlEqualization, and DlCompression) and related tests that verify the high performance of these operations.
- Python and Framework APIs: A large portion of the project (e.g., TrainingExtensions and various directories under aimet_common, aimet_onnx, aimet_torch, and aimet_tensorflow) provide user-facing APIs and both training and inference integrations.
- CI/CD, Packaging, and Build Infrastructure: Don’t forget that the repository includes extensive CI/CD scripts (GitHub workflows, Jenkins files), packaging, build scripts, and Dockerfiles. This indicates tight integration with automated testing, deployment, and release processes.
- Visualization and Evaluation Tools: These components help users assess the performance (accuracy vs. quantization trade-offs, compression ratios, etc.) and might be encapsulated as separate modules.

##### Relationships and interactions
– E.g., from raw model → Quantization Simulation and then to Adaptive Rounding and Bias Correction.
- Build/integration pipeline as a separate branch that ties into testing (unit tests, nightly tests) and deployment (packaging, Docker images).

##### Architectural patterns and technologies
- Modular design: Each technique (quantization, compression, visualization) exists as nearly independent modules that connect via well-defined APIs.
- Multi-framework support: Show that a single core engine caters to integrations with ONNX, TensorFlow, and PyTorch.
- Extensibility: API layers in Python that wrap lower-level implementations (C++ modules in ModelOptimizations) allow for both performance and easy customization.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Framework Integration Layer
    subgraph "Framework Integration Layer"
        FI1["ONNX Integration"]:::integration
        FI2["TensorFlow Integration"]:::integration
        FI3["PyTorch Integration"]:::integration
    end

    %% Core Processing (Quantization Engine and Compression Module)
    subgraph "Core Processing"
        subgraph "Core Quantization Engine"
            QE1["Quantization Simulation & QAT"]:::core
            QE2["AdaRound (ONNX)"]:::algo
            QE3["AdaRound (TensorFlow)"]:::algo
            QE4["AdaRound (PyTorch)"]:::algo
            QE5["Bias Correction"]:::algo
            QE6["Cross-Layer Equalization"]:::algo
        end
        subgraph "Compression Module"
            CM1["Compression Engine"]:::core
            CM2["Channel Pruner"]:::algo
        end
        QOUT["Optimized Model Output"]:::output
    end

    %% Optimization / Model Improvement Logic
    subgraph "Optimization/Model Improvement Logic"
        OML1["Connected Graph Analysis"]:::utility
        OML2["Layer Output Utilities"]:::utility
        OML3["Graph Searcher"]:::utility
    end

    %% Supporting Infrastructure: Underlying C/C++ Kernels
    subgraph "Performance-Critical Subsystems"
        CPS["C/C++ Kernels"]:::infra
    end

    %% Python and Framework APIs
    subgraph "Python & Framework APIs"
        API1["Common Python API"]:::api
        API2["ONNX Python API"]:::api
        API3["TensorFlow Python API"]:::api
        API4["PyTorch Python API"]:::api
    end

    %% CI/CD, Packaging & Build Infrastructure
    subgraph "CI/CD and Build Infrastructure"
        CI1["CI/CD Workflows"]:::infra
        CI2["Jenkins CI"]:::infra
        CI3["Packaging & Build Scripts"]:::infra
        CI4["CMake Build System"]:::infra
    end

    %% Visualization and Evaluation Tools
    subgraph "Visualization & Evaluation Tools"
        VIS1["Visualization Tools"]:::visual
        VIS2["Interactive Visualization Docs"]:::visual
    end

    %% Data Flow Connections
    
    %% Framework Integration to Core Processing (both Quantization and Compression)
    FI1 -->|"model input"| QE1
    FI1 -->|"model input"| CM1
    FI2 -->|"model input"| QE1
    FI2 -->|"model input"| CM1
    FI3 -->|"model input"| QE1
    FI3 -->|"model input"| CM1

    %% Within Core Quantization Engine: connections in processing sequence
    QE1 -->|"simulate"| QE2
    QE1 -->|"simulate"| QE3
    QE1 -->|"simulate"| QE4
    QE2 -->|"refine"| QE5
    QE3 -->|"refine"| QE5
    QE4 -->|"refine"| QE5
    QE5 -->|"equalize"| QE6

    %% Convergence from Quantization and Compression Modules to Output
    QE6 --> QOUT
    CM1 --> QOUT
    CM2 --> QOUT

    %% Optimized Model influences Optimization Logic and Visualization
    QOUT -->|"analyze"| OML1
    QOUT -->|"analyze"| OML2
    QOUT -->|"analyze"| OML3
    QOUT --> VIS1
    QOUT --> VIS2

    %% Underlying C/C++ Kernels support Core Processing
    CPS --- QE1
    CPS --- CM1

    %% Python & Framework APIs interface with Integration Layer and Output
    FI1 --- API2
    FI2 --- API3
    FI3 --- API4
    QOUT --- API1

    %% CI/CD and Build Infrastructure touches all components for continuous integration
    QOUT --- CI1
    QOUT --- CI2
    QOUT --- CI3
    QOUT --- CI4

    %% Optimization Logic feedback loop to Core Processing
    OML1 --- QE1
    OML2 --- QE1
    OML3 --- QE1

    %% Click Events for Component Mapping
    click FI1 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/onnx/src/python/aimet_onnx"
    click FI2 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/tensorflow/src/python/aimet_tensorflow"
    click FI3 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/torch/src/python/aimet_torch"

    click QE1 "https://github.com/quic/aimet/tree/develop/ModelOptimizations/DlQuantization"
    click QE2 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/onnx/src/python/aimet_onnx/adaround"
    click QE3 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/tensorflow/src/python/aimet_tensorflow/keras/adaround"
    click QE4 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/torch/src/python/aimet_torch/adaround"
    click QE5 "https://github.com/quic/aimet/blob/develop/TrainingExtensions/common/src/python/aimet_common/bias_correction.py"
    click QE6 "https://github.com/quic/aimet/blob/develop/TrainingExtensions/common/src/python/aimet_common/cross_layer_equalization.py"

    click CM1 "https://github.com/quic/aimet/tree/develop/ModelOptimizations/DlCompression"
    click CM2 "https://github.com/quic/aimet/blob/develop/TrainingExtensions/common/src/python/aimet_common/channel_pruner.py"

    click OML1 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/common/src/python/aimet_common/connected_graph"
    click OML2 "https://github.com/quic/aimet/blob/develop/TrainingExtensions/common/src/python/aimet_common/layer_output_utils.py"
    click OML3 "https://github.com/quic/aimet/blob/develop/TrainingExtensions/common/src/python/aimet_common/graph_searcher.py"

    click API1 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/common/src/python/aimet_common"
    click API2 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/onnx/src/python/aimet_onnx"
    click API3 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/tensorflow/src/python/aimet_tensorflow"
    click API4 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/torch/src/python/aimet_torch"

    click CI1 "https://github.com/quic/aimet/tree/develop/.github/workflows"
    click CI2 "https://github.com/quic/aimet/tree/develop/Jenkins"
    click CI3 "https://github.com/quic/aimet/tree/develop/packaging"
    click CI4 "https://github.com/quic/aimet/blob/develop/CMakeLists.txt"

    click VIS1 "https://github.com/quic/aimet/tree/develop/TrainingExtensions/torch/src/python/aimet_torch/visualization_tools"
    click VIS2 "https://github.com/quic/aimet/blob/develop/Docs/apiref/torch/interactive_visualization.rst"

    %% Styles
    classDef integration fill:#CDEBFF,stroke:#0277BD,stroke-width:2px;
    classDef core fill:#FFF2CC,stroke:#F1C232,stroke-width:2px;
    classDef algo fill:#D9EAD3,stroke:#6AA84F,stroke-width:2px;
    classDef utility fill:#FCE5CD,stroke:#E69138,stroke-width:2px;
    classDef infra fill:#F4CCCC,stroke:#CC0000,stroke-width:2px;
    classDef api fill:#EAD1DC,stroke:#B854A7,stroke-width:2px;
    classDef visual fill:#D0E0E3,stroke:#76A5AF,stroke-width:2px;
    classDef output fill:#F9CB9C,stroke:#BF9000,stroke-width:2px;
```
