# apple/coremltools architecture

> Architecture diagram of the GitHub repository apple/coremltools, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apple/coremltools
- Repository: https://github.com/apple/coremltools

## Overview

a. Input Frontends (Converters): These are the modules that ingest models from various external machine learning frameworks. In the file tree you see dedicated subdirectories under “coremltools/converters” for TensorFlow (both TF1 and TF2), PyTorch, scikit‑learn, XGBoost, and even LibSVM. They serve as adapters to translate differing model formats into a common representation.
b. Intermediate Representation – MIL: Once the input is received, models are converted into an internal “MIL” (Model Intermediate Language) representation. This is evidenced by directories such as “coremltools/converters/mil” and related “mil” passes (e.g., optimization passes under coremltools/converters/mil/backend and passes subdirectories). The MIL stage is the heart of the conversion pipeline.
c. Optimization and Transformation Modules: The repository has “coremltools/optimize” (with separate subdirectories for coreml, torch, etc.) that run optimization passes such as quantization, palettization, and pruning. These layers work on the MIL IR or on components of the model to boost performance.
d. Core ML Model Specification Builder: After conversion and optimization, the pipeline constructs the final Core ML model spec. This is handled by modules in “coremltools/models” (and “mlmodel” directories) which generate or parse the .mlmodel format.
e. Python–C++ / Objective‑C++ Bridging: The “coremlpython” folder contains native code (objective‑C++ and C++ headers/methods) that link Python components with the underlying executables on macOS. This acts as a bridge to support on-device verification (predictions) and integration with the Core ML APIs.
f. Auxiliary Systems: There are supportive directories for “tests”, “docs”, “examples”, and “CI/CD” (like .github, .gitlab-ci.yml, etc.) that ensure reliability and documentation but are not part of the main runtime pipeline.

– Additionally, include a side channel from the Core ML Model Builder into the “coremlpython” bridging module, which then connects to Core ML validation and prediction (often utilized on macOS).
– Finally, note that build tools (CMake, setup.py, Makefiles) in separate directories facilitate integration and cross-platform builds.

##### Architectural patterns and design principles
– Modular Design: Each functionality (input conversion, intermediate representation, optimization, model generation, bridging, testing) is isolated in its own directory and component.
– Abstraction: The conversion frontends abstract away differences among third‑party ML frameworks, converting them into a unified MIL.
– Extendibility: The use of subdirectories for different frameworks (e.g., “tensorflow”, “torch”, “sklearn”, “xgboost”) illustrates that the architecture is built to be extensible; additional converters can be attached at the frontend.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Global Entities
    External["External ML Frameworks<br/>(TensorFlow, PyTorch, scikit‑learn, XGBoost, LibSVM)"]:::external

    %% Input Frontends Subgraph
    subgraph "Input Frontends (Converters)"
        LS["LibSVM Converter"]:::converter
        SK["scikit‑learn Converter"]:::converter
        XG["XGBoost Converter"]:::converter
        TF["TensorFlow Converter"]:::converter
        PT["PyTorch Converter"]:::converter
    end

    %% MIL Intermediate Representation
    MIL_IR["MIL Intermediate Representation"]:::mil

    %% Optimization Modules (sequential representation)
    CMLOpt["Core ML Optimization"]:::optimization
    TorchOpt["Torch Optimization"]:::optimization

    %% Core ML Model Specification Builder
    ModelSpec["Core ML Model Specification Builder"]:::modelspec

    %% Python–C++ / Objective‑C++ Bridging Module
    PyBridge["Python–C++ Bridge"]:::bridge

    %% macOS Core ML Validation/Prediction
    Validation["macOS Core ML Validation/Prediction"]:::validation

    %% Auxiliary Systems: Testing and CI/CD
    Testing["Testing and CI/CD"]:::auxiliary

    %% Build & Packaging Tools
    Build["Build & Packaging"]:::build

    %% Connections: External frameworks to Input Frontends
    External -->|"feeds"| LS
    External -->|"feeds"| SK
    External -->|"feeds"| XG
    External -->|"feeds"| TF
    External -->|"feeds"| PT

    %% Connections: Input Frontends to MIL Conversion (each converter converts model)
    LS -->|"modelConversion"| MIL_IR
    SK -->|"modelConversion"| MIL_IR
    XG -->|"modelConversion"| MIL_IR
    TF -->|"modelConversion"| MIL_IR
    PT -->|"modelConversion"| MIL_IR

    %% Connections: MIL to Optimization Modules (sequential optimization passes)
    MIL_IR -->|"IRoptim"| CMLOpt
    CMLOpt -->|"IRoptim"| TorchOpt

    %% Connection: Optimization Modules to Core ML Model Specification
    TorchOpt -->|"codeGen"| ModelSpec

    %% Side Channel: Core ML Model Specification to Python–C++ Bridge
    ModelSpec -->|"runtimeBinding"| PyBridge

    %% Connection: Python–C++ Bridge to macOS Validation/Prediction
    PyBridge -->|"executes"| Validation

    %% Feedback Loop: Model Specification to Testing/CI-CD (dotted arrow)
    ModelSpec ---|"feedback"| Testing

    %% Build Tools: Testing to Build & Packaging (dotted arrow)
    Testing ---|"buildTrigger"| Build

    %% Styles
    classDef external fill:#8EC0E4,stroke:#333,stroke-width:2px;
    classDef converter fill:#F9E79F,stroke:#333,stroke-width:2px;
    classDef mil fill:#AED6F1,stroke:#333,stroke-width:2px;
    classDef optimization fill:#F5B7B1,stroke:#333,stroke-width:2px;
    classDef modelspec fill:#82E0AA,stroke:#333,stroke-width:2px;
    classDef bridge fill:#D7BDE2,stroke:#333,stroke-width:2px,stroke-dasharray: 4 2;
    classDef validation fill:#FDEBD0,stroke:#333,stroke-width:2px,stroke-dasharray: 2 2;
    classDef auxiliary fill:#FAD7A0,stroke:#333,stroke-width:2px,stroke-dasharray: 5 5;
    classDef build fill:#E6B0AA,stroke:#333,stroke-width:2px,stroke-dasharray: 3 3;

    %% Click Events for Component Mappings
    click LS "https://github.com/apple/coremltools/tree/main/coremltools/converters/libsvm"
    click SK "https://github.com/apple/coremltools/tree/main/coremltools/converters/sklearn"
    click XG "https://github.com/apple/coremltools/tree/main/coremltools/converters/xgboost"
    click TF "https://github.com/apple/coremltools/tree/main/coremltools/converters/mil/frontend/tensorflow"
    click PT "https://github.com/apple/coremltools/tree/main/coremltools/converters/mil/frontend/torch"
    click MIL_IR "https://github.com/apple/coremltools/tree/main/coremltools/converters/mil"
    click CMLOpt "https://github.com/apple/coremltools/tree/main/coremltools/optimize/coreml"
    click TorchOpt "https://github.com/apple/coremltools/tree/main/coremltools/optimize/torch"
    click ModelSpec "https://github.com/apple/coremltools/tree/main/mlmodel"
    click PyBridge "https://github.com/apple/coremltools/tree/main/coremlpython"
    click Build "https://github.com/apple/coremltools/blob/main/setup.py"
```
