# francescodaghero/treeco architecture

> Architecture diagram of the GitHub repository francescodaghero/treeco, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/francescodaghero/treeco
- Repository: https://github.com/francescodaghero/treeco

## Overview

##### Project Type and Purpose
- TreeCo is a compiler tool built in Python, based on the xDSL/MLIR framework, specifically targeting tree-based ensemble models (classifiers/regressors) for optimized inference.
- It ingests ONNX representations of tree ensembles, generates MLIR in three custom dialects (Crown, Trunk, TreeCo), applies a series of optimization and lowering passes, and emits code in MLIR, LLVM IR, or C.

##### High-Level File Structure Analysis
- src/treeco/__main__.py & src/treeco/compiler.py: CLI entry point and compiler orchestration
- src/treeco/frontend:
- parser.py – reads ONNX files via onnx-ml operations
- ir_gen.py – constructs the initial ONNX-ML IR
- src/treeco/dialects: definitions for Crown, Trunk, TreeCo, ONNX-ML and auxiliary dialects
- src/treeco/lowering: pipeline layers converting ONNX-ML → Crown → Trunk → Arith/ML_Program → Bufferized → LLVM IR
- src/treeco/transforms: optimization passes (quantization, pruning, padding, voting, etc.) on Crown/Trunk IR
- src/treeco/model: Python representation of ensembles for in-memory modifications and standalone slow predict
- src/treeco/targets: backends for code generation
- llvm/… – standalone, library, transforms to emit LLVM IR
- cpp/… – EmitC path to C++ code
- src/treeco/utils: helper routines (xdsl ↔ numpy conversions, general utilities)
- tests: end-to-end cases (e.g. test_llvm_target.py)

##### Additional Architectural Insights from README
- Three dialects separate concerns:
- Crown – high-level ensemble structure transformations
- Trunk – control-flow and tree traversal strategies
- TreeCo – custom data types (node, tree, ensemble)
- A three-stage lowering pipeline invoked by crown_transform, trunk_transform, root_transform, culminating in target_transform_and_dump.
- External dependencies: xdsl/MLIR toolchain (mlir-opt, mlir-translate), onnx, skl2onnx for front-end.

##### Core Components
a. User Interface / CLI Layer
- __main__.py launches parse_ensemble() → generate_ir(), then crown_transform(), trunk_transform(), root_transform(), target_transform_and_dump()
b. Frontend
- ONNX Parser (uses onnx, skl2onnx) → initial ONNX-ML IR in xDSL context
c. Dialect Registry
- Modules defining Crown, Trunk, TreeCo, extended dialect ops/types
d. Mid-end (Transform Passes)
- Crown optimization passes (crown_pad_to_perfect, crown_prune, etc.)
- Trunk transforms (tree peeling, unrolling, vectorization)
e. Lowering / Back-end
- Bufferization, mlir_opt pipeline via xdsl_opt
- EmitC conversion (emitc dialect) or LLVM IR generation
f. Model Layer
- Python Ensemble class for offline manipulations (in model/ensemble.py)
g. Targets
- llvm library/standalone codegen_main
- cpp standalone code via EmitC
h. External Services/Tools
- xdsl/MLIR command-line tools, clang (for final compilation), onnx library

##### Relationships and Data Flow
- Frontend → Dialect Registry: IR nodes tagged with custom dialect ops/types
- IR flows into Transform Pass Manager → Crown Passes → Trunk Passes → Root Pass

##### Architectural Patterns & Design Principles
- Compiler pipeline pattern: clear stages (frontend, mid-end, back-end)
- Plugin-style dialect registration for extensibility
- Separation of concerns: IR definitions (dialects) decoupled from transforms and codegen
- Python-only stack for rapid iteration, with hooks into external MLIR/clang tools

##### Technologies and Frameworks to Call Out
- Python 3.10+ packaging (pyproject.toml, requirements.txt)
- xDSL framework for IR building and pass management
- MLIR tools (mlir-opt, mlir-translate) for final lowering
- onnx / skl2onnx for front-end parsing
- clang for compiling emitted LLVM IR or C code

##### Essential Elements to Capture
- Precise component names and their code locations
- Data representations at each stage (ONNX-ML IR, Crown IR, Trunk IR, Bufferized IR, LLVM IR/C)
- External dependencies invoked during each stage
- Optional paths (EmitC vs LLVM backend)

## Diagram (Mermaid)

```mermaid
flowchart TB
  %% CLI Layer
  subgraph "CLI Layer"
    direction TB
    Main["__main__.py"]:::cli
    Compiler["compiler.py"]:::cli
  end

  Main -->|"invokes"| Compiler

  %% Frontend Layer
  subgraph "Frontend Layer"
    direction TB
    Parser["parser.py"]:::frontend
    IRGen["ir_gen.py"]:::frontend
  end

  Compiler -->|"parse_ensemble() / generate_ir()"| Parser
  Parser -->|"initial ONNX-ML IR"| IRGen

  %% Dialect Registry
  subgraph "Dialect Registry"
    direction TB
    CrownDialect["crown.py"]:::dialect
    TrunkDialect["trunk.py"]:::dialect
    TreeCoDialect["treeco.py"]:::dialect
    ONNXMLDialect["onnxml.py"]:::dialect
    EmitCDialect["emitc.py"]:::dialect
    subgraph "Extended Dialects"
      direction TB
      Bufferization["bufferization.py"]:::dialect
      MLProgram["ml_program.py"]:::dialect
      TensorDialect["tensor.py"]:::dialect
    end
  end

  IRGen -->|"register dialects"| CrownDialect
  IRGen --> TrunkDialect
  IRGen --> TreeCoDialect
  IRGen --> ONNXMLDialect
  IRGen --> EmitCDialect
  IRGen --> Bufferization
  IRGen --> MLProgram
  IRGen --> TensorDialect

  %% Mid-end Transform Passes
  subgraph "Mid-end Transform Passes"
    direction TB
    CPM["Transform Pass Manager"]:::transform
    CrownPad["crown_pad_to_perfect.py"]:::transform
    CrownPrune["crown_prune.py"]:::transform
    CrownQuant["crown_quantize.py"]:::transform
    CrownVote["crown_voting.py"]:::transform
    FuncLegalize["func_legalize.py"]:::transform
    MemrefMerge["memref_merge_subview.py"]:::transform
    MemrefQuant["memref_quantize_global_index.py"]:::transform
    MLGlobalQuant["ml_global_quantize_index.py"]:::transform
    PrepLLVMLower["prepare_llvm_lowering.py"]:::transform
    TrunkPad["trunk_pad_to_min_depth.py"]:::transform
  end

  CrownDialect --> CPM
  TrunkDialect --> CPM
  CPM --> CrownPad
  CPM --> CrownPrune
  CPM --> CrownQuant
  CPM --> CrownVote
  CPM --> FuncLegalize
  CPM --> MemrefMerge
  CPM --> MemrefQuant
  CPM --> MLGlobalQuant
  CPM --> PrepLLVMLower
  CPM --> TrunkPad

  %% Model Layer
  subgraph "Model Layer"
    direction TB
    Ensemble["ensemble.py"]:::model
    NodeModel["node.py"]:::model
  end

  CPM <-->|"optional modify IR"| Ensemble
  Ensemble --> NodeModel
  NodeModel --> CPM

  %% Lowering / Back-end Pipeline
  subgraph "Lowering / Back-end"
    direction TB
    ConvONNXtoCrown["convert_onnxml_to_crown.py"]:::lowering
    ConvCrownToTrunk["convert_crown_to_trunk.py"]:::lowering
    ConvMLProgToMemref["convert_ml_program_to_memref.py"]:::lowering
    ConvSCFtoCF["convert_scf_to_cf.py"]:::lowering
    Bufferize["bufferize.py"]:::lowering
    LowerTrunk["lower_trunk.py"]:::lowering
    LowerTreeCo["lower_treeco.py"]:::lowering
    MLIROpt["mlir_opt.py"]:::lowering
    subgraph "EmitC Lowering"
      direction TB
      ConvArithEmitC["convert_arith_to_emitc.py"]:::lowering
      ConvMemrefEmitC["convert_memref_to_emitc.py"]:::lowering
      ConvPrintfEmitC["convert_printf_to_emitc.py"]:::lowering
      ConvSCFEmitC["convert_scf_to_emitc.py"]:::lowering
    end
  end

  TrunkPad --> ConvONNXtoCrown
  ConvONNXtoCrown --> ConvCrownToTrunk
  ConvCrownToTrunk --> ConvMLProgToMemref
  ConvMLProgToMemref --> ConvSCFtoCF
  ConvSCFtoCF --> Bufferize
  Bufferize --> LowerTrunk
  LowerTrunk --> LowerTreeCo
  LowerTreeCo --> MLIROpt
  MLIROpt --> ConvArithEmitC
  MLIROpt --> ConvMemrefEmitC
  MLIROpt --> ConvPrintfEmitC
  MLIROpt --> ConvSCFEmitC

  %% Targets
  subgraph "Targets"
    direction TB
    CodegenMain["codegen_main.py"]:::target
    CPPStandalone["standalone.py"]:::target
    CPPTransforms["transforms.py"]:::target
    LLVMLib["library.py"]:::target
    LLVMStandalone["standalone.py"]:::target
    LLVMTransforms["transforms.py"]:::target
  end

  MLIROpt --> CodegenMain
  CodegenMain --> CPPStandalone
  CodegenMain --> CPPTransforms
  CodegenMain --> LLVMLib
  CodegenMain --> LLVMStandalone
  CodegenMain --> LLVMTransforms

  %% Outputs
  subgraph "Outputs"
    direction TB
    MLIRFile[".mlir/.ll files"]:::ext
    CFile[".c files"]:::ext
  end

  LLVMStandalone --> MLIRFile
  CPPStandalone --> CFile

  %% External Tools
  subgraph "External Tools"
    direction TB
    XDSLOPT["xdsl-opt / mlir-opt"]:::ext
    MLIRTrans["mlir-translate"]:::ext
    Clang["clang"]:::ext
    ONNXLib["onnx / skl2onnx"]:::ext
  end

  Parser --> ONNXLib
  MLIROpt --> XDSLOPT
  MLIROpt --> MLIRTrans
  LLVMStandalone --> Clang

  %% Click Events
  click Main "https://github.com/francescodaghero/treeco/blob/main/src/treeco/__main__.py"
  click Compiler "https://github.com/francescodaghero/treeco/blob/main/src/treeco/compiler.py"
  click Parser "https://github.com/francescodaghero/treeco/blob/main/src/treeco/frontend/parser.py"
  click IRGen "https://github.com/francescodaghero/treeco/blob/main/src/treeco/frontend/ir_gen.py"
  click CrownDialect "https://github.com/francescodaghero/treeco/blob/main/src/treeco/dialects/crown.py"
  click TrunkDialect "https://github.com/francescodaghero/treeco/blob/main/src/treeco/dialects/trunk.py"
  click TreeCoDialect "https://github.com/francescodaghero/treeco/blob/main/src/treeco/dialects/treeco.py"
  click ONNXMLDialect "https://github.com/francescodaghero/treeco/blob/main/src/treeco/dialects/onnxml.py"
  click EmitCDialect "https://github.com/francescodaghero/treeco/blob/main/src/treeco/dialects/emitc.py"
  click Bufferization "https://github.com/francescodaghero/treeco/blob/main/src/treeco/dialects/extended/bufferization.py"
  click MLProgram "https://github.com/francescodaghero/treeco/blob/main/src/treeco/dialects/extended/ml_program.py"
  click TensorDialect "https://github.com/francescodaghero/treeco/blob/main/src/treeco/dialects/extended/tensor.py"
  click CrownPad "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/crown_pad_to_perfect.py"
  click CrownPrune "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/crown_prune.py"
  click CrownQuant "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/crown_quantize.py"
  click CrownVote "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/crown_voting.py"
  click FuncLegalize "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/func_legalize.py"
  click MemrefMerge "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/memref_merge_subview.py"
  click MemrefQuant "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/memref_quantize_global_index.py"
  click MLGlobalQuant "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/ml_global_quantize_index.py"
  click PrepLLVMLower "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/prepare_llvm_lowering.py"
  click TrunkPad "https://github.com/francescodaghero/treeco/blob/main/src/treeco/transforms/trunk_pad_to_min_depth.py"
  click ConvONNXtoCrown "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/convert_onnxml_to_crown.py"
  click ConvCrownToTrunk "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/convert_crown_to_trunk.py"
  click ConvMLProgToMemref "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/convert_ml_program_to_memref.py"
  click ConvSCFtoCF "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/convert_scf_to_cf.py"
  click Bufferize "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/bufferize.py"
  click LowerTrunk "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/lower_trunk.py"
  click LowerTreeCo "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/lower_treeco.py"
  click MLIROpt "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/mlir_opt.py"
  click ConvArithEmitC "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/emitc/convert_arith_to_emitc.py"
  click ConvMemrefEmitC "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/emitc/convert_memref_to_emitc.py"
  click ConvPrintfEmitC "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/emitc/convert_printf_to_emitc.py"
  click ConvSCFEmitC "https://github.com/francescodaghero/treeco/blob/main/src/treeco/lowering/emitc/convert_scf_to_emitc.py"
  click Ensemble "https://github.com/francescodaghero/treeco/blob/main/src/treeco/model/ensemble.py"
  click NodeModel "https://github.com/francescodaghero/treeco/blob/main/src/treeco/model/node.py"
  click CodegenMain "https://github.com/francescodaghero/treeco/blob/main/src/treeco/targets/codegen_main.py"
  click CPPStandalone "https://github.com/francescodaghero/treeco/blob/main/src/treeco/targets/cpp/standalone.py"
  click CPPTransforms "https://github.com/francescodaghero/treeco/blob/main/src/treeco/targets/cpp/transforms.py"
  click LLVMLib "https://github.com/francescodaghero/treeco/blob/main/src/treeco/targets/llvm/library.py"
  click LLVMStandalone "https://github.com/francescodaghero/treeco/blob/main/src/treeco/targets/llvm/standalone.py"
  click LLVMTransforms "https://github.com/francescodaghero/treeco/blob/main/src/treeco/targets/llvm/transforms.py"
  click XDSLOPT "https://github.com/francescodaghero/treeco/blob/main/src/treeco/utils/xdsl_utils.py"
  click ONNXLib "https://github.com/francescodaghero/treeco/blob/main/src/treeco/utils/utils.py"

  %% Styles
  classDef cli fill:#D0E6FA,stroke:#0A4471
  classDef frontend fill:#E2F7E1,stroke:#2C662D
  classDef dialect fill:#FEF5C3,stroke:#8D6E00
  classDef transform fill:#FDE0DD,stroke:#A8185B
  classDef lowering fill:#FFE8CC,stroke:#B35E00
  classDef model fill:#F5D0D8,stroke:#8B0000
  classDef target fill:#E8D0F5,stroke:#5D147E
  classDef ext fill:#EEEEEE,stroke:#666666
```
