How francescodaghero/treeco works
Project Type and Purpose
• TreeCo is a compiler tool built in Python, based on the xDSL/MLIR framework, specifically targeting tree-based ensemble models (classifiers/regressors) for optimized inference.
• It ingests ONNX representations of tree ensembles, generates MLIR in three custom dialects (Crown, Trunk, TreeCo), applies a series of optimization and lowering passes, and emits code in MLIR, LLVM IR, or C.
High-Level File Structure Analysis
• src/treeco/__main__.py & src/treeco/compiler.py: CLI entry point and compiler orchestration
• src/treeco/frontend:
• parser.py – reads ONNX files via onnx-ml operations
• ir_gen.py – constructs the initial ONNX-ML IR
• src/treeco/dialects: definitions for Crown, Trunk, TreeCo, ONNX-ML and auxiliary dialects
• src/treeco/lowering: pipeline layers converting ONNX-ML → Crown → Trunk → Arith/ML_Program → Bufferized → LLVM IR
• src/treeco/transforms: optimization passes (quantization, pruning, padding, voting, etc.) on Crown/Trunk IR
• src/treeco/model: Python representation of ensembles for in-memory modifications and standalone slow predict
• src/treeco/targets: backends for code generation
• llvm/… – standalone, library, transforms to emit LLVM IR
• cpp/… – EmitC path to C++ code
• src/treeco/utils: helper routines (xdsl ↔ numpy conversions, general utilities)
• tests: end-to-end cases (e.g. test_llvm_target.py)
Additional Architectural Insights from README
• Three dialects separate concerns:
• Crown – high-level ensemble structure transformations
• Trunk – control-flow and tree traversal strategies
• TreeCo – custom data types (node, tree, ensemble)
• A three-stage lowering pipeline invoked by crown_transform, trunk_transform, root_transform, culminating in target_transform_and_dump.
• External dependencies: xdsl/MLIR toolchain (mlir-opt, mlir-translate), onnx, skl2onnx for front-end.
Core Components
a. User Interface / CLI Layer
• __main__.py launches parse_ensemble() → generate_ir(), then crown_transform(), trunk_transform(), root_transform(), target_transform_and_dump()
b. Frontend
• ONNX Parser (uses onnx, skl2onnx) → initial ONNX-ML IR in xDSL context
c. Dialect Registry
• Modules defining Crown, Trunk, TreeCo, extended dialect ops/types
d. Mid-end (Transform Passes)
• Crown optimization passes (crown_pad_to_perfect, crown_prune, etc.)
• Trunk transforms (tree peeling, unrolling, vectorization)
e. Lowering / Back-end
• Bufferization, mlir_opt pipeline via xdsl_opt
• EmitC conversion (emitc dialect) or LLVM IR generation
f. Model Layer
• Python Ensemble class for offline manipulations (in model/ensemble.py)
g. Targets
• llvm library/standalone codegen_main
• cpp standalone code via EmitC
h. External Services/Tools
• xdsl/MLIR command-line tools, clang (for final compilation), onnx library
Relationships and Data Flow
• Frontend → Dialect Registry: IR nodes tagged with custom dialect ops/types
• IR flows into Transform Pass Manager → Crown Passes → Trunk Passes → Root Pass
Architectural Patterns & Design Principles
• Compiler pipeline pattern: clear stages (frontend, mid-end, back-end)
• Plugin-style dialect registration for extensibility
• Separation of concerns: IR definitions (dialects) decoupled from transforms and codegen
• Python-only stack for rapid iteration, with hooks into external MLIR/clang tools
Technologies and Frameworks to Call Out
• Python 3.10+ packaging (pyproject.toml, requirements.txt)
• xDSL framework for IR building and pass management
• MLIR tools (mlir-opt, mlir-translate) for final lowering
• onnx / skl2onnx for front-end parsing
• clang for compiling emitted LLVM IR or C code
Essential Elements to Capture
• Precise component names and their code locations
• Data representations at each stage (ONNX-ML IR, Crown IR, Trunk IR, Bufferized IR, LLVM IR/C)
• External dependencies invoked during each stage
• Optional paths (EmitC vs LLVM backend)