How lambdaclass/cairo_native works
Main Components
• Compiler Core and Pipeline:
• Central compilation process that translates Sierra code into MLIR and then to native machine code. This pipeline is mainly implemented within the src/ directory (e.g., compiler.rs, context.rs, module.rs, runtime.rs, and more).
• Within this pipeline, highlight components such as:
⁃ Frontend: Parsing and intermediate representation management (Sierra input).
⁃ Conversion: The transformation phase that leverages MLIR and LLVM (notice dependencies, environment variables, and the integration of MLIR/LLVM as seen in the README).
⁃ Backend: Code generation and optimizations using LLVM toolchain.
• Execution Engines:
• The project provides two execution methods – Just-in-time (JIT) and Ahead-of-time (AOT) compilation. Components for these are found in src/executor (both aot.rs and jit.rs).
• Caching modules (src/cache directory) that store compiled libraries or AOT caches.
• Command-Line Tools:
• Several binaries (found in src/bin) act as user-facing interfaces: cairo-native-compile, cairo-native-dump, cairo-native-run, cairo-native-test, cairo-native-stress, as well as scarb-native variants.
• How these tools interact with the core library, essentially acting as wrappers to invoke the underlying compiler and execution engines.
• Supporting Libraries and Utilities:
• Under src/libfuncs and src/types, a number of library functions and types are implemented that likely support built-in operations for Cairo programs (from arrays and booleans to cryptographic functions).
• Auxiliary utilities (in src/utils) for memory tracing, safe runner functionality, and debugging.
• Testing & Benchmarks:
• The tests/ and benches/ directories showcase thorough testing including integration and benchmark tests (including stress tests with cairo-native-stress).
• A note that external benchmarking scripts, as well as continuous integration (GitHub workflows), are integrated with the project.
• Documentation and Examples:
• Directories such as docs/, examples/, and programs/ illustrate usage examples, walkthroughs, and test programs that serve as a guideline for end users.
Relationships and Interactions
• User Interaction: Command-line tools (frontend wrappers) invoke the compiler library.
• Data Flow:
⁃ Input: Cairo (Sierra) source code provided by users or test cases.
⁃ Processing: The compiler core processes the input, leveraging internal modules for conversion and optimization, and then passes through MLIR/LLVM for code generation.
⁃ Output: Machine code (or shared libraries), as produced by the backend, and test results when running programs.
• How the environment’s configuration (LLVM and MLIR installation, as set by env.sh or environment variables) is a prerequisite that feeds into the pipeline.
Architectural Patterns and Notable Design Principles
• Modular design: Clear separation between core libraries (in src), CLI tools (in src/bin), and tests or examples.
• Layered architecture:
1. Frontend CLI layer,
2. Compiler core layer with submodules for IR handling, caching, and execution,
3. Backend integration with external toolchains (MLIR, LLVM).
• Emphasis on extensibility: Directories like libfuncs and types imply a pluggable system for new operations, which can easily be integrated without affecting the core pipeline.
• Use of caching strategies (AOT/JIT modules found in src/cache and in executor subdirectories) to improve performance.
Technologies and External Dependencies
• MLIR, LLVM, Clang toolchain are critical for machine code generation.
• Rust is the primary programming language, documented by the Cargo files and rust-toolchain.
• Additional system dependencies from the README (e.g., dependency installation instructions, environment scripts).