How natalie-lang/natalie works
Main components
• Compiler Frontend: This part parses Ruby source code (the “src/” and parts of “lib/natalie/compiler”) and produces an intermediate representation. It manages elements like parsing, macro expansion, and method dispatch.
• C++ Code Generator / Backend: Once the IR is formed, the system translates Ruby code into C++ code (as seen in many “src/*.cpp” files). This backend then uses a system compiler (gcc/clang) to produce executables.
• Runtime & Core Libraries: The runtime is composed of many C++ implementations of Ruby objects and core libraries (found in “src/,” “include/,” and parts of “lib/”). This includes implementations for arrays, strings, classes, modules, threads, and more.
• Build and Test Infrastructure: There is a comprehensive test suite (in “spec/” and “test/”) and CI configurations (GitHub Actions, Dockerfile, etc.) that help ensure conformance and stability. The “Makefile” and “Rakefile” are used for building the project.
• REPL Interface: The “bin/natalie” binary, which enters an interactive Read-Eval-Print Loop, is an integral part of the user interface. It handles incremental compilation and immediate execution.
• Extensions & External Integrations: The “ext/” directory contains additional components (like onigmo, prism, tm, zlib) which hint at native extensions for extra functionality in the language.
Relationships and interactions
– User Input/Script: Ruby source code is input either via the REPL or via a Ruby script.
– Compiler Frontend: The input goes into the frontend (parsing, lexing – see “src/parser.cpp” and related modules) that produces an internal abstract representation of the code.
– Compiler Pipeline: This IR is then processed in multiple compilation passes (for example, the “pass1” to “pass4” in “lib/natalie/compiler/”) and transformed into either bytecode (the “bytecode” directories/tests) or directly to C++ code.
– Code Generation and Linking: The generated C++ code is compiled by the system C++ compiler (clang/gcc) to produce an executable.
– Runtime System: At runtime, the process uses the core libraries (the “src/”, “include/” directories) to handle Ruby’s object system, garbage collection, method dispatch, thread handling, etc.
– Test and CI: The tests (under “spec/” and “test/”) verify that implemented language features conform to MRI expectations. The CI workflow files ensure that every commit/build is exercised against these tests.
Architectural patterns/design principles
– Modular Design: Files are organized by functionality (compilation, runtime, core objects, extensions) which simplifies expansion and maintenance.
– Separation of Concerns: The components for user interface (REPL), language compilation, runtime, and testing are kept largely distinct.
– Layered Architecture: At the top is the user-facing REPL/CLI interface; below it sits the compiler/front-end, then the code generation/back-end; and finally, the runtime core libraries and native integrations.
– Extensibility Points: The “ext/” directory and modular layout of the compiler pipeline suggest that adding additional language features or backends is supported.
• User Interaction Layer: REPL, CLI, script invocation.
• Compilation Layer: Frontend, IR, compilation passes, code generation.
• Runtime Layer: Core object system, memory management, thread and process management.
• Auxiliary Layer: Test framework, Continuous Integration, and build scripts.
– For each component, list key files or directories as annotations (e.g., “bin/natalie” for REPL, “src/” for runtime implementation, “lib/natalie/compiler/” for compilation passes).