How yubrot/llrl works
Based on our analysis of the complete file tree and the README, the project is an experimental Lisp-like programming language called “llrl” that emphasizes creating a self-hosting compiler. There are two main compiler implementations provided:
• llrl0 – a Rust-based compiler that implements the language with a full compilation pipeline (parsing, semantic analysis, lowering, code generation, and driver) and supports multiple backends (LLVM, chibi, and native).
• llrl1 – a self-hosting version of the compiler written using llrl itself and ported to the LLVM-C API.
In addition, there are supporting libraries and tools:
• llvm0 and llvm1 – libraries/wrappers around LLVM functionality and tests.
• rt – the runtime effort that ties into glibc/Boehm GC and provides low-level runtime services.
• std – the standard library for llrl that provides built-in types, macros, and functionality.
• xten0, xten1, and xtencg – components for the x86_64 code generation, including a custom assembler and JIT linker used (for example) by the chibi backend.
• testcases and examples – which show various usage scenarios and serve as regression or demonstration tests.
Overall Structure and Components
a. Frontends and Parsing:
– Represent the S-expression lexer and parser found in both llrl0 and llrl1.
– Include AST construction, as shown by all the “ast.rs”, “parser”, “syntax”, and “sexp” modules.
b. Semantic Analysis:
c. Lowering and Transformation:
– Illustrate the “lowering” modules that transform high-level AST representations (with macros and S-expressions) into an intermediate representation (IR).
d. Code Generation and Backends:
– This is a critical piece. Separate the code generation pipeline into:
• LLVM Backend – directories under backend/llvm in both llrl0 and llrl1.
• Chibi/Natively targeted Backend – shown under backend/chibi and backend/native along with calls to the xten components.
e. Runtime and Standard Library:
– Include the “rt” module, the “std” directory (which contains standard library definitions and bootstrapping code), and mention the prerequisites (glibc, Boehm GC).
f. Supporting Libraries and Tools:
– Represent llvm0/llvm1 as shared LLVM utility libraries.
– Represent xten0, xten1, and xtencg as separate modules providing JIT/linking and assembly functionality.
– Show “testcases” and “examples” as external modules used for demonstration and verification.
Relationships and Flow
[Source Input] → [Lexing/Parsing] → [AST Formation] → [Semantic Analysis (imports, type inference, name resolution)] → [Lowering (IR generation and transformation)] → [Code Generation] → [Backend Processing (LLVM/chibi/native)]
– Also show a separate connection from the code generation component to “xten modules” if the chibi (x86_64-targeted) backend is used.
– Highlight that both compiler implementations (llrl0 and llrl1) share similar pipelines but differ in their implementation language and subtle processing differences.
Architectural Patterns and Design Principles
– Emphasize a modular layered design where each stage of the compilation process (parsing, semantic analysis, lowering, code generation) is encapsulated in its dedicated module.
– Demonstrate the separation between core language processing and the libraries/tools (i.e. standard library, runtime, assembler/JIT linker in xten).