How tarantool/luajit works
Project Type and Purpose
• This is a language runtime and just‑in‑time (JIT) compiler for Lua (LuaJIT 2.1.0‑beta3).
• Key goal: compile and execute Lua bytecode at high performance by translating hot code paths to machine code at runtime.
High‑Level Architecture Overview
Components:
A. Build System
• CMake scripts (CMakeLists.txt, cmake/…) and Makefiles (Makefile.original, src/Makefile.original).
• Host tools (src/host/*): minilua, buildvm, genlibbc – generate bytecode serializers and utility code at build time.
• DynASM preprocessor (dynasm/*.lua, *.dasc): platform‑agnostic assembler generator used by the JIT to emit machine code.
B. Interpreter & Core VM
• Bytecode reader/dispatcher (src/lj_bcread.c, lj_dispatch.c).
• Lua API implementation (src/lib_*.c, lauxlib.h, lua.h).
• Garbage collector & allocator (src/lj_gc.c, lj_alloc.c).
C. JIT Compiler Engine
• IR builder & optimizer (src/lj_ir.c, lj_opt_*.c).
• Trace recorder (src/lj_record.c, lj_snap.c).
• Machine‑code generator (src/lj_mcode.c, lj_asm.* / lj_emit_*.h).
• Arch‑specific backends (vm_x86.dasc, vm_arm64.dasc, etc. under src/).
• DynASM integration (dynasm.lua) to produce final binary code.
D. Foreign Function Interface (FFI)
• FFI API implementation (src/lib_ffi.c, lj_cdata.c/h).
• ffi‑semantics and tutorial docs (doc/ext_ffi*, doc/ext_ffi_api.html).
E. Standard Libraries & Extensions
• Lua standard libs (src/lib_base.c, lib_math.c, lib_string.c, etc.).
• Profiling tools (tools/memprof.lua, tools/sysprof.lua).
F. Test Suite & Examples
• Comprehensive tests (test/LuaJIT‑tests, PUC‑Rio‑Lua‑5.1‑tests, tarantool tests).
• Harness scripts (test/lua‑Harness‑tests).
Relationships & Data Flows
• Build System → generates → Host Tools binaries (minilua, buildvm)
• Host Tools + source DSL (vm_*.dasc) → generate → C code for VM and JIT backends
• CMake/Make → compiles → static interpreter & JIT engine into luajit executable/libluajit
• Runtime:
• Lua source → parser (lj_parse.c) → bytecode
• Interpreter (lj_dispatch) executes or, on hot paths → JIT engine
• Recorder logs traces → IR → optimizer → DynASM assembler → native machine code → execution cache
• FFI calls bypass VM to C functions (lib_ffi)
• GC & allocator manage memory for both interpreter and JIT objects
Architectural Patterns & Principles
• Layered pipeline: parsing → bytecode → IR → codegen
• Modular backends: dynasm‑based, one .dasc per architecture
• Code‑generation driven by a lightweight host Lua VM (minilua) during build and at runtime
• Separation of concerns: interpreter core vs. JIT engine vs. FFI vs. stdlib