How sin-ack/zigself works
Project’s Major Areas
• Language Frontend: The parts that tokenize, parse, and build an Abstract Syntax Tree (AST) from Self source files. Look at the files in the src/language directory (e.g., Tokenizer.zig, Parser.zig, grammar.zig, ASTPrinter.zig) – these make up the “compiler front-end.”
• Bytecode Generation: Files in src/runtime/bytecode (as well as the subdirectories “astcode” and “lowcode”) indicate that the system translates the parsed code into an intermediate bytecode form. Include these as the bridge between the source code and runtime.
• Virtual Machine and Runtime: The core execution engine is implemented in src/runtime. This includes the VirtualMachine.zig, Interpreter.zig, context handling, error reporting, and object management modules. Also, many runtime objects and primitives (such as Activation, Actor, ByteArray, and Scheduler) are defined and should be represented.
• Self Object Library: The project includes a set of built-in Self code objects (.self files) under the “objects” directory, which define fundamental language constructs like actor, array, string, and file. These are the built-in “standard library” of the Self world.
• Testing, Examples, and REPL: There are tests (in tests/ directory) and examples (in examples/), plus an experimental REPL (repl.self) that serves as entry points and usage examples.
• Build and Configuration: The build system driven by build.zig and related workflow configurations (in .github/workflows) indicate how the project is built, tested, and deployed, but are less central to the runtime system design.
a. Language Processing Pipeline:
• Source Input: Self source files (e.g., examples, tests, and REPL code).
• Tokenizer & Parser: Represented by src/language/Tokenizer.zig, Parser.zig, Grammar, and related diagnostic modules.
• AST Construction: Show components like ast.zig and its copy visitor as part of generating the code structure.
b. Intermediate Representation and Bytecode Generation:
• AST-to-Bytecode Transformation: Include modules such as AstGen.zig, CodeGen.zig, and supporting low-level code generation (lowcode/).
• Data Structures: Highlight supporting utilities like hash, array, and vector modules.
c. Runtime Execution Engine:
• Interpreter: Represent the interpreter module that isn’t purely compiled but interacts with runtime objects.
• Memory Management & Heap: Include Heap.zig, context management, and error handling.
• Actor System: Emphasize the actor modules under both src/runtime/objects and objects (e.g., actor.self, Actor.zig, Scheduler.zig) showing how the actor model is integrated.
• Object Management: Illustrate the object model (e.g., source files handling objects, slot addition, and inheritance/behavior sharing).
d. Utility and Supporting Modules:
• Utility Components: Highlight modules in src/utility (pointer management, hash handling, reference counting, etc.) as essential backbones that support the higher-level functionality.
Relationships and Data Flow
• Output of the language front-end to the bytecode generation process.
• That the generated bytecode is fed into the Virtual Machine/Interpreter within the runtime layer.
• That the built-in Self objects (from the “objects” directory) are loaded into or interact with the runtime world and contribute to behavior sharing.
Architectural Patterns and Technologies
• Modular separation between the front-end (language processing), intermediate representation (bytecode generation), and execution (runtime/actor system).
• The design uses a layered architecture: the language layer is isolated from bytecode generation, which in turn sits on top of a robust Virtual Machine.
• Actor model design within the runtime, inspired by Erlang’s message-passing and concurrency patterns.
• Use of Zig as the core language, with its build system integrated via build.zig and GitHub CI workflows, which supports cross-compilation (including for WASM).
• Drill down into each layer to show how submodules (tokenizer, parser, AST, etc.) interact within that layer.
• External interactions like the REPL, examples, and test harnesses which serve as entry points to the system.