How pes18fan/zen works
1. What this repository is
- Zen is a small dynamically typed language implemented in Odin.
- The repo contains the full language stack: command-line entrypoint, lexer/compiler/VM runtime, garbage collector, standard library, tests, benchmarks, docs, and editor syntax packages.
2. Top-level execution path
- zen/main.odin is the runtime entrypoint.
- x.py is the developer-facing orchestration script for building, running tests, running benchmarks, and generating docs.
- The runtime supports at least two modes implied by the repo: normal script execution and an interactive REPL, with REPL line editing handled by the bundled isocline dependency.
3. Frontend: source text to bytecode
- zen/lexer.odin is the source tokenizer.
- zen/compiler.odin is the core frontend component and likely owns parsing plus bytecode emission, since there is no separate parser file.
- zen/chunk.odin represents the compiled instruction container that the compiler produces and the VM consumes.
- zen/format.odin and zen/debug.odin support human-readable inspection of compiled chunks / runtime state.
4. Execution engine
- zen/vm.odin is the central interpreter for Zen bytecode.
- It executes compiled chunks, manages call frames, evaluates expressions/statements, and enforces language semantics.
- The VM is the boundary where features like function calls, closures, classes, inheritance, control flow, and module import behavior become runtime behavior.
5. Value and object model
- zen/value.odin defines the dynamically typed value representation used throughout the runtime.
- zen/object.odin defines heap-allocated runtime objects: functions, closures, classes, instances, bound methods, strings, and likely other language-level entities.
- This split suggests a classic VM architecture where primitive values travel on a stack, while structured runtime entities live on the heap and are referenced through tagged values.
6. Collections and symbol storage
- zen/table.odin is the repository’s hash-table implementation.
- It is likely used for globals, object fields, interned strings, and module/class method storage.
- The presence of extensive tests for fields, modules, and globals indicates this table layer is a foundational runtime primitive rather than a user-facing collection only.
7. Memory management
- zen/gc.odin provides garbage collection for heap objects.
- This is important architecturally because the VM, object model, and table storage must cooperate with tracing and object lifetime management.
- The language’s closures, classes, modules, and list/string operations all depend on GC-managed heap allocation.
8. Standard library and native functions
- zen/std.odin is the minimal standard library surface.
- The test suite shows native capabilities such as len, read, write, parse, replace, sort, sqrt, str, and typeof.
- These are likely exposed as built-in or native functions bridged directly from the VM into host-side Odin code.
9. Module system
- Zen has a file-based module system with relative imports and public/private exports.
- The tests under test/__tests__/modules show that modules can be imported, piped through, treated as values, and protected from direct modification.
- This means module loading is a distinct runtime concern, not just syntactic sugar: the VM/compiler must resolve file paths, cache module state, and enforce export visibility.
10. Language feature coverage is runtime-driven
- The black-box test suite suggests the implementation is organized around semantic features rather than internal classes.
- Major language areas exercised include assignments, closures, conditionals, loops, functions, lists, strings, operators, classes/inheritance, this/super, switch, pipes, and native calls.
- For an architecture diagram, these should be shown as language semantics implemented mainly in the compiler + VM layers, not as separate subsystems.
11. Testing architecture
- test/__tests__ is a large set of feature-focused .zn programs.
- This indicates end-to-end testing: the interpreter is run against real Zen programs and expected behavior is validated externally.
- test/run_tests.py orchestrates those cases, while test/run_benchmarks.py runs performance workloads.
12. Benchmarks and performance validation
- test/benchmark contains representative programs for closures, trees, list ops, string ops, classes, pipes, and method calls.
- These benchmarks imply the runtime is performance-sensitive and that the VM, GC, and object model are core optimization targets.
13. Developer and documentation surface
- DOCUMENTATION.md is the authoritative language reference.
- examples/ provides runnable sample programs and serves as usage documentation.
- Editor integration lives under syntaxes/ for Sublime, VS Code, and Vim; this is peripheral to execution but useful for language adoption.
14. Important architectural boundary
- The clearest boundary in the repo is:
- source programs (.zn)
- -> lexer/compiler
- -> bytecode chunk
- -> VM runtime
- -> heap objects / tables / GC
- Everything else in the repo supports that pipeline: native library functions, module loading, testing, benchmarks, and syntax tooling.
Main entry points: vm.odin VM · main.odin Main · chunk.odin Chunk · x.py x.py
Components 18
Core runtime
Language implementation and execution engine
- Main · CLI entrypoint for running Zen code and REPL mode
zen/main.odin - Lexer · Tokenizes source text
zen/lexer.odin - Compiler · Parses and emits bytecode chunks
zen/compiler.odin - Chunk · Compiled instruction container consumed by the VM
zen/chunk.odin - VM · Executes bytecode and implements language semantics
zen/vm.odin - Values · Tagged runtime values used across the interpreter
zen/value.odin - Objects · Heap-allocated functions, closures, classes, instances, strings, and methods
zen/object.odin - Tables · Associative storage for globals, fields, methods, and interned names
zen/table.odin - GC · Tracing garbage collector for heap allocation
zen/gc.odin - Stdlib · Built-in native functions and host bridges
zen/std.odin - Debug · Human-readable runtime and chunk inspection support
zen/debug.odin - Format · Pretty-printing and chunk formatting helpers
zen/format.odin - Isocline · Bundled line-editing support for the interactive REPL
zen/isocline/isocline.odin
Tooling
Build, test, benchmark, and docs support
- x.py · Developer script for builds, tests, benchmarks, and docs
x.py
Content
Examples, tests, benchmarks, and reference material
- Tests · Feature-focused Zen programs validated by the interpreter
test/__tests__ - Benchmarks · Performance workloads for runtime hot paths
test/benchmark - Docs · Language reference, manpage source, README, and examples
DOCUMENTATION.md
Editor support
Syntax definitions and integration files
- Syntax · Editor grammar and configuration packages
syntaxes
Connections 21
- Main → Lexer: tokenize · Source text enters the frontend
- Main → Compiler: compile · Entry point drives compilation before execution
- Compiler → Chunk: emit · Compiler produces bytecode chunks
- Main → VM: run · Runtime entrypoint invokes the virtual machine
- VM → Chunk: execute · VM consumes compiled chunks
- VM → Values: uses · Stack and control flow operate on tagged values
- VM → Objects: allocates · Runtime objects are created and manipulated during execution
- Objects → Tables: stores · Objects and classes rely on table-backed maps
- VM → Tables: globals · Global/module/class state is stored in hash tables
- VM → GC: manage · Heap objects are traced and reclaimed by the GC
- Objects → GC: trace · Heap object graph participates in garbage collection
- Stdlib → VM: native calls · Built-ins are exposed through the runtime boundary
- Debug → Chunk: inspect · Debug helpers print chunk contents
- Format → Chunk: render · Formatting helpers produce readable chunk output
- x.py → Tests: orchestrate · Developer script runs the end-to-end test suite
- x.py → Benchmarks: orchestrate · Developer script runs benchmark workloads
- x.py → Docs: generate · Developer tooling supports documentation workflows
- Tests → VM: exercise · Feature tests execute real Zen programs through the interpreter
- Benchmarks → VM: measure · Benchmarks stress the runtime and GC
- Syntax → Docs: support · Editor packages and docs support language adoption
- Main → Isocline: repl input · Interactive mode uses bundled line editing