How elkarouh/adascript works
1. Overall shape
- This repo is a source-to-source language toolchain for Adascript.
- The primary flow is: .ady source → parse/AST → emit Python 3 or Nim.
- The two main compilation targets are split cleanly into separate backends under TO_PYTHON/ and TO_NIM/.
2. Frontend boundary: tokenization and parsing
- The shared frontend is centered on hek_tokenize.py, hek_parsec.py, and hek_helpers.py.
- hek_tokenize is not just lexical analysis; it also performs Adascript-specific preprocessing such as tick-attribute rewriting and preserving comment/newline structure.
- hek_parsec provides the parser-combinator engine used to define the language grammar and build the AST.
3. Language definition layer
- ADASCRIPT_GRAMMAR/ contains the grammar split by language area:
- expressions
- simple statements
- compound statements
- declarations / type syntax
- This is the authoritative boundary for Adascript syntax, independent of any output target.
4. AST-to-target translation is backend-specific
- The backend modules attach to_py() and to_nim() methods to grammar nodes.
- Python emission lives in TO_PYTHON/hek_py3_*.py.
- Nim emission lives in TO_NIM/hek_nim_*.py.
- This means the repo is organized around a single parsed representation with two code generators, not around two separate parsers.
5. Entry points and compilation flow
- TO_PYTHON/py2py.py is the Python backend CLI.
- TO_NIM/py2nim.py is the Nim backend CLI.
- Both parse the input file and walk the AST, but they diverge after parsing in their target-specific lowering rules.
- py2nim also has a more involved build pipeline because it can compile, run, cache artifacts, and create source-directory symlinks.
6. Nim build/runtime boundary
- The Nim path is more than transpilation: it includes a compile-and-run pipeline.
- Generated .nim, compiled binaries, and nimcache artifacts are stored in a cache directory under ~/.cache/hparsec/, keyed by source path.
- The backend also inserts a local Nim shim, TO_NIM/stdlib.nim, to cover Python-like runtime conveniences that Nim does not provide directly.
7. Python interoperability and runtime adaptation
- The language supports ordinary Python imports, but the backend decides whether they map natively to Nim modules or need nimpy.
- This makes the compiler responsible for runtime boundary translation, not just syntax conversion.
- Shell integration, regex literals, and Python-object coercions are also handled as compile-time rewrites into backend-specific runtime calls.
8. Adascript features that affect architecture
- Several syntax features are first-class in the compiler and require dedicated lowering logic:
- enum and subrange types
- variant records / discriminated records
- case/when pattern matching
- regex literals and substitution
- shell blocks
- bash-style variables and file tests
- tick attributes like T'First / T'Last
- These are not library add-ons; they are baked into the parse and emit pipeline.
9. Example programs as integration tests
- EXAMPLES/ is a major architectural asset, not just documentation.
- It contains real Adascript programs spanning algorithms, shell processing, timetabling, regex usage, and object-oriented examples.
- Many examples exist in multiple forms (.ady, generated .nim, sometimes .py) and serve as end-to-end validation of both backends.
10. Bundled language/runtime libraries
- TO_NIM/STDLIB/ and TO_NIM/awk.ady provide bundled support code for the Nim target.
- These are treated like installable language libraries and can be imported through nimport.
- This creates a boundary between core compiler output and target-side support modules.
11. Editor/tooling ecosystem
- The repo also ships language tooling for editor integration:
- LSP/adascript_ls.py for language-server behavior
- Emacs, Sublime, and VS Code support packages
- indentation helpers in ADA_INDENT/
- These are ancillary to compilation, but they reinforce the fact that Adascript is a real language ecosystem, not just a transpiler script.
12. Documentation and spec surface
- The README, TUTORIAL.md, TUTORIAL_FOR_LLM.md, OPTIONAL_TYPES.md, and PATTERN_MATCHING.md collectively describe the language contract.
- From an architecture-diagram perspective, these documents sit outside the runtime flow but define the intended semantics that the parser and backends implement.
13. Operational constraints
- The parser state is effectively global, so parsing is not thread-safe across concurrent runs.
- Incremental rebuild logic in py2nim means the compiler is stateful across invocations through its cache, which matters if you are diagramming execution and artifact flow.
14. Mental model for a diagram
- A useful diagram would show:
- Adascript source files at the left
- shared tokenizer/parser/grammar in the center
- two target emitters on the right
- Nim-specific compile/cache/runtime support as a separate branch
- examples, tests, and editor tooling as surrounding ecosystem components
- The key architectural idea is one language frontend, two target-specific lowering pipelines, plus target runtime shims and tooling.
Main entry points: py2nim.py py2nim · py2py.py py2py · HPARSEC Parser core · README.md Docs
Components 19
Frontend
Shared tokenization, parsing, and grammar definition for Adascript.
- Ady source · Adascript source files that enter the compiler pipeline.
- Tokenizer · Shared lexer and Adascript-specific preprocessing such as tick-attribute rewriting and comment/newline preservation.
TO_PYTHON/hek_py3_parser.py - Parser core · Grammar engine and parser-combinator machinery used to build the AST.
HPARSEC - Grammar · Authoritative Adascript syntax split across expressions, statements, compound statements, and declarations.
ADASCRIPT_GRAMMAR - AST · Single parsed representation shared by both code generators.
Backends
Target-specific emitters and entry points for Python and Nim.
- py2py · Python backend entry point that parses input and emits Python 3.
TO_PYTHON/py2py.py - Py expr · Expression lowering for Python output.
TO_PYTHON/hek_py3_expr.py - Py stmt · Statement lowering for Python output.
TO_PYTHON/hek_py3_stmt.py - Py decls · Declaration and type lowering for Python output.
TO_PYTHON/hek_py_declarations.py - py2nim · Nim backend entry point with compile, run, cache, and artifact management.
TO_NIM/py2nim.py - Nim expr · Expression lowering for Nim output.
TO_NIM/hek_nim_expr.py - Nim stmt · Statement lowering for Nim output.
TO_NIM/hek_nim_stmt.py - Nim decls · Declaration and type lowering for Nim output.
TO_NIM/hek_nim_declarations.py
Nim support
Compile, cache, and runtime support used by the Nim pipeline.
- Nim shim · Local Nim runtime shim for Python-like conveniences missing from Nim.
TO_NIM/STDLIB/stdlib.nim - Bundled libs · Bundled target-side support modules and importable libraries for Nim output.
TO_NIM/STDLIB
Ecosystem
Examples, editor tooling, and documentation around the compiler.
- Examples · Real Adascript programs and generated artifacts used as end-to-end validation.
EXAMPLES - LSP · Language-server and editor integrations for Adascript.
LSP - Indent tools · Indentation and syntax-support helpers for editor integration.
ADA_INDENT - Docs · Tutorials, language notes, and feature documentation that define the language contract.
README.md
Connections 19
- Ady source → Tokenizer: lexes · Source files are preprocessed and tokenized before parsing.
- Tokenizer → Parser core: feeds · Token stream enters the parser-combinator engine.
- Parser core → Grammar: uses · Grammar rules define the language accepted by the parser.
- Parser core → AST: builds · Parser constructs the shared AST representation.
- AST → py2py: routes · Parsed programs can be lowered through the Python backend.
- AST → py2nim: routes · Parsed programs can be lowered through the Nim backend.
- py2py → Py expr: lowers · Python CLI delegates expression emission to target-specific modules.
- py2py → Py stmt: lowers · Python CLI delegates statement emission to target-specific modules.
- py2py → Py decls: lowers · Python CLI delegates declarations and type lowering to target-specific modules.
- py2nim → Nim expr: lowers · Nim CLI delegates expression emission to target-specific modules.
- py2nim → Nim stmt: lowers · Nim CLI delegates statement emission to target-specific modules.
- py2nim → Nim decls: lowers · Nim CLI delegates declarations and type lowering to target-specific modules.
- py2nim → Nim shim: shims · Generated Nim code relies on bundled runtime shims for Python-like behavior.
- py2nim → Bundled libs: imports · Nim pipeline can import bundled support libraries through nimport.
- Examples → Ady source: exercises · Examples provide end-to-end input programs for the compiler.
- LSP → Docs: supports · Editor tooling follows the documented language contract.
- Indent tools → Docs: supports · Indentation helpers align with the documented syntax and formatting rules.
- Docs → Grammar: specifies · Documentation describes the intended syntax and semantics implemented by the grammar.
- Examples → py2nim: validates · Some examples are compiled through the Nim pipeline and cached artifacts.