How sharir/jai works
1. This repository is a native C++ implementation of an alternate Jai compiler. The architecture is centered in csrc/, with a small set of frontend/compiler components and a few support abstractions around files, tokens, and platform handling.
2. The top-level entry point is likely csrc/jai.cpp, which ties together the compilation pipeline. build.sh is the project-level build driver, and the repo is meant to be opened in Sublime Text with a local build task rather than deployed as a service.
3. Source input enters through the file layer in csrc/file.cpp and csrc/file.h. This looks like the boundary for reading source text from disk and representing source files before they are transformed by the compiler pipeline.
4. The preprocessing stage is split out explicitly in csrc/preprocessor.cpp and csrc/preprocessor.h. That suggests the compiler does not send raw source directly to parsing; instead it first expands or normalizes input, probably handling source-level directives or include-like behavior.
5. Lexing is handled separately in csrc/lexer.cpp and csrc/lexer.h. The lexer converts preprocessed character streams into tokens, and csrc/token.h defines the token data model shared by later stages.
6. Parsing is isolated in csrc/parser.cpp and csrc/parser.h. This stage consumes tokens and builds the language’s structural representation, likely producing an AST or similar intermediate form for the compiler to operate on.
7. csrc/compiler.cpp and csrc/compiler.h appear to orchestrate the overall compilation flow. In architectural terms, this is the coordination layer that connects preprocessing, lexing, parsing, and any later semantic or output-generation steps.
8. csrc/os.h provides a platform boundary, and csrc/utils.cpp / csrc/utils.h provide shared helper routines. These support modules keep OS-specific behavior and generic helpers out of the language-processing stages, which helps keep the frontend logic cleaner.
9. The grammar/ directory is a language reference asset rather than runtime code. It likely documents or experiments with Jai syntax and serves as a specification aid for the parser and lexer implementation.
10. The test/ directory contains small Jai source files such as hello.jai, lexerror.jai, and multiline.jai. These are likely used as manual or scripted validation inputs for the frontend pipeline, especially lexing and parsing behavior.
11. The main data flow is therefore: source file on disk → file loader → preprocessor → lexer → token stream → parser → compiler orchestration. Any architecture diagram for this repo should emphasize that it is primarily a compiler frontend with a clear stage-by-stage pipeline, plus thin cross-cutting support for files, utilities, and OS concerns.
12. The project is intentionally experimental and not positioned as a specification-perfect Jai compiler. That means the architecture is driven more by an incremental implementation of compiler stages than by a large runtime or toolchain ecosystem.
Main entry points: compiler.cpp Compiler · lexer.cpp Lexer · parser.cpp Parser · preprocessor.cpp Preprocessor
Components 21
Compiler Core
Native C++ compiler frontend and orchestration code in csrc/
- jai.cpp · Program entrypoint; starts the compiler pipeline
csrc/jai.cpp - Compiler · Coordinates the overall compilation flow across stages
csrc/compiler.cpp - File I/O · Reads source text and models files before compilation
csrc/file.cpp - Preprocessor · Normalizes or expands source before lexing
csrc/preprocessor.cpp - Lexer · Converts character input into tokens
csrc/lexer.cpp - Token Model · Shared token representation consumed by later stages
csrc/token.h - Parser · Consumes tokens and builds the language structure
csrc/parser.cpp - OS Boundary · Platform-specific abstraction used by the compiler frontend
csrc/os.h - Utils · Shared helper routines used across compiler stages
csrc/utils.cpp - file.h · File layer interface
csrc/file.h - preprocessor.h · Preprocessor interface
csrc/preprocessor.h - lexer.h · Lexer interface
csrc/lexer.h - parser.h · Parser interface
csrc/parser.h - compiler.h · Compiler orchestration interface
csrc/compiler.h - utils.h · Utilities interface
csrc/utils.h
Language Assets
Reference grammar and sample Jai inputs used to guide and validate the compiler
- Grammar · Language reference and syntax asset for Jai
grammar - hello.jai · Basic frontend validation input
test/hello.jai - lexerror.jai · Lexing edge-case input
test/lexerror.jai - multiline.jai · Multi-line source sample for frontend behavior
test/multiline.jai
Connections 21
- jai.cpp → Compiler: starts · Entrypoint launches the compilation pipeline
- Compiler → File I/O: loads · Compiler begins by reading source files
- Compiler → Preprocessor: runs · Compilation flow sends source through preprocessing
- Preprocessor → Lexer: feeds · Preprocessed text is handed to the lexer
- Lexer → Token Model: emits · Lexer produces shared token objects
- Token Model → Parser: consumed by · Tokens are consumed by the parser
- Parser → Compiler: returns · Parsed structure feeds back into orchestration
- Compiler → OS Boundary: uses · Compiler relies on platform abstractions
- Compiler → Utils: uses · Compiler shares generic helper routines
- build.sh → jai.cpp: builds · Build script compiles the native entrypoint
- README → build.sh: documents · Repository documentation describes build and usage
- Grammar → Parser: guides · Grammar asset informs parsing behavior
- hello.jai → Lexer: validates · Sample input exercises lexer behavior
- lexerror.jai → Lexer: validates · Edge-case sample targets lexing errors
- multiline.jai → Parser: validates · Sample input exercises multi-line parsing behavior
- file.h → File I/O: declares · File layer interface pairs with implementation
- preprocessor.h → Preprocessor: declares · Preprocessor interface pairs with implementation
- lexer.h → Lexer: declares · Lexer interface pairs with implementation
- parser.h → Parser: declares · Parser interface pairs with implementation
- compiler.h → Compiler: declares · Compiler interface pairs with implementation
- utils.h → Utils: declares · Utilities interface pairs with implementation