# happyseaweed/lemon-language architecture

> Architecture diagram of the GitHub repository happyseaweed/lemon-language, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/happyseaweed/lemon-language
- Repository: https://github.com/happyseaweed/lemon-language

## Overview

1. **What this repo is**
   - This is a work-in-progress compiler for the Lemon language.
   - The language is designed around basic control flow, functions, and tensor-oriented operations, with LLVM used for IR generation, optimization, and execution.

2. **Top-level structure**
   - The repository contains two compiler generations: `lemon-0` and `lemon-1`.
   - `lemon-0` is the smaller initial pipeline.
   - `lemon-1` is the more complete architecture and is the best reference for the current system design.

3. **Source inputs and examples**
   - `.lem` files are Lemon programs.
   - The repo includes sample programs such as `example.lem`, `simple.lem`, `complex.lem`, `tensor.lem`, and `test/sanity.lem`.
   - `grammar.md` in `lemon-1` documents the language syntax, so it acts as the contract between the parser and the rest of the compiler.

4. **Front-end pipeline**
   - The compiler pipeline starts with `Lexer`, which tokenizes source text.
   - `Parser` consumes tokens and builds the AST.
   - `AST` is the central internal representation shared by later phases.
   - In `lemon-0`, this front-end is the main focus; in `lemon-1`, it feeds additional analysis and lowering stages.

5. **Semantic and type layer in `lemon-1`**
   - `SemanticAnalyzer` validates the parsed program before code generation.
   - `SymbolTable` tracks scoped identifiers, functions, and variables.
   - `Types` defines the language’s type system, including the key distinction between `float` and `tensor`.
   - `BuilderStack` appears to support nested construction or traversal state during analysis/codegen, likely helping manage scoped blocks and expression building.

6. **Tensor-specific runtime model**
   - `Tensor` is a dedicated subsystem in `lemon-1`, indicating tensors are not just syntactic sugar but a first-class runtime/compiler concept.
   - This likely forms the bridge between high-level tensor syntax and the generated LLVM-side representation.
   - The presence of tensor-specific files suggests the compiler is intentionally structured around ML-style data operations rather than only scalar arithmetic.

7. **Code generation and LLVM boundary**
   - `Codegen` is the lowering boundary from validated AST into LLVM IR.
   - `output.ll` at the repo root suggests the compiler can emit LLVM IR as a visible artifact.
   - This separation means the compiler architecture is: parse and analyze in Lemon-specific code, then hand off to LLVM for optimization/execution.

8. **Execution/JIT boundary**
   - `LemonJIT.h` exists in both versions, showing the project supports JIT-style execution rather than only static compilation.
   - The JIT sits at the end of the pipeline after IR generation, acting as the runtime execution layer for compiled Lemon code.

9. **Entry points and tooling**
   - `src/lemon.cc` is the executable entry point in both versions.
   - CMake is the build system, so the repo is organized around a native C++ build and binary output.
   - This is not an interpreted scripting project; it is a compiled toolchain with a command-line driver.

10. **Debugging and observability**
   - `ShowAST.cc` in `lemon-1` is a diagnostic/inspection utility for visualizing or printing the AST.
   - That indicates the compiler has explicit developer-facing tooling to inspect internal IR, which is useful for architecture separation and debugging parser/semantic issues.

11. **Version progression**
   - `lemon-0` contains the basic compiler skeleton: lexer, parser, AST, and JIT integration.
   - `lemon-1` expands that into a real language implementation with semantic analysis, symbol management, tensor support, type definitions, and LLVM code generation.
   - For architecture diagrams, `lemon-1` should be treated as the authoritative pipeline, with `lemon-0` as an earlier prototype.

12. **Overall data flow**
   - The main flow is: `.lem` source file → lexer → parser → AST → semantic analysis / symbol resolution / type handling → code generation to LLVM IR → JIT execution or IR output.
   - The main architectural boundary is between Lemon-specific language processing and LLVM-backed lowering/execution.
   - The key domain-specific subsystem is tensor support, which likely influences parsing, typing, and codegen more than scalar-only features would.

Main entry points: [`lemon-1/include/SemanticAnalyzer.h`](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/SemanticAnalyzer.h) (Semantic), [`lemon-1/include/AST.h`](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/AST.h) (AST), [`lemon-1/include/Codegen.h`](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Codegen.h) (Codegen), [`lemon-0/src/lemon.cc`](https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/src/lemon.cc) (Driver)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_root["Repository"]
  node_root_readme["README<br/>project doc<br/>[README.md]"]
  node_root_output["LLVM IR<br/>generated artifact<br/>[output.ll]"]
end

subgraph group_lemon0["lemon-0"]
  node_l0_entry["Driver<br/>compiler entry<br/>[lemon.cc]"]
  node_l0_lexer["Lexer<br/>[Lexer.h]"]
  node_l0_parser["Parser<br/>[Parser.h]"]
  node_l0_ast(("AST<br/>syntax tree<br/>[AST.h]"))
  node_l0_jit["JIT<br/>execution runtime<br/>[LemonJIT.h]"]
end

subgraph group_lemon1["lemon-1"]
  node_l1_entry["Driver<br/>compiler entry<br/>[lemon.cc]"]
  node_l1_lexer["Lexer<br/>[Lexer.h]"]
  node_l1_parser["Parser<br/>[Parser.h]"]
  node_l1_ast(("AST<br/>syntax tree<br/>[AST.h]"))
  node_l1_semantic["Semantic<br/>analysis pass<br/>[SemanticAnalyzer.h]"]
  node_l1_symbols["Symbols<br/>scope table<br/>[SymbolTable.h]"]
  node_l1_types["Types<br/>type system<br/>[Types.h]"]
  node_l1_stack["Builder Stack<br/>analysis state<br/>[BuilderStack.h]"]
  node_l1_tensor[("Tensor<br/>[Tensor.h]")]
  node_l1_codegen["Codegen<br/>llvm lowering<br/>[Codegen.h]"]
  node_l1_jit["JIT<br/>execution runtime<br/>[LemonJIT.h]"]
  node_l1_showast["Show AST<br/>diagnostic tool<br/>[ShowAST.cc]"]
end

subgraph group_examples["Programs"]
  node_root_example["Example<br/>sample program<br/>[example.lem]"]
  node_l1_grammar["Grammar<br/>language spec<br/>[grammar.md]"]
end

node_root_readme -->|"describes"| node_l0_entry
node_root_readme -->|"describes"| node_l1_entry
node_root_example -->|"feeds"| node_l0_entry
node_root_example -->|"feeds"| node_l1_entry
node_l1_grammar -->|"defines"| node_l1_parser
node_l0_entry -->|"starts"| node_l0_lexer
node_l0_lexer -->|"tokens"| node_l0_parser
node_l0_parser -->|"builds"| node_l0_ast
node_l0_ast -->|"executes via"| node_l0_jit
node_l1_entry -->|"starts"| node_l1_lexer
node_l1_lexer -->|"tokens"| node_l1_parser
node_l1_parser -->|"builds"| node_l1_ast
node_l1_ast -->|"analyzed by"| node_l1_semantic
node_l1_semantic -->|"uses"| node_l1_symbols
node_l1_semantic -->|"checks"| node_l1_types
node_l1_semantic -->|"tracks"| node_l1_stack
node_l1_ast -->|"contains"| node_l1_tensor
node_l1_tensor -->|"typed by"| node_l1_types
node_l1_semantic -->|"lowers to"| node_l1_codegen
node_l1_codegen -->|"emits"| node_root_output
node_l1_codegen -->|"hands off to"| node_l1_jit
node_l1_ast -.->|"inspected by"| node_l1_showast
node_l0_jit -.->|"evolves into"| node_l1_jit

click node_root_readme "https://github.com/happyseaweed/lemon-language/blob/main/README.md"
click node_root_output "https://github.com/happyseaweed/lemon-language/blob/main/output.ll"
click node_root_example "https://github.com/happyseaweed/lemon-language/blob/main/example.lem"
click node_l0_entry "https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/src/lemon.cc"
click node_l0_lexer "https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/include/Lexer.h"
click node_l0_parser "https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/include/Parser.h"
click node_l0_ast "https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/include/AST.h"
click node_l0_jit "https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/include/LemonJIT.h"
click node_l1_entry "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/src/lemon.cc"
click node_l1_grammar "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/grammar.md"
click node_l1_lexer "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Lexer.h"
click node_l1_parser "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Parser.h"
click node_l1_ast "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/AST.h"
click node_l1_semantic "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/SemanticAnalyzer.h"
click node_l1_symbols "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/SymbolTable.h"
click node_l1_types "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Types.h"
click node_l1_stack "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/BuilderStack.h"
click node_l1_tensor "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Tensor.h"
click node_l1_codegen "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Codegen.h"
click node_l1_jit "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/LemonJIT.h"
click node_l1_showast "https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/src/ShowAST.cc"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_root_readme,node_root_output toneBlue
class node_l0_entry,node_l0_lexer,node_l0_parser,node_l0_ast,node_l0_jit toneAmber
class node_l1_entry,node_l1_lexer,node_l1_parser,node_l1_ast,node_l1_semantic,node_l1_symbols,node_l1_types,node_l1_stack,node_l1_tensor,node_l1_codegen,node_l1_jit,node_l1_showast toneMint
class node_root_example,node_l1_grammar toneRose
```

## Components

### Repository

Top-level Lemon compiler workspace with two generations and shared examples.

- [README](https://github.com/happyseaweed/lemon-language/blob/main/README.md): project doc · Repository overview and project context. · `README.md`
- [LLVM IR](https://github.com/happyseaweed/lemon-language/blob/main/output.ll): generated artifact · Visible LLVM output emitted by the compiler. · `output.ll`

### lemon-0

Early compiler prototype focused on lexing, parsing, AST, and JIT hookup.

- [Driver](https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/src/lemon.cc): compiler entry · Command-line entry point for the first compiler generation. · `lemon-0/src/lemon.cc`
- [Lexer](https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/include/Lexer.h): frontend lexer · Tokenizes Lemon source text. · `lemon-0/include/Lexer.h`
- [Parser](https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/include/Parser.h): frontend parser · Builds the AST from token streams. · `lemon-0/include/Parser.h`
- [AST](https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/include/AST.h): syntax tree · Shared abstract syntax tree used by later phases. · `lemon-0/include/AST.h`
- [JIT](https://github.com/happyseaweed/lemon-language/blob/main/lemon-0/include/LemonJIT.h): execution runtime · Execution boundary for compiled code in the prototype pipeline. · `lemon-0/include/LemonJIT.h`

### lemon-1

Current compiler architecture with semantics, tensors, lowering, and diagnostics.

- [Driver](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/src/lemon.cc): compiler entry · Command-line entry point for the current compiler generation. · `lemon-1/src/lemon.cc`
- [Lexer](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Lexer.h): frontend lexer · Tokenizes Lemon source text. · `lemon-1/include/Lexer.h`
- [Parser](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Parser.h): frontend parser · Constructs the AST from tokens. · `lemon-1/include/Parser.h`
- [AST](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/AST.h): syntax tree · Central internal representation shared across analysis and lowering. · `lemon-1/include/AST.h`
- [Semantic](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/SemanticAnalyzer.h): analysis pass · Validates names, scopes, and program rules before code generation. · `lemon-1/include/SemanticAnalyzer.h`
- [Symbols](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/SymbolTable.h): scope table · Tracks scoped identifiers, functions, and variables. · `lemon-1/include/SymbolTable.h`
- [Types](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Types.h): type system · Defines Lemon types, including scalar and tensor distinctions. · `lemon-1/include/Types.h`
- [Builder Stack](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/BuilderStack.h): analysis state · Supports nested construction and traversal state during compilation. · `lemon-1/include/BuilderStack.h`
- [Tensor](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Tensor.h): tensor subsystem · First-class tensor representation and lowering support. · `lemon-1/include/Tensor.h`
- [Codegen](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/Codegen.h): llvm lowering · Lowers validated Lemon AST into LLVM IR. · `lemon-1/include/Codegen.h`
- [JIT](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/include/LemonJIT.h): execution runtime · LLVM-backed execution boundary for compiled Lemon programs. · `lemon-1/include/LemonJIT.h`
- [Show AST](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/src/ShowAST.cc): diagnostic tool · Developer utility for inspecting the AST. · `lemon-1/src/ShowAST.cc`

### Programs

Sample Lemon inputs and language contract documents.

- [Example](https://github.com/happyseaweed/lemon-language/blob/main/example.lem): sample program · Top-level Lemon sample input. · `example.lem`
- [Grammar](https://github.com/happyseaweed/lemon-language/blob/main/lemon-1/grammar.md): language spec · Syntax contract for the Lemon front end. · `lemon-1/grammar.md`

## Connections

- README → Driver: describes · Repository documentation points to the prototype compiler.
- README → Driver: describes · Repository documentation points to the current compiler.
- Example → Driver: feeds · Sample Lemon source can be compiled by the prototype.
- Example → Driver: feeds · Sample Lemon source can be compiled by the current pipeline.
- Grammar → Parser: defines · Grammar is the contract consumed by the parser.
- Driver → Lexer: starts · Driver invokes lexical analysis.
- Lexer → Parser: tokens · Tokens are passed to the parser.
- Parser → AST: builds · Parser produces the AST.
- AST → JIT: executes via · Prototype pipeline hands off to JIT execution.
- Driver → Lexer: starts · Driver invokes lexical analysis.
- Lexer → Parser: tokens · Tokens are passed to the parser.
- Parser → AST: builds · Parser produces the AST.
- AST → Semantic: analyzed by · Semantic analysis validates the AST.
- Semantic → Symbols: uses · Semantic analysis relies on scoped symbol tracking.
- Semantic → Types: checks · Semantic analysis enforces type rules.
- Semantic → Builder Stack: tracks · Analysis uses builder state for nested traversal/construction.
- AST → Tensor: contains · Tensor constructs are part of the language model.
- Tensor → Types: typed by · Tensor handling depends on the type system.
- Semantic → Codegen: lowers to · Validated programs are lowered for code generation.
- Codegen → LLVM IR: emits · Code generation produces LLVM IR output.
- Codegen → JIT: hands off to · Lowered IR can be executed by the JIT boundary.
- AST → Show AST: inspected by · AST inspection utility reads the tree for debugging.
- JIT → JIT: evolves into · The runtime boundary persists across compiler generations.
