# dasunpubudumal/rlox_lexer architecture

> Architecture diagram of the GitHub repository dasunpubudumal/rlox_lexer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dasunpubudumal/rlox_lexer
- Repository: https://github.com/dasunpubudumal/rlox_lexer

## Overview

##### Project type and purpose
- This is a Rust library (“crate”) implementing the **lexer** for the Lox language (from Crafting Interpreters).
- It is **not** a full-stack app or service; it’s a compiler-component focusing exclusively on **lexical analysis**.
- Its public API is exposed in `src/lib.rs`, and it’s exercised via unit tests in `tests/lexer_test.rs`.

##### File structure
- Top-level:
- Cargo.toml / Cargo.lock – Rust package manifest
- .github/workflows/rust.yml – CI build/test pipeline
- src/ – primary source files
- tests/ – integration tests + fixtures
- src/ breakdown:
- constants/ – token type definitions, reserved words, character classes
- error_handler.rs – error reporting utilities (line numbers, messages)
- scanner/ – lexical analysis code (`lexical_analysis.rs`)
- lib.rs – public API surface
- error_handler.rs & constants/mod.rs – support modules

##### README for additional insights
- Purpose: “Lexer written in Rust for Lox.”
- No multi-module application layers beyond scanning.
- CI badge indicates automated testing with GitHub Actions.
- Tests use debug logging; no external services or databases.

##### Main components & relationships
2) **Scanner Module** (`src/scanner/lexical_analysis.rs`)
3) **Constants Module** (`src/constants`) – token definitions, literals
4) **Error Handler** (`src/error_handler.rs`)
5) **Public API** (`src/lib.rs`) – orchestrates scanning calls
6) **Test Harness** (`tests/lexer_test.rs` + fixtures)
7) **CI Pipeline** (GitHub Actions) – external service

- Input Source → Public API (consumes a string or file path)
- Public API → Scanner Module (invokes `scan_tokens`)
- Scanner Module → Constants Module (looks up token types / keywords)
- Scanner Module → Error Handler (invokes on unrecognized lexemes)
- Scanner Module → Public API (returns list of tokens)
- Public API → Test Harness (tests invoke API)
- Tests → Fixtures (input programs)
- Commits/PRs → CI Pipeline (triggers build & tests)

c. Architectural patterns & principles
- **Modular decomposition**: each concern in its own Rust module
- **Single Responsibility**: scanning logic vs. error reporting vs. definitions
- **Layered calls**: API → Scanner → Support modules
- **Test-driven**: all lexical features covered by tests

d. Key technologies
- Rust (edition as per Cargo.toml)
- Cargo (build system)
- GitHub Actions (CI)

– **Constants Module** (token definitions)
– **Error Handler** (error_handler.rs)
7) At the top or side, show **GitHub Actions CI** as a service that consumes pushes/PRs and runs tests.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Main Components
    subgraph "Source"
        Input["Input Source\n(Lox code)"]:::external
    end

    subgraph "rlox_lexer Library"
        API["Public API\n(src/lib.rs)"]:::module
        subgraph "Scanner Module"
            ScannerEntry["mod.rs"]:::module
            ScannerLogic["lexical_analysis.rs"]:::module
        end
        Constants["Constants Module\n(src/constants/mod.rs)"]:::module
        ErrorHandler["Error Handler\n(src/error_handler.rs)"]:::module
    end

    subgraph "Testing"
        Test["Test Harness\n(tests/lexer_test.rs)"]:::test
        Fixtures["Fixtures\n(tests/fixtures)"]:::external
    end

    CI["GitHub Actions CI\n(.github/workflows/rust.yml)"]:::ci

    %% Data Flows
    Input -->|"&str input"| API
    API -->|"calls scan_tokens"| ScannerEntry
    ScannerEntry -->|"delegates to"| ScannerLogic
    ScannerLogic -->|"lookup token types"| Constants
    ScannerLogic -->|"invoke on errors"| ErrorHandler
    ScannerLogic -->|"returns Vec<Token>"| API
    API -->|"tested by"| Test
    Fixtures -->|"provides inputs"| Test
    CI -.->|"on push/PR triggers"| API
    CI -.->|"runs tests"| Test

    %% Legend
    subgraph Legend
        M["Module"]:::module
        E["External"]:::external
        T["Test Harness"]:::test
        C["CI Pipeline"]:::ci
    end

    %% Click Events
    click Input "https://github.com/dasunpubudumal/rlox_lexer/tree/main/tests/fixtures"
    click API "https://github.com/dasunpubudumal/rlox_lexer/blob/main/src/lib.rs"
    click ScannerLogic "https://github.com/dasunpubudumal/rlox_lexer/blob/main/src/scanner/lexical_analysis.rs"
    click ScannerEntry "https://github.com/dasunpubudumal/rlox_lexer/blob/main/src/scanner/mod.rs"
    click Constants "https://github.com/dasunpubudumal/rlox_lexer/blob/main/src/constants/mod.rs"
    click ErrorHandler "https://github.com/dasunpubudumal/rlox_lexer/blob/main/src/error_handler.rs"
    click Test "https://github.com/dasunpubudumal/rlox_lexer/blob/main/tests/lexer_test.rs"
    click CI "https://github.com/dasunpubudumal/rlox_lexer/blob/main/.github/workflows/rust.yml"

    %% Styles
    classDef module fill:#cce5ff,stroke:#0366d6;
    classDef external fill:#d4edda,stroke:#155724;
    classDef test fill:#fff3cd,stroke:#856404;
    classDef ci fill:#e2e3e5,stroke:#6c757d;
```
