# itsamziii/tagparse architecture

> Architecture diagram of the GitHub repository itsamziii/tagparse, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/itsamziii/tagparse
- Repository: https://github.com/itsamziii/tagparse

## Overview

##### Project type and purpose
- This is an open-source, Node.js library (“tagparse”) for high-performance, stream-based lexing and parsing of custom tag-based syntax.
- Its main goal is to let real-time and high-concurrency apps feed in large text streams (or strings), and extract/transform tokens (literals, variables, functions) into an AST or value stream with minimal memory overhead.

##### File structure
- **src/index.ts**
- Library entry-point that exports `Lexer`, `Parser`, types, etc.
- **src/lib**
- **Errors.ts** – custom error definitions
- **Lexer.ts** – implements the streaming lexer
- **Stream.ts** – a wrapper/adapter over Node.js Readable streams
- **Parser/Parser.ts** – orchestrates token consumption into AST nodes
- **Parser/TagParsers** – contains pluggable tag handlers:
– `VariableTagParser.ts`
– `FunctionTagParser.ts`
– `TagParser.ts` (base interface)
- **src/types.ts** – shared enums & interfaces (e.g., `NodeType`)
- **src/utils.ts** – helper functions
- **tsup.config.ts** – build & bundling config
- **test/** – unit tests for lexer, strict mode, tag parsing
- **.github/** – CI/Publish workflows (ci.yml, publish.yml)

##### README for additional insights
- Core runtime flow:
Input → Lexer (stream or string) → token stream → Parser → AST nodes → consumer logic (variable/function resolution)
- Configurable options: `tagStart`, `tagEnd`, `parseTags`, `strict`, plus custom `functionParser` & `variableParser`.
- Use cases: templating engines, chat systems, dashboards, real-time data processing.

a. Main components:
1. **Consumer/Application** – the calling code (e.g., a templating engine or real-time processor)
2. **Input Source** – can be a raw string or a Node.js Readable stream
3. **Stream Adapter** (`Stream.ts`) – normalizes input into an async iterable of chunks (optional step)
4. **Lexer** (`Lexer.ts`) – reads chunks, identifies tag boundaries & literal segments, emits tokens
5. **Token Stream** – asynchronous stream of tokens with types (literal, tagStart, tagContent, tagEnd)
6. **Parser** (`Parser.ts`) – consumes tokens, applies TagParsers to build AST nodes (Variable, Function, Literal)
7. **TagParsers** – pluggable handlers for each tag type (FunctionTagParser, VariableTagParser)
8. **AST/Node List** – output array of nodes, each carrying raw, type, children, etc.
9. **Consumer Logic** – application-specific code that takes the AST & resolves values (e.g., lookup variables, execute functions)
10. **Error Handler** – central Errors module to throw or recover from syntax issues (strict vs. non-strict)

b. Relationships & interactions:
- The Consumer injects input into the Input Source → Stream Adapter →
- Lexer pulls from Stream Adapter → Token Stream →
- Parser reads from Token Stream → calls appropriate TagParser →
- TagParser returns node fragments back to Parser → AST nodes → back to Consumer → final output.
- At each stage, Errors may be raised (strict mode).

c. Architectural patterns & design principles:
- **Streaming Pipeline** – non-blocking, back-pressure capable.
- **Plugin Pattern** – TagParsers implement a common interface for extensibility.
- **Separation of Concerns** – Lexer ≠ Parser ≠ Consumer logic.
- **Async/Await & Iterators** – flow control via async functions and iterators.

d. Key technologies and libraries:
- Native **Node.js Streams** (Readable)
- **TypeScript** for types and interfaces
- **tsup** for bundling
- **GitHub Actions** for CI and publishing
- **Jest/Mocha** (implied by .test.ts) for unit testing

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Consumer/Application"
        Consumer["Consumer Application<br/>(example.mjs)"]:::external
    end

    Consumer -->|"input stream/string"| Index

    subgraph "Library Entry" 
        Index["Library Entry Point<br/>(src/index.ts)"]:::core
    end

    Index -->|"raw input"| StreamAdapter
    StreamAdapter["Stream Adapter<br/>(src/lib/Stream.ts)"]:::core
    StreamAdapter -->|"chunks"| LexerComp

    LexerComp["Lexer<br/>(src/lib/Lexer.ts)"]:::core
    LexerComp -->|"tokens"| TokenStream

    class TokenStream cylinder
    TokenStream["Token Stream"]:::data
    TokenStream -->|"token async iterable"| ParserComp

    ParserComp["Parser<br/>(src/lib/Parser/Parser.ts)"]:::core
    ParserComp -->|"AST nodes"| AST

    AST["AST / Node List"]:::data
    AST -->|"AST array"| Consumer

    subgraph "Plugin: TagParsers"
        TagInterface["TagParser Interface<br/>(src/lib/Parser/TagParsers/TagParser.ts)"]:::plugin
        VarParser["VariableTagParser<br/>(src/lib/Parser/TagParsers/VariableTagParser.ts)"]:::plugin
        FuncParser["FunctionTagParser<br/>(src/lib/Parser/TagParsers/FunctionTagParser.ts)"]:::plugin
    end

    ParserComp -->|"uses interface"| TagInterface
    TagInterface --> VarParser
    TagInterface --> FuncParser
    VarParser --> ParserComp
    FuncParser --> ParserComp

    subgraph "Configuration & Support"
        Errors["Error Handler<br/>(src/lib/Errors.ts)"]:::config
        Types["Shared Types & Enums<br/>(src/types.ts)"]:::config
        Utils["Utility Functions<br/>(src/utils.ts)"]:::config
        Tsup["Build Config<br/>(tsup.config.ts)"]:::config
        CI["CI Workflow<br/>(.github/workflows/ci.yml)"]:::config
        Publish["Publish Workflow<br/>(.github/workflows/publish.yml)"]:::config
    end

    class ErrorStream review
    LexerComp -->|raises errors| Errors
    ParserComp -->|raises errors| Errors
    Index -->|exports types| Types
    Index -->|uses utils| Utils

    click Consumer "https://github.com/itsamziii/tagparse/blob/main/example.mjs"
    click Index "https://github.com/itsamziii/tagparse/blob/main/src/index.ts"
    click StreamAdapter "https://github.com/itsamziii/tagparse/blob/main/src/lib/Stream.ts"
    click LexerComp "https://github.com/itsamziii/tagparse/blob/main/src/lib/Lexer.ts"
    click ParserComp "https://github.com/itsamziii/tagparse/blob/main/src/lib/Parser/Parser.ts"
    click TagInterface "https://github.com/itsamziii/tagparse/blob/main/src/lib/Parser/TagParsers/TagParser.ts"
    click VarParser "https://github.com/itsamziii/tagparse/blob/main/src/lib/Parser/TagParsers/VariableTagParser.ts"
    click FuncParser "https://github.com/itsamziii/tagparse/blob/main/src/lib/Parser/TagParsers/FunctionTagParser.ts"
    click Errors "https://github.com/itsamziii/tagparse/blob/main/src/lib/Errors.ts"
    click Types "https://github.com/itsamziii/tagparse/blob/main/src/types.ts"
    click Utils "https://github.com/itsamziii/tagparse/blob/main/src/utils.ts"
    click Tsup "https://github.com/itsamziii/tagparse/blob/main/tsup.config.ts"
    click CI "https://github.com/itsamziii/tagparse/blob/main/.github/workflows/ci.yml"
    click Publish "https://github.com/itsamziii/tagparse/blob/main/.github/workflows/publish.yml"

    classDef core fill:#add8e6,stroke:#000,stroke-width:1px
    classDef plugin fill:#90ee90,stroke:#000,stroke-width:1px
    classDef config fill:#ffa500,stroke:#000,stroke-width:1px
    classDef data fill:#cccccc,stroke:#000,stroke-width:1px
    classDef external fill:#ffffff,stroke:#000,stroke-width:1px,stroke-dasharray: 5 5
```
