# rsonquery/rsonpath architecture

> Architecture diagram of the GitHub repository rsonquery/rsonpath, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rsonquery/rsonpath
- Repository: https://github.com/rsonquery/rsonpath

## Overview

##### Project type and purpose
- This is a Rust-based, SIMD-accelerated JSONPath engine and CLI tool (“rq”) composed of multiple interdependent crates:
- rsonpath       – the command-line executable and runner
- rsonpath-lib   – the core query compilation and execution engine
- rsonpath-syntax – the query parser
- rsonpath-benchmarks, rsonpath-test, rsonpath-test-codegen – performance and compliance test harnesses
- book            – documentation site generated by mdBook
- Purpose: parse JSONPath queries, compile them to a fast automaton, execute against streamed or memory-mapped JSON, output matched values or counts, all with SIMD acceleration where available.

##### File structure
- Top-level directories:
- crates/… – library and tool crates
- book/…   – user/developer book source
- fuzz/…   – fuzz targets for ClusterFuzzLite
- .github/… – CI, fuzz, release workflows
- Key config files: Cargo.toml (workspace), Justfile (build/test script), .clusterfuzzlite (fuzz CI), rust-toolchain.toml, shell.nix.
- Sub-crates indicate a layered architecture: syntax → lib → CLI.

##### Extract additional insights from README
- Notes for SIMD feature flags, supported CPU targets.
- Benchmarks and documentation generation processes.

a. Main components
- User & Environment:
- Terminal / Std-in / File system
- CLI (rsonpath crate using clap, simple-logger)
- Query pipeline:
- Parser (rsonpath-syntax crate, nom)
- Compiler (automaton builder in rsonpath-lib)
- Execution Engine (rsonpath-lib: engine modules for head/tail skipping, classification, NFA, SIMD kernels)
- Data I/O:
- Input reader (memmap2 or buffered reader module)
- Output serializer (JSON serializer or line-delimited)
- Support tooling:
- Benchmarks (rsonpath-benchmarks)
- Compliance tests (rsonpath-test, rsonpath-syntax-proptest)
- Docs generator (mdBook in book/)
- CI / Fuzzing:
- GitHub workflows
- ClusterFuzzLite (.clusterfuzzlite)

b. Relationships & data flow
1. User invokes `rq` CLI
2. CLI parses flags & query string (clap)
3. CLI calls Parser → produces AST
4. Compiler transforms AST → automaton (vector-map, smallvec)
5. Engine executes automaton over JSON input (memmap2, SIMD modules)
6. Engine emits matches → Serializer → Stdout or file
7. Optional paths: benchmarking harness invokes engine directly; fuzz targets feed arbitrary input into parser & engine.

c. Architectural patterns & principles
- Layered modular design: parser → compiler → executor
- Feature flags for conditional SIMD modules (cfg-if)
- Separation of concerns: CLI vs. core library
- Reusable crates for testing and benchmarking

d. Key technologies & libraries
- Rust crates: clap, nom, memmap2, smallvec, vector-map, thiserror, eyre
- SIMD acceleration: AVX2/SSE modules under classification and engine in rsonpath-lib
- Build/test: cargo + Just, GitHub Actions, ClusterFuzzLite, shell.nix

## Diagram (Mermaid)

```mermaid
graph TB
    %% CLI Crate
    subgraph "rsonpath CLI (Crate)" 
        CLI_Main["main.rs"]:::primary
        CLI_Args["args.rs"]:::primary
        CLI_Runner["runner.rs"]:::primary
        CLI_Input["input.rs"]:::primary
        CLI_Logger["logger.rs"]:::primary
        CLI_Error["error.rs"]:::primary
    end

    %% Parser Crate
    subgraph "rsonpath-syntax (Parser Crate)"
        Syntax_Lib["lib.rs"]:::primary
        Syntax_Parser["parser.rs"]:::primary
        Syntax_Builder["builder.rs"]:::primary
        Syntax_Prelude["prelude.rs"]:::primary
        Syntax_Error["error.rs"]:::primary
    end

    %% Core Engine Crate
    subgraph "rsonpath-lib (Core Engine Crate)"
        Lib_Public["lib.rs"]:::primary
        Automaton["automaton.rs"]:::primary
        Engine["engine.rs"]:::primary
        Classification["classification.rs"]:::primary
        Input_Reader["input.rs"]:::primary
        Output_Ser["serde.rs"]:::io
    end

    %% Data I/O
    subgraph "Data I/O"
        FS["File System / stdin"]:::io
        Memmap["memmap2 Reader"]:::io
        Buffered["Buffered Reader"]:::io
        Serializer["Output Serializer"]:::io
    end

    %% Benchmarks & Tests
    subgraph "Benchmarks & Test Harnesses"
        Bench["rsonpath-benchmarks"]:::harness
        SyntaxTest["rsonpath-syntax-proptest"]:::harness
        Codegen["rsonpath-test-codegen"]:::harness
        TestHarness["rsonpath-test"]:::harness
    end

    %% CI & Fuzz
    subgraph "CI & Fuzz"
        GH["GitHub Actions"]:::ci
        CF["ClusterFuzzLite"]:::ci
    end

    %% Data Flow & Dependencies
    CLI_Runner -->|"parses query"| Syntax_Parser
    Syntax_Parser -->|"produces AST"| Automaton
    Automaton -->|"builds automaton"| Engine
    FS -->|reads JSON| Input_Reader
    Memmap -->|memory-map| Input_Reader
    Buffered -->|buffered read| Input_Reader
    Input_Reader -->|feeds JSON| Engine
    Engine -->|matches| Serializer
    Serializer -->|writes output| FS

    %% Harness Relationships
    Bench -->|uses engine| Engine
    SyntaxTest -->|tests parser| Syntax_Lib
    Codegen -->|generates tests| TestHarness
    TestHarness -->|validates| Lib_Public
    TestHarness -->|validates| Syntax_Lib

    %% CI & Fuzz Pipelines
    GH -.-> CLI_Main
    GH -.-> Syntax_Lib
    GH -.-> Lib_Public
    GH -.-> Bench
    CF -.-> SyntaxTest
    CF -.-> CF

    %% Click Events
    click CLI_Main "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath/src/main.rs"
    click CLI_Args "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath/src/args.rs"
    click CLI_Runner "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath/src/runner.rs"
    click CLI_Input "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath/src/input.rs"
    click CLI_Logger "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath/src/logger.rs"
    click CLI_Error "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath/src/error.rs"

    click Syntax_Lib "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-syntax/src/lib.rs"
    click Syntax_Parser "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-syntax/src/parser.rs"
    click Syntax_Builder "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-syntax/src/builder.rs"
    click Syntax_Prelude "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-syntax/src/prelude.rs"
    click Syntax_Error "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-syntax/src/error.rs"

    click Lib_Public "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/lib.rs"
    click Automaton "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/automaton.rs"
    click Engine "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/engine.rs"
    click Classification "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/classification.rs"
    click Input_Reader "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/input.rs"
    click Output_Ser "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/engine/serde.rs"

    click Bench "https://github.com/rsonquery/rsonpath/tree/main/crates/rsonpath-benchmarks/"
    click SyntaxTest "https://github.com/rsonquery/rsonpath/tree/main/crates/rsonpath-syntax-proptest/"
    click Codegen "https://github.com/rsonquery/rsonpath/tree/main/crates/rsonpath-test-codegen/"
    click TestHarness "https://github.com/rsonquery/rsonpath/tree/main/crates/rsonpath-test/"

    click FS "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/input.rs"
    click Memmap "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/input/memmap.rs"
    click Buffered "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/input/buffered.rs"
    click Serializer "https://github.com/rsonquery/rsonpath/blob/main/crates/rsonpath-lib/src/engine/serde.rs"

    click GH "https://github.com/rsonquery/rsonpath/tree/main/.github/workflows/"
    click CF "https://github.com/rsonquery/rsonpath/blob/main/.clusterfuzzlite/project.yaml"

    %% Styles
    classDef primary fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef io fill:#98FB98,stroke:#333,stroke-width:1px
    classDef harness fill:#FFD700,stroke:#333,stroke-width:1px
    classDef ci stroke-dasharray: 5 5,stroke:#333
```
