# pkm-m84/shim architecture

> Architecture diagram of the GitHub repository pkm-m84/shim, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pkm-m84/shim
- Repository: https://github.com/pkm-m84/shim

## Overview

1. **What this repository is**
   - This is a Rust CLI project that ships a drop-in replacement for `rg` (ripgrep) called `shim` / `smart-rg`.
   - Its core architectural idea is: intercept search invocations locally, classify them, and either redirect to `ast-grep` for structural code search or fall back to real ripgrep for plain text search.

2. **Primary runtime boundary**
   - The binary is intended to sit on the user’s PATH as `rg`, so external tools that shell out to `rg` invoke this project transparently.
   - There is a second, tool-specific boundary for Claude Code: the installer also sets `USE_BUILTIN_RIPGREP=0` in `~/.claude/settings.json` so Claude uses PATH-based `rg` instead of its bundled binary.

3. **Core execution flow**
   - `src/main.rs` is the main application entrypoint and contains the end-to-end search pipeline.
   - The flow is: parse rg-like flags and query → classify pattern → translate when structural → execute ast-grep or real rg → normalize output back into rg-compatible lines → log the result locally.

4. **Search classification and translation**
   - One subsystem decides whether a query is structural enough for ast-grep.
   - Another subsystem converts rg-style search patterns into ast-grep patterns and maps rg language/type flags to ast-grep language selection.
   - This is the key semantic boundary in the repo: “text search” vs “code-structure search.”

5. **Execution backends**
   - The project has two execution paths:
     - `run_ast_grep()` for structural searches, where ast-grep output is parsed and reformatted to match rg expectations.
     - `exec_real_rg()` for passthrough searches, using the actual ripgrep binary to avoid infinite recursion through the shim itself.
   - The README indicates the real rg path is resolved absolutely to avoid symlink loops.

6. **Logging and local data store**
   - The project persists usage metrics to a local SQLite database under `~/.smart-rg/stats.db`.
   - Logged data is split into:
     - `events`: each intercepted search, including agent, pattern, reason, language, matches, and timestamp.
     - `comparisons`: rg-vs-ast-grep savings data, including file counts, estimated token savings, and cost savings.
   - This is a local-only analytics subsystem; no network telemetry is part of the architecture.

7. **Stats and reporting subsystem**
   - The CLI includes reporting commands beyond search interception: `stats`, `report`, `prune`, and `reset`.
   - `compute_stats()` aggregates event history, while `generate_report()` emits a self-contained HTML dashboard.
   - The HTML report template is compiled into the binary via `include_str!`, so the reporting UI is static and offline.

8. **Installer and environment management**
   - `install.sh` is a major part of the repo’s operational architecture even though it is not Rust code.
   - It manages the user environment by creating `~/.smart-rg/`, placing the shim binary there, ensuring that directory is first on PATH, and optionally modifying Claude settings.
   - It also supports dependency setup, update/migration, uninstall, purge, and a `--check` dry-run mode.

9. **Compatibility and fallback strategy**
   - The project is designed to fail open: if classification fails, ast-grep is unavailable, flags are unsupported, or parsing breaks, it falls back to real ripgrep.
   - That means the shim is a non-breaking compatibility layer, not a hard dependency gate.
   - This behavior is essential to the architecture: correctness of search always outranks optimization.

10. **Concurrency and persistence model**
   - Multiple short-lived `rg` invocations can write to the same SQLite DB concurrently.
   - The README describes WAL mode and a busy timeout, which means logging is best-effort and should not block search execution.
   - This keeps the search path fast while allowing shared local accounting across agents and shells.

11. **Repository structure**
   - `src/main.rs`: the whole Rust CLI application and all runtime logic.
   - `src/report.html`: bundled HTML template for the report output.
   - `docs/stats.html`: likely static documentation or preview material for the report/dashboard.
   - `tests/install_test.sh`: shell-based integration test focused on installer behavior and settings merging.
   - `.github/workflows/*`: CI for install testing and release packaging.

12. **Build and release shape**
   - The project is a Cargo-based Rust binary with a small release footprint and bundled SQLite via `rusqlite`.
   - Release automation is handled in GitHub Actions, with a dedicated release workflow and an install/test workflow.
   - The release model includes prebuilt macOS binaries, while Linux is currently source-build oriented according to the README.

13. **Key external dependencies and their role**
   - `clap` drives the CLI contract and rg-like flag parsing.
   - `rusqlite` provides embedded SQLite access without requiring a system SQLite install.
   - `serde`/`serde_json` are used to consume ast-grep JSON output.
   - `ast-grep` and `ripgrep` are the two external search engines that form the functional backend pair.

14. **What an architecture diagram should show**
   - The central node is the shim binary in the user PATH.
   - Upstream consumers are terminal agents and CLI tools invoking `rg`.
   - Downstream branches are ast-grep for structural searches and real ripgrep for passthrough.
   - A side channel writes to local SQLite, which feeds CLI stats and HTML report generation.
   - The installer sits outside the runtime path but is critical for environment wiring and Claude configuration.

Main entry points: [`install.sh`](https://github.com/pkm-m84/shim/blob/main/install.sh) (install.sh), [`src/main.rs`](https://github.com/pkm-m84/shim/blob/main/src/main.rs) (shim binary), [`tests/install_test.sh`](https://github.com/pkm-m84/shim/blob/main/tests/install_test.sh) (install test), [`src/report.html`](https://github.com/pkm-m84/shim/blob/main/src/report.html) (report template)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_path_rg(("rg callers<br/>external clients"))
  node_shim_bin["shim binary<br/>rg shim<br/>[main.rs]"]
  node_arg_parse["flag parse<br/>CLI parser<br/>[main.rs]"]
  node_classify["classify query<br/>search classifier<br/>[main.rs]"]
  node_translate["translate pattern<br/>pattern mapper<br/>[main.rs]"]
  node_ast_grep[("ast-grep<br/>structural backend")]
  node_real_rg[("ripgrep<br/>text backend")]
  node_normalize["normalize output<br/>result adapter<br/>[main.rs]"]
end

subgraph group_analytics["Analytics"]
  node_stats_db[("stats.db<br/>sqlite store")]
  node_log_events["event log<br/>usage recorder<br/>[main.rs]"]
  node_compute_stats["compute stats<br/>aggregator<br/>[main.rs]"]
  node_generate_report["generate report<br/>html emitter<br/>[main.rs]"]
  node_report_tpl["report template<br/>bundled html<br/>[report.html]"]
end

subgraph group_install["Install"]
  node_installer["install.sh<br/>installer<br/>[install.sh]"]
  node_claude_cfg["Claude settings<br/>env config"]
  node_path_setup["PATH wiring<br/>environment setup"]
end

subgraph group_delivery["Delivery"]
  node_install_test["install test<br/>integration test<br/>[install_test.sh]"]
  node_ci_install["install CI<br/>workflow<br/>[install-test.yml]"]
  node_ci_release["release CI<br/>workflow<br/>[release.yml]"]
end

node_path_rg -->|"invokes"| node_shim_bin
node_shim_bin -->|"parses"| node_arg_parse
node_arg_parse -->|"classifies"| node_classify
node_classify -->|"structural"| node_translate
node_classify -->|"fallback"| node_real_rg
node_translate -->|"executes"| node_ast_grep
node_ast_grep -->|"returns"| node_normalize
node_real_rg -->|"returns"| node_normalize
node_normalize -->|"emits"| node_shim_bin
node_shim_bin -->|"records"| node_log_events
node_log_events -->|"persists"| node_stats_db
node_stats_db -->|"feeds"| node_compute_stats
node_compute_stats -->|"renders"| node_generate_report
node_report_tpl -->|"template"| node_generate_report
node_installer -->|"configures"| node_path_setup
node_installer -->|"updates"| node_claude_cfg
node_installer -->|"installs"| node_shim_bin
node_install_test -->|"validates"| node_installer
node_ci_install -->|"runs"| node_install_test
node_ci_release -->|"packages"| node_installer

click node_shim_bin "https://github.com/pkm-m84/shim/blob/main/src/main.rs"
click node_arg_parse "https://github.com/pkm-m84/shim/blob/main/src/main.rs"
click node_classify "https://github.com/pkm-m84/shim/blob/main/src/main.rs"
click node_translate "https://github.com/pkm-m84/shim/blob/main/src/main.rs"
click node_normalize "https://github.com/pkm-m84/shim/blob/main/src/main.rs"
click node_log_events "https://github.com/pkm-m84/shim/blob/main/src/main.rs"
click node_compute_stats "https://github.com/pkm-m84/shim/blob/main/src/main.rs"
click node_generate_report "https://github.com/pkm-m84/shim/blob/main/src/main.rs"
click node_report_tpl "https://github.com/pkm-m84/shim/blob/main/src/report.html"
click node_installer "https://github.com/pkm-m84/shim/blob/main/install.sh"
click node_install_test "https://github.com/pkm-m84/shim/blob/main/tests/install_test.sh"
click node_ci_install "https://github.com/pkm-m84/shim/blob/main/.github/workflows/install-test.yml"
click node_ci_release "https://github.com/pkm-m84/shim/blob/main/.github/workflows/release.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_path_rg,node_shim_bin,node_arg_parse,node_classify,node_translate,node_ast_grep,node_real_rg,node_normalize toneBlue
class node_stats_db,node_log_events,node_compute_stats,node_generate_report,node_report_tpl toneAmber
class node_installer,node_claude_cfg,node_path_setup toneMint
class node_install_test,node_ci_install,node_ci_release toneRose
```

## Components

### Runtime

Path-intercepting search execution

- **rg callers**: external clients · Terminal tools and agents that invoke rg on PATH
- [shim binary](https://github.com/pkm-m84/shim/blob/main/src/main.rs): rg shim · Drop-in replacement that intercepts rg invocations · `src/main.rs`
- [flag parse](https://github.com/pkm-m84/shim/blob/main/src/main.rs): CLI parser · Reads rg-like arguments and search input · `src/main.rs`
- [classify query](https://github.com/pkm-m84/shim/blob/main/src/main.rs): search classifier · Decides structural search vs plain text fallback · `src/main.rs`
- [translate pattern](https://github.com/pkm-m84/shim/blob/main/src/main.rs): pattern mapper · Converts rg-style queries and language flags to ast-grep form · `src/main.rs`
- **ast-grep**: structural backend · Executes code-structure search when classification allows it
- **ripgrep**: text backend · Absolute-path passthrough backend used to avoid recursion
- [normalize output](https://github.com/pkm-m84/shim/blob/main/src/main.rs): result adapter · Reformats backend output back into rg-compatible lines · `src/main.rs`

### Analytics

Local stats, reporting, and dashboard generation

- **stats.db**: sqlite store · Local SQLite database under ~/.smart-rg/ for events and comparisons
- [event log](https://github.com/pkm-m84/shim/blob/main/src/main.rs): usage recorder · Persists intercepted searches, reasons, languages, matches, and timestamps · `src/main.rs`
- [compute stats](https://github.com/pkm-m84/shim/blob/main/src/main.rs): aggregator · Aggregates event history for reporting and summaries · `src/main.rs`
- [generate report](https://github.com/pkm-m84/shim/blob/main/src/main.rs): html emitter · Produces an offline self-contained dashboard · `src/main.rs`
- [report template](https://github.com/pkm-m84/shim/blob/main/src/report.html): bundled html · Static HTML template compiled into the binary · `src/report.html`

### Install

Environment wiring and toolchain setup

- [install.sh](https://github.com/pkm-m84/shim/blob/main/install.sh): installer · Installs the shim, manages PATH, updates, uninstall, and Claude settings · `install.sh`
- **Claude settings**: env config · Optional ~/.claude/settings.json update to disable builtin ripgrep
- **PATH wiring**: environment setup · Places ~/.smart-rg first on PATH so rg resolves to the shim

### Delivery

CI and release automation

- [install test](https://github.com/pkm-m84/shim/blob/main/tests/install_test.sh): integration test · Checks installer behavior and settings merging · `tests/install_test.sh`
- [install CI](https://github.com/pkm-m84/shim/blob/main/.github/workflows/install-test.yml): workflow · GitHub Actions workflow for install testing · `.github/workflows/install-test.yml`
- [release CI](https://github.com/pkm-m84/shim/blob/main/.github/workflows/release.yml): workflow · GitHub Actions workflow for packaging and release automation · `.github/workflows/release.yml`

## Connections

- rg callers → shim binary: invokes · External callers hit the shim transparently through PATH
- shim binary → flag parse: parses · Reads rg-like flags and query text
- flag parse → classify query: classifies · Determines whether the search is structural enough for ast-grep
- classify query → translate pattern: structural · Structural queries are translated into ast-grep patterns
- classify query → ripgrep: fallback · Unsupported or unsafe queries fall back to real ripgrep
- translate pattern → ast-grep: executes · Translated query is sent to the structural backend
- ast-grep → normalize output: returns · ast-grep output is parsed and reshaped to rg format
- ripgrep → normalize output: returns · Passthrough backend output is normalized consistently
- normalize output → shim binary: emits · Final rg-compatible lines are written back to the caller
- shim binary → event log: records · Each intercepted search is logged locally as best-effort telemetry
- event log → stats.db: persists · Events and comparison data are stored in SQLite
- stats.db → compute stats: feeds · Historical records are aggregated for CLI stats
- compute stats → generate report: renders · Aggregated stats are turned into an HTML dashboard
- report template → generate report: template · Bundled HTML is compiled into the report output
- install.sh → PATH wiring: configures · Ensures the shim directory is first on PATH
- install.sh → Claude settings: updates · Optionally disables Claude's builtin ripgrep in favor of PATH rg
- install.sh → shim binary: installs · Places the binary into the user environment
- install test → install.sh: validates · Integration test exercises installer behavior
- install CI → install test: runs · Workflow executes installer-focused tests
- release CI → install.sh: packages · Release automation builds and ships the installable artifact