pkm-m84/shim

1. What this repository is

- This is a Rust CLI project that ships a drop-in replacement for rg (ripgrep) called shim / smart-rg.

- Its core architectural idea is: intercept search invocations locally, classify them, and either redirect to ast-grep for structural code search or fall back to real ripgrep for plain text search.

2. Primary runtime boundary

- The binary is intended to sit on the user’s PATH as rg, so external tools that shell out to rg invoke this project transparently.

- There is a second, tool-specific boundary for Claude Code: the installer also sets USE_BUILTIN_RIPGREP=0 in ~/.claude/settings.json so Claude uses PATH-based rg instead of its bundled binary.

3. Core execution flow

- src/main.rs is the main application entrypoint and contains the end-to-end search pipeline.

- The flow is: parse rg-like flags and query → classify pattern → translate when structural → execute ast-grep or real rg → normalize output back into rg-compatible lines → log the result locally.

4. Search classification and translation

- One subsystem decides whether a query is structural enough for ast-grep.

- Another subsystem converts rg-style search patterns into ast-grep patterns and maps rg language/type flags to ast-grep language selection.

- This is the key semantic boundary in the repo: “text search” vs “code-structure search.”

5. Execution backends

- The project has two execution paths:

- run_ast_grep() for structural searches, where ast-grep output is parsed and reformatted to match rg expectations.

- exec_real_rg() for passthrough searches, using the actual ripgrep binary to avoid infinite recursion through the shim itself.

- The README indicates the real rg path is resolved absolutely to avoid symlink loops.

6. Logging and local data store

- The project persists usage metrics to a local SQLite database under ~/.smart-rg/stats.db.

- Logged data is split into:

- events: each intercepted search, including agent, pattern, reason, language, matches, and timestamp.

- comparisons: rg-vs-ast-grep savings data, including file counts, estimated token savings, and cost savings.

- This is a local-only analytics subsystem; no network telemetry is part of the architecture.

7. Stats and reporting subsystem

- The CLI includes reporting commands beyond search interception: stats, report, prune, and reset.

- compute_stats() aggregates event history, while generate_report() emits a self-contained HTML dashboard.

- The HTML report template is compiled into the binary via include_str!, so the reporting UI is static and offline.

8. Installer and environment management

- install.sh is a major part of the repo’s operational architecture even though it is not Rust code.

- It manages the user environment by creating ~/.smart-rg/, placing the shim binary there, ensuring that directory is first on PATH, and optionally modifying Claude settings.

- It also supports dependency setup, update/migration, uninstall, purge, and a --check dry-run mode.

9. Compatibility and fallback strategy

- The project is designed to fail open: if classification fails, ast-grep is unavailable, flags are unsupported, or parsing breaks, it falls back to real ripgrep.

- That means the shim is a non-breaking compatibility layer, not a hard dependency gate.

- This behavior is essential to the architecture: correctness of search always outranks optimization.

10. Concurrency and persistence model

- Multiple short-lived rg invocations can write to the same SQLite DB concurrently.

- The README describes WAL mode and a busy timeout, which means logging is best-effort and should not block search execution.

- This keeps the search path fast while allowing shared local accounting across agents and shells.

11. Repository structure

- src/main.rs: the whole Rust CLI application and all runtime logic.

- src/report.html: bundled HTML template for the report output.

- docs/stats.html: likely static documentation or preview material for the report/dashboard.

- tests/install_test.sh: shell-based integration test focused on installer behavior and settings merging.

- .github/workflows/*: CI for install testing and release packaging.

12. Build and release shape

- The project is a Cargo-based Rust binary with a small release footprint and bundled SQLite via rusqlite.

- Release automation is handled in GitHub Actions, with a dedicated release workflow and an install/test workflow.

- The release model includes prebuilt macOS binaries, while Linux is currently source-build oriented according to the README.

13. Key external dependencies and their role

- clap drives the CLI contract and rg-like flag parsing.

- rusqlite provides embedded SQLite access without requiring a system SQLite install.

- serde/serde_json are used to consume ast-grep JSON output.

- ast-grep and ripgrep are the two external search engines that form the functional backend pair.

14. What an architecture diagram should show

- The central node is the shim binary in the user PATH.

- Upstream consumers are terminal agents and CLI tools invoking rg.

- Downstream branches are ast-grep for structural searches and real ripgrep for passthrough.

- A side channel writes to local SQLite, which feeds CLI stats and HTML report generation.

- The installer sits outside the runtime path but is critical for environment wiring and Claude configuration.