How uber/piranha works
Project Type and Purpose
• PolyglotPiranha is a polyglot, rule-driven code transformation toolset, implemented primarily in Rust with language-agnostic support via Tree-Sitter grammars and exposed as both a CLI and a Python library.
• Its core use case is large-scale automated refactoring (e.g., stale feature-flag cleanup) by loading user-defined rules and executing tree-based analyses + transformations.
High-Level Architecture Components
a. Presentation Layers
• CLI Binary (“piranha”) – built from the Rust crates/core crate (entry point main.rs).
• Python API – a Python package (polyglot_piranha), using bindings to the Rust core (via pyproject.toml and polyglot_piranha.pyi).
• Tree-Sitter Playground – a static web app under playground/tree-sitter for grammar and query prototyping.
b. Core Engine
• crates/core – orchestrates rule graph execution, dataflow analyses, cleanup rules, model definitions, Tree-Sitter adapter, and output generation.
• crates/concrete-syntax – defines parsing logic (Pest grammar and Tree-Sitter adapter) and syntax-model layer for matching and rewriting.
• Cleanup Rules – language-specific TOML configs under crates/core/src/cleanup_rules/<lang>/* and rule graph edges.
c. Plugin System
• plugins/ – Python-based transformation plugins (e.g., spark_upgrade) that load custom rule sets and invoke the core engine.
d. Config & Test Artifacts
• Rule Definitions & Edges – .toml under crates/core/src/cleanup_rules and demo folders.
• Unit & Integration Tests – Rust tests in crates/*/unit_tests and Python tests under tests/test_piranha.py.
e. CI/CD
• GitHub Actions workflows in .github/workflows for building, testing, and releasing both Rust binary and Python package.
Component Interactions & Data Flow
1. User invokes CLI or Python API →
2. Core engine loads:
– Concrete syntax module to parse input code into AST.
– Rule Graph (seed rules + outgoing edges) from TOML or programmatic API.
3. Core performs matching, transformation passes, post-processing cleanup.
4. Transformed code is serialized and returned on stdout (CLI) or as Python objects.
5. Plugins can be registered to supply additional rules or drive end-to-end workflows.
Key Architectural Patterns
• Modular “crate” layering (separation of parsing vs analysis vs I/O).
• Plugin architecture (Python modules discoverable by core or invoked as separate runners).
• Configuration-driven transformations (TOML-backed rule definitions and edges).
• Dual-interface design (Rust for performance, Python for ease of use).