How clearbluejar/ghidriff works
Project type and purpose
• This is an open-source, Python-based command-line binary diffing engine named “ghidriff.”
• It orchestrates Ghidra headless analysis via JPype/PyGhidra, matches functions between two binaries, and emits JSON, Markdown, and optional HTML diffs.
• It also includes a static documentation/web site (Docusaurus) under www/.
File structure
• Top-level Python package (ghidriff/) holds core logic:
• __main__.py (CLI entrypoint)
• ghidra_diff_engine.py (base class with template workflow steps)
• simple_diff.py, structural_graph_diff.py, version_tracking_diff.py (engine implementations)
• correlators.py (function-matching algorithms), markdown.py (Markdown renderer), parser.py, utils.py
• tests/ validates the package.
• setup.py/setup.cfg define packaging and entrypoint.
• Dockerfile and .devcontainer/ support containerized dev and CI.
• www/ contains a standalone Docusaurus-based documentation site.
Key insights from the README
• Architecture: A template-method pattern (GhidraDiffEngine) that implements common Ghidra headless steps (project creation, import, analysis) and delegates find_matches() to pluggable implementations.
• Workflow:
1. CLI parses arguments →
2. Instantiate chosen Engine (Simple, Graph, VersionTracking) →
3. Launch Ghidra headless via JPype →
4. Import/analyze binaries →
5. Run correlators to match functions →
6. Post-process and render outputs (JSON, Markdown, HTML).
• External services: Symbol servers (MSDL, Chromium, Mozilla, Intel, NVIDIA, AMD) for PDB lookups.
• CLI Interface (Rectangle, e.g., blue)
• Engine Core (GhidraDiffEngine base class)
• Engine Implementations (three plugin classes)
• Correlators module (function-matching strategies)
• Ghidra Integration (JPype/PyGhidra bridge + headless Java process)
• Symbol Server Client (HTTP clients to external symbol servers)
• Output Renderers (Markdown, HTML side-by-side)
• Data Stores (JSON output directory, ghidra_projects, symbols cache)
• Testing Suite (tests/)
• Documentation Site (Docusaurus under www/)
1. User → runs ghidriff (CLI)
2. CLI → instantiates and configures Engine Core
3. Engine Core → spawns Ghidra headless (via JPype)
4. Ghidra → imports/analyzes binaries → writes to Ghidra Project folder
5. Engine Core → invokes Engine Implementation (find_matches)
6. Engine Implementation ↔ uses Correlators to compute function matches
7. Engine Core → calls Output Renderers → writes JSON, Markdown, optional HTML to disk
8. Engine Core → interacts with Symbol Server Client to fetch PDBs into local cache
9. Documentation Site is independent; it reads Markdown examples under www/diffs and www/docs for publication.
c. Architectural patterns and principles:
• Template Method: GhidraDiffEngine defines the diffing workflow skeleton.
• Plugin/Strategy: Diff algorithms (SimpleDiff, etc.) and correlators are swappable.
• Separation of Concerns: CLI parsing, engine workflow, algorithmic matching, and output rendering are each in their own modules.
• Infrastructure as Code: Devcontainer and GitHub Actions for CI, Docker for packaging.
d. Key technologies/frameworks:
• Python 3 with pyghidra and jpype
• Ghidra Headless (Java)
• Docusaurus (React/Node) for docs site
• Docker / Devcontainer / GitHub Actions for CI/CD
• HTTP symbol servers for PDB retrieval
Diagramming Recommendations
• Labels: Clearly name each module or external service.
• Core modules (blue rectangles)
• Plugins/strategies (light green)
• External systems (gray clouds)
• Documentation site (purple)
Capturing Essential Elements
• How a new diff engine plugs into the base class.
• Ghidra headless subprocess and its artifacts (project files).
• Symbol fetching as a side-flow into the analysis step.