How noobone123/typeforge works
Project Type and Purpose
• TypeForge is a Ghidra extension/tool for binary reverse engineering, specifically aimed at recovering composite data types (structures, unions) from stripped binaries.
• It consists of two main phases:
• Program Analysis phase (implemented in Java as a Ghidra extension)
• LLM-assisted Refinement phase (orchestration in Python scripts)
File Structure
• Top-level:
• build.gradle, gradle/…: build configuration for the Java extension
• extension.properties: Ghidra extension metadata
• src/main/java/typeforge: core Java modules
• ghidra_scripts: Java entrypoint scripts for Ghidra (TypeForge.java, GroundTruth.java)
• scripts/: Python orchestration (TypeInference, GroundTruthExtractor, judge/double_elimination)
• data/: configuration files for languages and Sleigh definitions
• lib/: third-party JARs (Jackson, JGraphT, jHeaps)
• Patterns:
• Modular Java packages: analyzer (entrypoints), base (core analysis algorithms: dataflow, graph, node, parallel, passes), utils
• Python wrappers build pipelines around the Java extension
• Demo folder holds sample JSON outputs
README Insights
• Two-phase workflow clearly described
• Headless Ghidra invocation via analyzeHeadless + TypeForge.java
• Batch processing via Python scripts
• Demo JSON format, judge scripts for refinement
Components
a. Main Components:
1. Ghidra Host
• GUI Mode
• Headless Mode (analyzeHeadless)
2. Java Extension (TypeForge)
• Analyzer package (Generator, ReTyper, TypeAnalyzer)
• Base package:
• dataflow (AccessPoints, ConflictGraph, TypeFlowGraph, Solvers)
• graph (CallGraph, SDGraph)
• node (FunctionNode, DataTypeNode)
• parallel (PrepareFunctionNodeCallback)
• passes (SlidingWindowProcessor, Window)
• Utils (DecompilerHelper, Logging, etc.)
3. Data Store
• data/ languages and Sleigh definitions
• lib/ third-party JARs
4. Python Orchestration Layer
• TypeInference.py (calls Ghidra headless + TypeForge.java)
• GroundTruthExtractor.py
• scripts/judge/double_elimination.py (LLM interactions, refinement)
5. External Services
• LLM API (e.g., OpenAI GPT)
• GitHub (source, issue tracking)
6. Demo & Output
• demo/ JSON result files
b. Relationships & Interactions:
1. User provides stripped binary → Python TypeInference.py
2. TypeInference.py invokes Ghidra headless + TypeForge.java
3. Java extension loads data/ configs and lib jars, runs Program Analysis, emits JSON
4. JSON → Python judge script → LLM API calls for double elimination → refined JSON
5. Final JSON results saved to output directory / demo
6. GroundTruthExtractor similarly wraps GroundTruth.java for debug-info binaries
c. Architectural Patterns & Principles:
• Layered architecture:
• Data/config layer (data/, lib/)
• Core analysis layer (Java packages under src/main/java/typeforge)
• Orchestration layer (Python scripts)
• Pipeline / Workflow pattern: two discrete phases chained by JSON artifacts
• Plugin pattern: implemented as a Ghidra extension, loaded dynamically by Ghidra
• Separation of concerns: analysis engine vs. UI/CLI wrappers vs. refinement logic
d. Technologies & Frameworks:
• Java 11+ (Ghidra 11.0.3 compatibility)
• Ghidra extension APIs
• Gradle build system
• Python 3.x (scripts), PyYAML/config, requests (for LLM calls)
• Third-party libs: Jackson (JSON), JGraphT (graph algorithms), jHeaps (priority heaps)
• LLM service (e.g., OpenAI GPT-4)