How positron11/fuzzer works
Project Type and Purpose
• This is an open‑source grammar fuzzer framework. It provides two approaches to grammar‑based fuzzing:
a. Generic fuzzers written directly in C, which take a JSON grammar at runtime.
b. “Compiled” fuzzers: Python scripts that read a JSON grammar and emit specialized C fuzzer code, then compile it.
• Core use‑case: generate deterministic, depth‑limited random strings from context‑free grammars for testing parsers.
File‑Structure Analysis
• Top‑level directories:
• compiled/ → code‑generation pipeline for “compiled fuzzers”
• generic/ → C sources for a reusable, runtime‑configurable fuzzer
• grammars/ → JSON grammar definitions (http.json, JSON.json, tinyc.json)
• utilities/ → shared Python utilities (grammar cost computation, helpers)
• docs/ → LaTeX source and profiling outputs
• Build artifacts:
• Each of compiled/ and generic/ has its own Makefile and build/ folder.
• JSON grammars are inputs to both pipelines.
• Key components:
• Python “compilers” (iterative.py, recursive.py)
• Fuzzer driver in generic/src (fuzz-iter.c, fuzz-recr.c)
• Shared fuzzutils.h/.c for random expansion and depth control
• gramutils.py for grammar processing (cost, selection heuristics)
README Insights
• Two fuzzing strategies (iterative vs. recursive).
• Build invocation: make [iterative|recursive] grammar=[name].
• Outputs deterministic strings given a seed & max depth.
• Encourages reusability via generic C fuzzer or generation of specialized C code.
a. Main Components
1. Grammar Definitions
– JSON files in grammars/
2. Code‑Generation Pipeline (Compiled Fuzzers)
– Python scripts (compiled/compilers/iterative.py & recursive.py)
– utilities/gramutils.py & utilities/gramutils.py helpers
– Outputs C source (_*.c) into compiled/build/
– Makefile drives gcc into executables
3. Generic Fuzzer Runtime
– C driver (generic/src/fuzz-*.c)
– Shared utilities (generic/src/fuzzutils.c/.h, gstruct.h)
– Reads JSON grammar at runtime
4. Build System
– Makefiles (compiled/Makefile, generic/Makefile)
– Clean, compile, link steps
5. Executable Interface
– CLI: gi_fuzz-tinyc <seed> [max_depth]
6. Profiling & Documentation
– docs/ (LaTeX + CSV profiling data)
b. Component Interactions
• JSON grammar → Python generator → generated C code → gcc compiler → compiled fuzzer binary
• JSON grammar → generic fuzzer binary (linked with fuzzutils) → runtime grammar parser → fuzz generation
• Executable → STDOUT fuzzed strings → optionally piped into target under test → profiling scripts → docs/profiling
c. Architectural Patterns and Principles
• Separation of concerns: code generation vs. runtime fuzzing
• Pluggable grammars: JSON‑based grammar definitions decouple grammar from engine
• Deterministic execution via seeded RNG
• Two execution modes (iterative vs. recursive expansion) illustrating strategy pattern
d. Technologies and Frameworks
• Python 3 for code generation
• C (gcc) for fuzzer engine
• Makefiles for build automation
• JSON for grammar specification
• LaTeX and CSV for documentation and profiling