How chataflndss/chatafl works
Project type and purpose
• ChatAFL Artifact is a research‐oriented protocol fuzzer that extends AFLNet by integrating large language models (LLMs) for three strategies:
1. Structure‐aware mutations using an LLM-derived grammar
2. Seed‐set enrichment via LLM-generated messages
3. Plateau-breaking LLM prompts during state stalls
• It is packaged as a ProfuzzBench extension (benchmark harness) and includes scripts for setup (Docker images), execution (run.sh), and analysis (analyze.sh).
File structure
• Top-level directories:
• ChatAFL-CL1, ChatAFL-CL2, ChatAFL – three fuzzer implementations (ablation variants and full)
• aflnet – modified AFLNet core
• benchmark – ProfuzzBench subset with text protocols and Lighttpd
• scripts: deps.sh, setup.sh, run.sh, analyze.sh, clean.sh
• Key source files in each fuzzer:
• afl-fuzz.c (entry point)
• chat-llm.c/.h (LLM helper)
• config.h, chat-llm.h (parameters)
• Dockerfiles in each fuzzer and benchmark subjects
• Output directories created at runtime:
• protocol-grammars, stall-interactions, queue (seed queue), benchmark/result-*
README insights
• Architecture is three-component: LLM grammar extractor, seed enricher, plateau breaker
• Integrates with ProfuzzBench via Docker images and run.sh orchestration
• .sh parses archives and generates coverage/state plots
a. Main components:
• ChatAFL Fuzzer Container (one per fuzzer variant)
– Core AFLNet engine (afl-fuzz)
– LLM integration module (chat-llm)
– Configuration (config.h, chat-llm.h)
• ProfuzzBench Orchestrator (run.sh + Docker Compose)
• Subject Under Test Container (e.g. pure-ftpd, kamailio, etc.)
• OpenAI LLM Service (external API endpoint)
• Analysis Module (analyze.sh + Python scripts with pandas/matplotlib)
• Benchmark Storage (volumes for results, archives, protocol-grammars)
b. Relationships and data flow:
1. run.sh invokes Docker to spin up:
– N ChatAFL containers connecting via network to N Subject containers
2. ChatAFL container startup (setup.sh):
– Build fuzzer binary (Dockerfile)
– Load config (EPSILON_CHOICE, thresholds)
3. Grammar Extraction Phase (setup_llm_grammars):
– chat-llm prompts → OpenAI API → returns protocol grammar → stored in protocol-grammars
4. Seed Enrichment Phase (get_seeds_with_message_types):
– chat-llm prompts original seeds → OpenAI API → enriched seeds added to AFL queue
5. Main Fuzz Loop (fuzz_one):
– AFLNet mutation + feedback → network messages to Subject
– On plateau: uninteresting_times triggers chat-llm prompts → OpenAI API → new test messages
6. Coverage/state logging: AFLNet instruments states, transitions → logged to result-archive
7. Container tear-down: archives volume persists testcases, logs, stall-interactions
8. .sh reads archives → CSVs → PNG plots → res_<subject>
c. Architectural patterns/design principles:
– Microservice style: each fuzzer-subject pair is an isolated container (horizontal scale)
– Modular LLM integration: chat-llm separated from AFLNet core
– Data‐driven: all artifacts (grammars, enriched seeds, stall logs) are persisted for analysis
– Configuration‐first: thresholds and retry counts in headers for easy tuning
d. Technologies/frameworks:
– C (AFLNet core, ChatAFL patches)
– Bash scripts (orchestration, benchmark harness)
– Docker (containerization)
– OpenAI GPT-3.5 API (LLM)
– Python pandas/matplotlib (analysis)
– ProfuzzBench conventions