# fuzzware-fuzzer/hoedur-experiments architecture

> Architecture diagram of the GitHub repository fuzzware-fuzzer/hoedur-experiments, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fuzzware-fuzzer/hoedur-experiments
- Repository: https://github.com/fuzzware-fuzzer/hoedur-experiments

## Overview

##### Project Type and Purpose
- This is neither a typical web app nor a library; it is a research experiment orchestration framework.
- Purpose: reproduce, distribute, and evaluate embedded‐firmware fuzzing experiments using Hoedur and Fuzzware on multiple hosts, collect results (coverage, bug discovery timings), and post‐process them into tables and plots.

##### Key Architectural Components
a. Local Orchestration Layer
- generate_host_run_config.py
- run_experiment.py
- reproduce_targets_and_configs.py
- install.py, install scripts under scripts/
- experiment-config directory (profiles.yml, host‐run‐configs, available_hosts.txt)
b. Execution Layer (Remote Hosts)
- SSH endpoints as defined in experiment-config/available_hosts.txt
- Docker Engine on each host
- Docker containers for:
- Hoedur fuzzing (hoedur-convert-fuzzware-config, fuzz-local.py, fuzz-coverage-list.py)
- Fuzzware fuzzing (fuzzware genconfig, custom Dockerfiles)
- AFL/AFL++ or libfuzzer backends invoked by fuzzers
c. Data Storage Layer
- NFS or shared fs (results directories: 01-bug-finding-ability…04-prev-unknown-vulns) on controller or central storage
- Targets directory (pre-built firmware and unpatched set)
- experiment-config/host-run-configs (YAML dispatch files)
d. Results Processing Layer
- compute_metrics.py
- scripts/eval_data_processing (Docker env, parsing.py, plotters, Makefile)
- scripts/summarize_bug_discovery_timings.py, check_data.py, check_install.py, clean_data.py
- sync_experiment_data.py, rsync

##### Relationships and Data Flow
1. Developer invokes `install.py` → sets up local environment (Python dependencies, Docker images).
2. User configures hosts in `experiment-config/available_hosts.txt`.
3. `generate_host_run_config.py` reads profiles.yml + available_hosts.txt → produces per‐host run‐configs (YAML).
4. (Optionally) `--upload` pushes run‐configs via SSH to remote hosts.
5. On each host: `run_experiment.py <host-config>.yml`
→ spawns Docker containers running Hoedur and Fuzzware fuzzers
→ fuzzers read target binaries from `targets/arm/...`, use hook scripts, corpus seeds
→ generate per-target results in results/fuzzing-runs, coverage, bug-reproducers.
6. After fuzzing completes, sync results back: `sync_experiment_data.py` or rsync
→ centralizes results under project’s results directories.
7. `compute_metrics.py` reads raw results → computes summary statistics (execs/sec, discovery times).
8. `scripts/eval_data_processing` Docker environment → runs parsing, plotting, PDF/table generation.
9. Final artifacts: .tex tables, .pdf plots, coverage charts, result summaries in each experiment directory.

##### Architectural Patterns and Principles
- Pipeline pattern: clearly defined stages (configuration → distribution → execution → collection → analysis).
- Distributed execution: work split across hosts, using SSH + Docker.
- Containerization: reproducible environments for fuzzers and analysis (Dockerfiles under scripts/fuzzware/env/).
- Separation of concerns:
- Orchestration vs. Execution vs. Analysis vs. Data Storage
- Auto-generation of configs (fuzzware ↔ hoedur conversion)
- Data immutability: targets and results directories are append‐only; patches and binary versions are versioned.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Swimlanes
    subgraph "Local Controller Machine"
        direction TB
        ORC_INSTALL["install.py"]:::orchestration
        ORC_GEN_CFG["generate_host_run_config.py"]:::orchestration
        ORC_REPRO["reproduce_targets_and_configs.py"]:::orchestration
        ORC_RUN_LOCAL["run_experiment.py"]:::orchestration
        subgraph "experiment-config" 
            direction TB
            CFG_PROFILES["profiles.yml"]:::orchestration
            CFG_HOSTS["available_hosts.txt"]:::orchestration
            CFG_RUNS["host-run-configs/"]:::orchestration
        end
    end

    subgraph "Remote Execution Hosts"
        direction TB
        SSH["SSH Endpoint"]:::external
        DOCKER_ENG["Docker Engine"]:::external
        subgraph "Hoedur Container" 
            direction TB
            HD_RUN["hoedur-run_experiment.py"]:::execution
            HD_BUILD["build_docker.sh"]:::execution
            HD_DOCKERFILE["hoedur-fuzzware.dockerfile"]:::execution
            HD_PATCHES["patches/"]:::execution
        end
        subgraph "Fuzzware Container"
            direction TB
            FW_RUN["run_experiment.py"]:::execution
            FW_BUILD["build_fuzzware_docker.sh"]:::execution
            FW_PREP["set_limits_and_prepare_afl.sh"]:::execution
            subgraph "Fuzzware Subdirs"
                direction TB
                FW_COV["coverage/"]:::execution
                FW_STATS["statistics/"]:::execution
                FW_BUG_TIM["bug-discovery-timings/"]:::execution
            end
        end
    end

    subgraph "Central Data Storage"
        direction TB
        STORAGE["NFS / Shared FS"]:::storage
        subgraph "targets"
            direction TB
            TGT_ARM_HOEDUR["arm/Hoedur/..."]:::storage
            TGT_ARM_FUZZ["arm/Fuzzware/..."]:::storage
            TGT_UNPATCH["arm/unpatched-est-data-set/..."]:::storage
        end
        CFG_RUNS_CENTRAL["experiment-config/host-run-configs/"]:::storage
        RES1["01-bug-finding-ability/"]:::storage
        RES2["02-coverage-est-data-set/"]:::storage
        RES3["03-advanced-mutations/"]:::storage
        RES4["04-prev-unknown-vulns/"]:::storage
    end

    subgraph "Results Processing & Analysis"
        direction TB
        METRICS["compute_metrics.py"]:::analysis
        subgraph "eval_data_processing"
            direction TB
            EVAL_DOCKER["env/Dockerfile & build.sh"]:::analysis
            PARSE["parsing.py"]:::analysis
            PLOT["plot_*.py"]:::analysis
            PLOTTING_DIR["plotting/"]:::analysis
        end
        SB_TIM["summarize_bug_discovery_timings.py"]:::analysis
        CHECK_DATA["check_data.py"]:::analysis
        CLEAN["clean_data.py"]:::analysis
        SYNC["sync_experiment_data.py"]:::analysis
    end

    %% Flows
    ORC_INSTALL -->|"sets up env & images"| ORC_GEN_CFG
    ORC_GEN_CFG -->|"reads profiles & hosts"| CFG_PROFILES
    ORC_GEN_CFG --> CFG_HOSTS
    ORC_GEN_CFG --> CFG_RUNS
    ORC_GEN_CFG -->|"produces run-configs"| CFG_RUNS
    ORC_GEN_CFG -->|"optional upload"| SSH
    SSH -->|"runs via SSH"| DOCKER_ENG
    DOCKER_ENG --> HD_RUN
    DOCKER_ENG --> FW_RUN
    HD_RUN -->|"reads targets"| TGT_UNPATCH
    FW_RUN -->|"reads targets"| TGT_UNPATCH
    HD_RUN -->|"writes results"| RES1
    FW_RUN --> RES1
    RES1 --> SYNC
    SYNC --> STORAGE
    STORAGE --> METRICS
    METRICS --> PARSE
    PARSE --> PLOT
    PLOT --> PLOTTING_DIR
    SB_TIM --> PLOTTING_DIR

    %% Click Events
    click ORC_INSTALL "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/install.py"
    click ORC_GEN_CFG "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/generate_host_run_config.py"
    click ORC_REPRO "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/reproduce_targets_and_configs.py"
    click ORC_RUN_LOCAL "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/run_experiment.py"
    click CFG_PROFILES "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/experiment-config/profiles.yml"
    click CFG_HOSTS "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/experiment-config/available_hosts.txt"
    click CFG_RUNS "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/experiment-config/host-run-configs/"
    click HD_RUN "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/hoedur/hoedur-run_experiment.py"
    click HD_BUILD "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/hoedur/build_docker.sh"
    click HD_DOCKERFILE "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/hoedur/hoedur-fuzzware.dockerfile"
    click HD_PATCHES "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/scripts/hoedur/patches/"
    click FW_RUN "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/fuzzware/run_experiment.py"
    click FW_BUILD "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/fuzzware/build_fuzzware_docker.sh"
    click FW_PREP "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/fuzzware/set_limits_and_prepare_afl.sh"
    click FW_COV "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/scripts/fuzzware/coverage/"
    click FW_STATS "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/scripts/fuzzware/statistics/"
    click FW_BUG_TIM "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/scripts/fuzzware/bug-discovery-timings/"
    click TGT_ARM_HOEDUR "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/targets/arm/Hoedur/"
    click TGT_ARM_FUZZ "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/targets/arm/Fuzzware/"
    click TGT_UNPATCH "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/targets/arm/unpatched-est-data-set/"
    click CFG_RUNS_CENTRAL "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/experiment-config/host-run-configs/"
    click RES1 "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/01-bug-finding-ability/"
    click RES2 "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/02-coverage-est-data-set/"
    click RES3 "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/03-advanced-mutations/"
    click RES4 "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/04-prev-unknown-vulns/"
    click METRICS "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/compute_metrics.py"
    click EVAL_DOCKER "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/scripts/eval_data_processing/env/Dockerfile"
    click PARSE "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/eval_data_processing/parsing.py"
    click PLOT "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/eval_data_processing/plot_*.py"
    click PLOTTING_DIR "https://github.com/fuzzware-fuzzer/hoedur-experiments/tree/main/scripts/eval_data_processing/plotting/"
    click SB_TIM "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/summarize_bug_discovery_timings.py"
    click CHECK_DATA "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/check_data.py"
    click CLEAN "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/scripts/clean_data.py"
    click SYNC "https://github.com/fuzzware-fuzzer/hoedur-experiments/blob/main/sync_experiment_data.py"

    %% Styles
    classDef orchestration fill:#87CEFA,stroke:#000,stroke-width:1px
    classDef execution fill:#90EE90,stroke:#000,stroke-width:1px
    classDef storage fill:#FFA500,stroke:#000,stroke-width:1px
    classDef analysis fill:#DDA0DD,stroke:#000,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px
```
