How fuchuanpu/hypervision works
Project Type & Purpose
• HyperVision is a standalone detection pipeline (mostly C++ with Python for post-analysis).
• It ingests packet captures (pcap), constructs flow interaction graphs, runs graph-based detection algorithms, and produces evaluation metrics.
File Structure
• Top-level modules (each a CMake subproject):
• packet_parse – raw PCAP parsing
• flow_construct – turn parsed packets into interaction flows
• dataset_construct – build basic datasets for training/testing
• graph_analyze – build & analyze flow interaction graphs (core detection logic)
• Supporting directories:
• configuration – JSON templates for various scan/malware profiles
• env – install scripts for dependencies (Z3, PCAP libs, etc.)
• script – orchestration scripts (build, expand dataset, run batches)
• main.cpp ties everything together under a single binary.
README
• Core pipeline: build & run via CMake and shell scripts, then post-process with Python.
• Uses JSON configs to parameterize scan types (bruteforce, lrscan, malware, misc, web).
• Output logs under build/log; result_analyze reads these to compute AU_ROC.
Main Components & Interactions
a. Data Ingestion & Configuration
• External dataset tarball → unpack → JSON config files drive scan profiles
• env/install_all.sh sets up system libraries
b. Packet Parser (packet_parse)
• Input: .pcap files
• Output: packet_info structures → fed to flow constructor
c. Flow Constructor (flow_construct)
• Input: parsed packets
• Output: raw flows & basic datasets (flows grouped by 5‐tuple)
d. Dataset Constructor (dataset_construct)
• Uses flow outputs + configuration → constructs feature vectors or labeled data
e. Graph Analyzer (graph_analyze)
• Builds flow interaction graph from flow_construct outputs
• Executes graph analysis routines (basic/short/long) → detection score per flow
f. Result Analyzer (result_analyze)
• Python batch_analyzer.py & basic_analyzer.py
• Reads log files, computes AU_ROC & other metrics, produces graphs/tables
g. Orchestration Scripts (script/)
• rebuild.sh → cleans + CMake build + compile
• expand.sh → expands dataset into pcap splits or directories
• run_all_*.sh → run detection across all JSON profiles
Architectural Patterns & Principles
• Modular pipeline: each phase is a self-contained component, built/Edit/Test independently
• Configuration-driven: JSON files allow adding new attack profiles without code changes
• Command-line orchestration: Bash scripts glue modules end-to-end
• Separation of concerns: C++ for performance-critical parsing & graph algorithms, Python for flexible analysis/plotting
Essential Details to Capture
• CMakeLists.txt in each subdirectory enabling modular builds
• Dependency on libpcap (pcap_parser), Z3 (if used), and Python3 for result_analyze
• The five categories of configuration profiles (bruteforce, lrscan, malware, misc, web) feeding into the scan runs
• The scripts folder as the single entrypoint for users to rebuild, run, and analyze