How 0ndt/vectorscan-rs works
Project Type and Purpose
• This is a Rust “sys”-style crate plus a safe wrapper (vectorscan-rs) that bundles, builds, and links a vendored copy of the Vectorscan (Hyperscan fork) C/C++ library.
• It exposes minimal FFI bindings from Vectorscan’s block-based regex matching API into Rust, via bindgen when requested.
High-Level Architecture
• Front-end: Rust crate (Cargo, src/lib.rs, src/bindings.rs)
• Build system: build.rs (Rust build script) + CMake (in vectorscan-sys)
• Vendored C/C++ sources: vectorscan-sys/…
• External tools/dependencies: CMake, Clang (for bindgen), bindgen, Docker (optional)
Main Components & Relationships
[Developer]
↓ invokes
[Cargo Build]
↓ runs
[build.rs]
├─(1) spawns CMake → compiles vectorscan-sys sources → produces libvectorscan.a / .so
└─(2, if “gen” feature) calls bindgen (Clang) on hs.h → generates src/bindings.rs
↓ then
[Rust Compiler]
compiles src/lib.rs + bindings.rs → links against libvectorscan
↓ outputs
[vectorscan-rs Crate]
Detailed Components
• Rust Layer
• Cargo.toml – crate metadata, features (e.g., “gen”)
• build.rs – orchestrates CMake and bindgen
• src/bindings.rs – raw unsafe FFI signatures (auto-generated)
• src/lib.rs – safe wrapper functions/types around FFI
• Vendored Layer (vectorscan-sys)
• CMakeLists.txt + cmake/… – cross-platform build
• src/… – hundreds of C/C++ source files implementing regex engines (chimera, fdr, nfa, rose, etc.)
• /… – headers for public API
• tools/, examples/, unit/… – benchmarks, tests, fuzzers (disabled by default)
• External Tools
• CMake – configure & build the C/C++ library
• Clang & bindgen – generate Rust FFI bindings when “gen” is enabled
Key Interactions & Data Flow
• Cargo → build.rs: “Run build script”
• build.rs → CMake: “Configure & build vendored library”
• build.rs → bindgen (optional): “Parse hs.h → emit Rust declarations”
• Rust code → libvectorscan.a: “Link FFI”
• Rust public API → application code: “Call regex matching functions”
Architectural Patterns & Principles
• Layered Build: Rust layer on top, build layer in middle, C/C++ layer at bottom
• FFI Boundary: clearly separated via bindings.rs and lib.rs
• Feature-gated code generation (bindgen)
• Vendoring: avoids external C/C++ version drift
• Build-time code gen and linking
• C/C++ components (vectorscan-sys, CMake) in another
• External tools (bindgen, Clang, Docker) in a third
Essential Elements
• Clear labels for: Cargo, build.rs, CMake (vectorscan-sys), bindgen, src/lib.rs, src/bindings.rs, libvectorscan.a