How apostolos00tapsas/vectorscan works
Vectorscan is a high-performance regular-expression matching library, compatible with the Hyperscan 5.4 API and designed to match many expressions across block, vectored, or streaming data. The main workflow is to compile expressions into a database, provide that database and input to a scan API, and receive match reports. Internally, compilation parses expressions, builds and optimizes graphs, then selects matching engines; runtime support includes database handling and scratch state. The README documents the product workflow and portability goals; source excerpts are limited, so some internal links are omitted.
Main entry points: compiler.cpp Compiler orchestration · hs.h Scan API · hs_compile.h Compile API · dispatcher.c Runtime dispatcher
Components 17
Public API
- Compile API
src/hs_compile.h - Scan API
src/hs.h - Match reports
src/report.h
Compilation
- Expression parser
src/parser/Parser.h - Graph builder
src/nfagraph/ng_builder.cpp - Graph optimization
src/nfagraph/ng.cpp - Compiler orchestration
src/compiler/compiler.cpp - Report manager
src/util/report_manager.cpp
Matching engines
- Literal matcher
src/hwlm/hwlm.c - FDR engine
src/fdr/fdr.c - NFA engines
src/nfa/nfa_api_dispatch.c - Rose matcher
src/rose/match.c
Scan runtime
- Compiled database
src/database.c - Runtime dispatcher
src/dispatcher.c - Scratch state
src/scratch.c - Stream state
src/rose/stream.c
Other components
- Application
Connections 17
- Application → Compile API: submits expressions
- Compile API → Compiled database: produces database
src/hs_compile.h - Application → Scan API: provides input
- Scan API → Compiled database: uses database
- Scan API → Match reports: receives matches
- Compile API → Compiler orchestration: compiles patterns
- Compiler orchestration → Expression parser: parses expressions
- Compiler orchestration → Graph builder: builds graphs
- Graph builder → Graph optimization: optimizes graphs
- Compiler orchestration → Report manager: assigns reports
src/util/report_manager.cpp - Compiler orchestration → Literal matcher: configures matching
- Compiler orchestration → FDR engine: selects engine
- Compiler orchestration → NFA engines: selects engine
- Compiler orchestration → Rose matcher: builds matcher
- Scan API → Runtime dispatcher: dispatches scans
- Runtime dispatcher → Scratch state: uses scratch
- Runtime dispatcher → Stream state: maintains streams