How intel/hyperscan works
Main Components (Top Level)
a. API Layer (Top)
b. Compiler Layer (Middle)
c. Runtime Layer (Bottom)
Compiler Components (Middle Layer)
• Parser System (/src/parser/)
• Component handling
• Pattern parsing
• UTF-8 validation
• NFAGraph System (/src/nfagraph/)
• Pattern analysis
• Graph optimization
• Component transformation
• Rose Engine (/src/rose/)
• Pattern matching engine
• Build system
• Runtime matcher
• FDR Engine (/src/fdr/)
• Fast pattern matching
• Literal matching optimization
Runtime Components (Bottom Layer)
Essential runtime elements:
• Database Management (/src/database.*)
• Scratch Space Management (/src/scratch.*)
• Stream Handling (/src/rose/stream.*)
• SOM (Start of Match) Runtime (/src/som/)
Supporting Systems (Side Panels)
Create side containers for:
• Utility Functions (/src/util/)
• Platform-Specific Optimizations (/src/util/simd_*)
• Memory Management (/src/alloc.*)
• Pattern compilation flow: API → Parser → NFAGraph → Rose/FDR
• Runtime matching flow: API → Database → Runtime Engines
• Stream data flow through the runtime system
Integration Points
Interfaces for:
• PCRE compatibility layer
• Chimera subsystem integration
• External tool integration
Performance Features
Specialized components:
• Hardware acceleration support
• Pattern optimization paths
• Memory management optimization
Use these visual elements:
• Rectangles: Major components
• Rounded rectangles: APIs and interfaces
• Hexagons: Processing engines
• Diamonds: Decision points
• Labels: Clear component names and versions
Memory management strategy
Keep the abstraction level high enough to show the overall architecture while including sufficient detail to understand the pattern matching pipeline and runtime execution flow.
This design reflects Hyperscan's nature as a high-performance regex matching library with distinct compilation and runtime phases, optimized for processing large numbers of patterns simultaneously across data streams.