How bradleyjkemp/sigma-go works
Project type and purpose
• This is an open‑source Go library (sigma‑go) for parsing and evaluating Sigma detection rules.
• It provides a parser (for Sigma rule syntax and config files) and an evaluation engine you embed in your own event‑processing pipeline.
• It is *not* a standalone service, but a code library with clear extension points for aggregation logic.
File‑structure overview and architectural patterns
• Top‑level Go files (ast.go, rule_parser.go, config_parser.go, condition_parser.go) implement a parser → AST pipeline.
• The “evaluator” directory provides the core engine that receives ASTs and applies them to events.
• Subpackages under evaluator:
• aggregators: in‑memory toy implementations of count, max, etc.
• modifiers: transformations applied to field values.
• fieldmappings and indexes: support mapping Sigma field names to event fields and fast lookups.
• evaluate_search.go and evaluate_aggregation.go: split between stateless pattern matching and stateful aggregations.
• internal/grammar holds the PEG or hand‑rolled grammar for aggregation_expression and search_expression parsing.
• internal/slidingstatistics implements time‑windowed counters (used by aggregation).
• testdata and fuzz.go / fuzz tests demonstrate how rules and events flow through parser and evaluator.
Key components and their interactions
a. Parser layer
• ConfigParser: reads platform-specific config files (e.g. sysmon).
• RuleParser: reads Sigma .rule.yml files.
• Output: an abstract syntax tree (AST) representation (ast.go).
b. Bundle builder
• rule_or_config.go and bundle.go group one or more ASTs into a RuleBundle.
c. Evaluator factory
• sigma.Evaluator(ruleBundle, options…): inject user‑supplied aggregation functions (CountFunc, MaxFunc, etc.).
d. Evaluation engine
• evaluate_search: applies simple field matches by iterating indexes and applying modifiers.
• evaluate_aggregation: uses slidingstatistics to maintain state across events and call aggregator functions.
• fieldmappings/indexes/modifiers work together for mapping and transforming event fields.
e. Runtime / user application
• Caller streams events into Evaluator.Matches(ctx, event) and reacts to match results (e.g. raise alerts).
f. Extension points
• Custom implementations of aggregation functions (CountFunc, MaxFunc).
• Custom modifiers or field mappings if necessary.
Architectural design principles
• Separation of concerns: parsing → AST → evaluation.
• Pluggability: aggregator and modifier functions are injected.
• Modular internal packages: grammar, sliding statistics, evaluator submodules.
• Test‑driven design: extensive unit tests and fuzz tests.