# bradleyjkemp/sigma-go architecture

> Architecture diagram of the GitHub repository bradleyjkemp/sigma-go, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bradleyjkemp/sigma-go
- Repository: https://github.com/bradleyjkemp/sigma-go

## Overview

##### Project type and purpose
- This is an open‑source Go library (sigma‑go) for parsing and evaluating Sigma detection rules.
- It provides a parser (for Sigma rule syntax and config files) and an evaluation engine you embed in your own event‑processing pipeline.
- It is *not* a standalone service, but a code library with clear extension points for aggregation logic.

##### File‑structure overview and architectural patterns
- Top‑level Go files (ast.go, rule_parser.go, config_parser.go, condition_parser.go) implement a parser → AST pipeline.
- The “evaluator” directory provides the core engine that receives ASTs and applies them to events.
- Subpackages under evaluator:
- aggregators: in‑memory toy implementations of count, max, etc.
- modifiers: transformations applied to field values.
- fieldmappings and indexes: support mapping Sigma field names to event fields and fast lookups.
- evaluate_search.go and evaluate_aggregation.go: split between stateless pattern matching and stateful aggregations.
- internal/grammar holds the PEG or hand‑rolled grammar for aggregation_expression and search_expression parsing.
- internal/slidingstatistics implements time‑windowed counters (used by aggregation).
- testdata and fuzz.go / fuzz tests demonstrate how rules and events flow through parser and evaluator.

##### Key components and their interactions
a. Parser layer
- ConfigParser: reads platform-specific config files (e.g. sysmon).
- RuleParser: reads Sigma .rule.yml files.
- Output: an abstract syntax tree (AST) representation (ast.go).
b. Bundle builder
- rule_or_config.go and bundle.go group one or more ASTs into a RuleBundle.
c. Evaluator factory
- sigma.Evaluator(ruleBundle, options…): inject user‑supplied aggregation functions (CountFunc, MaxFunc, etc.).
d. Evaluation engine
- evaluate_search: applies simple field matches by iterating indexes and applying modifiers.
- evaluate_aggregation: uses slidingstatistics to maintain state across events and call aggregator functions.
- fieldmappings/indexes/modifiers work together for mapping and transforming event fields.
e. Runtime / user application
- Caller streams events into Evaluator.Matches(ctx, event) and reacts to match results (e.g. raise alerts).
f. Extension points
- Custom implementations of aggregation functions (CountFunc, MaxFunc).
- Custom modifiers or field mappings if necessary.

##### Architectural design principles
- Separation of concerns: parsing → AST → evaluation.
- Pluggability: aggregator and modifier functions are injected.
- Modular internal packages: grammar, sliding statistics, evaluator submodules.
- Test‑driven design: extensive unit tests and fuzz tests.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Sources
    subgraph "External Sources" 
        direction TB
        ES1["Audit Logs"]:::external
        ES2["Sysmon"]:::external
        ES3["Zeek"]:::external
        ES4["Kafka"]:::external
    end

    %% Parser Layer
    subgraph "Parser Layer" 
        direction TB
        CP["ConfigParser"]:::parser
        RP["RuleParser"]:::parser
        CCP["ConditionParser"]:::parser
        AST["AST Representation"]:::parser
    end

    %% Bundle Builder
    subgraph "Bundle Builder"
        direction TB
        ROC["RuleOrConfig"]:::bundle
        RB["RuleBundle"]:::bundle
    end

    %% Evaluator Factory
    subgraph "Evaluator Factory"
        direction TB
        OPT["Options"]:::factory
        EVF["sigma.Evaluator()"]:::factory
    end

    %% Evaluation Engine
    subgraph "Evaluation Engine"
        direction TB
        EVAL["Core Entry Point"]:::engine
        subgraph "Stateless Search" 
            direction TB
            SEARCH["evaluate_search"]:::engine
        end
        subgraph "Stateful Aggregation"
            direction TB
            AGG["evaluate_aggregation"]:::engine
            SS["SlidingStatistics"]:::internal
        end
        FM["FieldMappings"]:::engine
        IDX["Indexes"]:::engine
        MOD["Modifiers"]:::engine
        MEM["Aggregators (In-Memory)"]:::engine
    end

    %% Internal Grammar
    subgraph "Internal Grammar"
        direction TB
        G1["Search Expression"]:::internal
        G2["Aggregation Expression"]:::internal
    end

    %% Test & Fuzz
    subgraph "Testdata & Fuzz"
        direction TB
        TEST["testdata & fuzz.go"]:::test
    end

    %% CI/CD
    subgraph "CI/CD & Release"
        direction TB
        CI["GitHub Actions"]:::ci
        RC["GoReleaser"]:::ci
    end

    %% User Application
    subgraph "User Application"
        direction TB
        UA["Matches(ctx, event)"]:::app
        ALERT["Alert Logic"]:::app
    end

    %% Flow: Rules to Parser
    RFILES["Sigma Rule YAML"]:::external -->|reads| RP
    CFILES["Config Files"]:::external -->|reads| CP
    RP -->|"AST"| AST
    CP -->|"AST"| AST
    CCP -->|"AST Ext"| AST

    %% Build Bundle
    AST -->|"AST(s)"| ROC
    ROC --> RB

    %% Evaluator Factory
    RB --> EVF
    OPT --> EVF

    %% Engine Entry
    EVF --> EVAL

    %% Engine Modules
    EVAL --> FM
    EVAL --> IDX
    EVAL --> MOD
    EVAL --> SEARCH
    EVAL --> AGG
    AGG --> SS
    AGG --> MEM

    %% Internal Grammar to Engine
    G1 --> SEARCH
    G2 --> AGG

    %% Test harness
    TEST --> RP
    TEST --> EVAL

    %% CI/CD & Release
    CI -->|runs tests| TEST
    CI -->|deploy| RC

    %% User Application Flow
    ES1 --> UA
    ES2 --> UA
    ES3 --> UA
    ES4 --> UA
    UA -->|calls| EVAL
    SEARCH -->|boolean| ALERT
    AGG -->|boolean| ALERT

    %% Click Events
    click AST "https://github.com/bradleyjkemp/sigma-go/blob/main/ast.go"
    click RP "https://github.com/bradleyjkemp/sigma-go/blob/main/rule_parser.go"
    click CP "https://github.com/bradleyjkemp/sigma-go/blob/main/config_parser.go"
    click CCP "https://github.com/bradleyjkemp/sigma-go/blob/main/condition_parser.go"
    click ROC "https://github.com/bradleyjkemp/sigma-go/blob/main/rule_or_config.go"
    click RB "https://github.com/bradleyjkemp/sigma-go/blob/main/evaluator/bundle.go"
    click OPT "https://github.com/bradleyjkemp/sigma-go/blob/main/evaluator/options.go"
    click EVAL "https://github.com/bradleyjkemp/sigma-go/blob/main/evaluator/evaluate.go"
    click SEARCH "https://github.com/bradleyjkemp/sigma-go/blob/main/evaluator/evaluate_search.go"
    click AGG "https://github.com/bradleyjkemp/sigma-go/blob/main/evaluator/evaluate_aggregation.go"
    click FM "https://github.com/bradleyjkemp/sigma-go/blob/main/evaluator/fieldmappings.go"
    click IDX "https://github.com/bradleyjkemp/sigma-go/blob/main/evaluator/indexes.go"
    click MOD "https://github.com/bradleyjkemp/sigma-go/blob/main/evaluator/modifiers/modifiers.go"
    click MEM "https://github.com/bradleyjkemp/sigma-go/blob/main/evaluator/aggregators/memory.go"
    click TEST "https://github.com/bradleyjkemp/sigma-go/tree/main/evaluator/testdata"
    click TEST "https://github.com/bradleyjkemp/sigma-go/blob/main/fuzz.go"
    click G1 "https://github.com/bradleyjkemp/sigma-go/blob/main/internal/grammar/search_expression.go"
    click G2 "https://github.com/bradleyjkemp/sigma-go/blob/main/internal/grammar/aggregation_expression.go"
    click SS "https://github.com/bradleyjkemp/sigma-go/blob/main/internal/slidingstatistics/average.go"
    click SS "https://github.com/bradleyjkemp/sigma-go/blob/main/internal/slidingstatistics/count.go"
    click CI "https://github.com/bradleyjkemp/sigma-go/blob/main/.github/workflows/go.yml"
    click RC "https://github.com/bradleyjkemp/sigma-go/blob/main/.goreleaser.yml"

    %% Styling
    classDef external fill:#ffe4b5,stroke:#a0522d,stroke-width:1px
    classDef parser fill:#e0ffff,stroke:#20b2aa,stroke-width:1px
    classDef bundle fill:#fafad2,stroke:#bdb76b,stroke-width:1px
    classDef factory fill:#e6e6fa,stroke:#9370db,stroke-width:1px
    classDef engine fill:#f0fff0,stroke:#32cd32,stroke-width:1px
    classDef internal fill:#fff0f5,stroke:#db7093,stroke-width:1px
    classDef test fill:#f5f5f5,stroke:#696969,stroke-width:1px
    classDef ci fill:#f0f8ff,stroke:#4682b4,stroke-width:1px
    classDef app fill:#fffacd,stroke:#ffa500,stroke-width:1px
```
