# yamato-security/hayabusa architecture

> Architecture diagram of the GitHub repository yamato-security/hayabusa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yamato-security/hayabusa
- Repository: https://github.com/yamato-security/hayabusa

## Overview

##### Project Type and Purpose
- Hayabusa is a command-line tool designed for rapid forensic analysis and threat hunting via Windows event logs. It reads Windows evtx (or JSON) files, applies detection rules (written in YAML with Sigma compatibility), enriches events (for example with GeoIP data), aggregates timeline data, and finally exports analysis results as CSV, JSON, or HTML. Its design supports both live system analysis and offline batch processing, with multi-threading for improved performance.

##### Break Down the File Structure into Major Components
- Top-level directories such as “src”, “config”, “rules”, “doc”, and “test_files” indicate clear separation of concerns.
- The “src” directory holds the core Rust code (with submodules like “detections”, “options”, “timeline”, “notify”, etc.) responsible for:
– Parsing logs and handling file I/O.
– Running detection analysis and rule matching.
– Computing and outputting various metrics and timelines.
- The “rules” directory (including a submodule via Git) contains the detection rules (YAML files) and associated configuration (for adjusting rule levels, mapping fields, expand placeholders, etc.).
- Other directories like “doc”, “logs”, and “art” support documentation, sample data, and user art – but the core architecture is in src, rules, and config.

##### Main Architectural Components
- Command-Line Interface (CLI) & Input Manager:
– Parses arguments (including subcommands like “csv-timeline”, “json-timeline”, “computer-metrics”, “level-tuning”, etc.) and dispatches the workload.
- File Loader & Log Parser:
– Handles various input sources: reading evtx files, JSON log files, live analysis from Windows event log folders.
– Employs the evtx parsing library (or its fork) with support for recovering and carving event records.
- Multi-threaded Processing Core:
– Orchestrates parallel processing of files.
– Ensures each file is fed into the processing engine for performance.
- Detection Engine & Rule Matcher:
– Loads detection rules from the “rules” directory (including Sigma-converted rules).
– Applies rule matching to parsed events, including signature matching and condition filtering.
– Manages special processing features (like “expand list” for dynamic mapping, field data mapping, and level tuning adjustments from the configuration).
- Enrichment & Aggregation Module:
– Optionally integrates with GeoIP data (using supplied MaxMind databases) to add ASN, city, and country details.
– Aggregates metrics (such as computer metrics, event ID metrics, logon summaries, etc.) and prepares timeline data.
- Output & Reporting Module:
– Exports the forensic timeline in multiple formats (CSV, JSON/JSONL, or HTML reports).
– Applies output profiles (minimal, standard, verbose, super-verbose, Timesketch-compatible, etc.) that determine which fields and aliases are included.
- Configuration & Customization Layer:
– Loads global configuration (from “config” and “rules/config” directories) for field mappings, level tuning, critical systems, and abbreviation rules.
– Offers commands to update rules, tune alert levels, and set default output profiles.
- Logging, Error Handling, and Notification System:
– Manages progress indicators (progress bars), logs processing errors, and even triggers notifications when needed.

##### Relationships and Data Flow
- Start at the CLI layer where the user selects an analysis mode.
- The Input Manager then directs files (or live sources) to the Log Parser.
- Parsed events are dispatched by the multi-threaded core to the Detection Engine.
- The Detection Engine consults loaded YAML detection rules to flag “alerts” versus normal “informational” events.
- Enrichment (such as GeoIP lookup) is then optionally applied.
- Aggregated detection events and metrics are assembled by the Aggregation Module.
- Finally, the Output Module formats the timeline or metrics into the chosen file (CSV, JSON, HTML) using pre-defined output profiles.
- Throughout, configuration files (from “config” and “rules/config”) modify behavior at almost every stage.

##### Important Architectural Patterns & Technologies
- Modular design: Separate concerns between input, processing, rule matching, enrichment, and output.
- Plugin-like rule processing: The dynamic loading and matching of YAML rules (including Sigma rule conversion) supports high flexibility.
- Multi-threading: Distributed processing of evtx files to improve speed.
- Configurability: With external configuration files for tuning, field mapping, and output formatting.
- Command pattern: The variety of subcommands for metrics, search, timeline, and level tuning each correspond to discrete operations within the tool.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Layer
    subgraph "CLI Layer"
        CLI_Main["CLI Interface (Main)"]:::cli
        CLI_Options["CLI Options (Subcommands)"]:::cli
        CLI_Main -->|"subcommands"| CLI_Options
    end

    %% Processing Pipeline
    subgraph "Processing Pipeline"
        YAML_Loader["File Loader (YAML)"]:::loader
        YAML_Expander["Configuration Expander (YAML)"]:::loader
        Processing_Core["Processing Core (Multi-threaded)"]:::core
        Detection_Engine["Detection Engine (Rule Matcher)"]:::engine
        GeoIP_Enrichment["GeoIP Enrichment"]:::enrichment
        Timeline_Aggregator["Timeline Aggregation"]:::enrichment

        CLI_Options -->|"provides_input_files"| YAML_Loader
        YAML_Loader -->|"reads_file_data"| YAML_Expander
        YAML_Expander -->|"prepares_configuration"| Processing_Core
        Processing_Core -->|"distributes_events"| Detection_Engine
        Detection_Engine -->|"applies_detection_rules"| GeoIP_Enrichment
        GeoIP_Enrichment -->|"aggregates_timeline"| Timeline_Aggregator
    end

    %% Output Layer
    subgraph "Output Layer"
        HTML_Report["HTML Report Output"]:::output
        Timeline_Output["Timeline Export (CSV/JSON)"]:::output

        Timeline_Aggregator -->|"exports_HTML_report"| HTML_Report
        Timeline_Aggregator -->|"exports_timeline_data"| Timeline_Output
    end

    %% Configuration & Support
    subgraph "Configuration & Support"
        Global_Config["Global Config Files"]:::config
        Test_Config["Test Config Files"]:::config
        Logger["Error Logging"]:::log
        Notifier["Notification System"]:::log

        Global_Config -->|"supplies_configurations"| YAML_Expander
        Global_Config -->|"tunes_detection"| Detection_Engine
        Global_Config -->|"adjusts_aggregation"| Timeline_Aggregator
        Global_Config -->|"formats_output"| HTML_Report
        Test_Config -->|"tests_configurations"| Global_Config

        Logger -->|"logs_errors"| Processing_Core
        Logger -->|"logs_errors"| Detection_Engine
        Logger -->|"logs_errors"| HTML_Report

        Notifier -->|"sends_notifications"| Processing_Core
        Notifier -->|"sends_notifications"| HTML_Report
    end

    %% Styles
    classDef cli fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
    classDef loader fill:#C8E6C9,stroke:#388E3C,stroke-width:2px;
    classDef core fill:#BBDEFB,stroke:#1976D2,stroke-width:2px;
    classDef engine fill:#E1BEE7,stroke:#8E24AA,stroke-width:2px;
    classDef enrichment fill:#FFF9C4,stroke:#FBC02D,stroke-width:2px;
    classDef output fill:#FFCDD2,stroke:#C62828,stroke-width:2px;
    classDef config fill:#D1C4E9,stroke:#5E35B1,stroke-width:2px;
    classDef log fill:#CFD8DC,stroke:#546E7A,stroke-width:2px;

    %% Click Events
    click CLI_Main "https://github.com/yamato-security/hayabusa/blob/main/src/main.rs"
    click CLI_Options "https://github.com/yamato-security/hayabusa/tree/main/src/options"
    click YAML_Loader "https://github.com/yamato-security/hayabusa/blob/main/src/yaml.rs"
    click YAML_Expander "https://github.com/yamato-security/hayabusa/blob/main/src/yaml_expand.rs"
    click Processing_Core "https://github.com/yamato-security/hayabusa/blob/main/src/lib.rs"
    click Detection_Engine "https://github.com/yamato-security/hayabusa/tree/main/src/detections"
    click GeoIP_Enrichment "https://github.com/yamato-security/hayabusa/blob/main/src/options/geoip_search.rs"
    click Timeline_Aggregator "https://github.com/yamato-security/hayabusa/tree/main/src/timeline"
    click HTML_Report "https://github.com/yamato-security/hayabusa/blob/main/src/options/htmlreport.rs"
    click Timeline_Output "https://github.com/yamato-security/hayabusa/blob/main/src/timeline/timelines.rs"
    click Global_Config "https://github.com/yamato-security/hayabusa/tree/main/config"
    click Test_Config "https://github.com/yamato-security/hayabusa/tree/main/test_files/config"
    click Logger "https://github.com/yamato-security/hayabusa/tree/main/logs"
    click Notifier "https://github.com/yamato-security/hayabusa/tree/main/src/notify"
```
