# coreruleset/machine-learning-integration-plugin architecture

> Architecture diagram of the GitHub repository coreruleset/machine-learning-integration-plugin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/coreruleset/machine-learning-integration-plugin
- Repository: https://github.com/coreruleset/machine-learning-integration-plugin

## Overview

##### Project Type and Purpose
- This is an open-source WAF (Web Application Firewall) plugin for the OWASP ModSecurity Core Rule Set (CRS) that adds a machine-learning decision layer.
- It consists of three logical parts:
- Plugin rules (.conf) loaded into CRS
- A Lua “client” script that fires on each request
- A Flask-based micro-service (ml_model_server) that computes an “anomaly score”

##### File Structure and Architectural Patterns
- Top-level directories:
- plugin/ : holds the ModSecurity rule files and the Lua script
- ml_model_server/ : the Python Flask application stub
- tests/ : YAML test data and a Python test harness
- Configuration files in plugin/ define thresholds, plugin mode, and the ml_server_url.
- The system follows a lightweight microservice pattern:
- The WAF (CRS) + Lua script form the entry point.
- The Flask server is a separate service, reachable via HTTP POST.
- No database is in play; the “model” is a stub within placeholder.py, but extensibility points exist under ml_model_server/saved_models.

##### Key Components
- External Client or Browser — initiates HTTP request
- CRS WAF Engine — loads machine-learning-before.conf and machine-learning-after.conf
- Lua Script (machine-learning-client.lua) — invoked by SecRuleScript
- HTTP Transport — POST from Lua -> Flask server
- Flask Server (ml_model_server/placeholder.py) — extracts parameters, invokes ML model stub or real model in saved_models
- ML Model Stub / Real Model — returns anomaly_score and status (0 or 1)
- CRS Decision Rules — examine inbound_ml_status to allow/block
- Optional: Test Harness (tests/) illustrating how to validate behavior

##### Relationships and Data Flow
- From WAF → Lua Script (label: SecRuleScript)
- Lua → Flask Server (label: HTTP POST with request metadata)
- Flask → ML Model (inside same process or separate sub-component)
- ML Model → Flask (returns anomaly_score → HTTP status code 200 or 401)
- Flask → Lua → WAF (inbound_ml_status=0|1)
- WAF → Client (block or pass)

##### Architectural Patterns and Design Principles
- Single Responsibility: separate WAF rules, request transformation (Lua), and ML scoring (Flask).
- Configuration-driven behavior: all thresholds and modes live in .conf files.
- Extensibility: ML model folder “saved_models” can be populated with real artifacts without changing core logic.
- Testability: standalone tests under tests/ use YAML fixtures and datavalidation_test.py.

##### Must-Have Elements
- Clear, human-readable labels on every component.
- Annotations for configuration files and thresholds to highlight extensibility points.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Client
    Client["External Client / Browser"]:::external

    %% CRS WAF Engine with rule files
    subgraph "CRS WAF Engine"
        direction TB
        beforeConf["machine-learning-before.conf"]:::config
        afterConf["machine-learning-after.conf"]:::config
        configConf["machine-learning-config.conf"]:::config
    end

    %% Lua Script
    Lua["Lua Script\nmachine-learning-client.lua"]:::script

    %% Flask ML Server
    subgraph "Flask ML Server"
        direction TB
        initPy["__init__.py"]:::service
        placeholder["placeholder.py"]:::service
        subgraph "ML Model"
            direction TB
            mlStub["Model Stub\n(saved_models)"]:::ml
        end
    end

    %% Test Harness (optional)
    subgraph "Test Harness" 
        direction TB
        classDef test fill:#fff,stroke:#666,stroke-dasharray: 5 5
        dvTest["datavalidation_test.py"]:::test
        y1["9516110.yaml"]:::test
        y2["9516120.yaml"]:::test
        y3["9516310.yaml"]:::test
        y4["9516320.yaml"]:::test
        initTest["__init__.py"]:::test
    end

    %% Data Flow
    Client -->|"HTTP Request"| beforeConf
    beforeConf --> afterConf
    afterConf --> configConf
    configConf --> Lua
    Lua -->|"SecRuleScript call"| initPy
    initPy --> placeholder
    placeholder -->|"invoke ML model"| mlStub
    mlStub -->|"anomaly_score,status"| placeholder
    placeholder -->|"HTTP Response\ninbound_ml_status"| Lua
    Lua -->|"return result"| Client

    %% Click Events
    click beforeConf "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/plugin/machine-learning-before.conf"
    click afterConf "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/plugin/machine-learning-after.conf"
    click configConf "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/plugin/machine-learning-config.conf"
    click Lua "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/plugin/machine-learning-client.lua"
    click initPy "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/ml_model_server/__init__.py"
    click placeholder "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/ml_model_server/placeholder.py"
    click dvTest "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/tests/datavalidation_test.py"
    click y1 "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/tests/9516110.yaml"
    click y2 "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/tests/9516120.yaml"
    click y3 "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/tests/9516310.yaml"
    click y4 "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/tests/9516320.yaml"
    click initTest "https://github.com/coreruleset/machine-learning-integration-plugin/blob/main/tests/__init__.py"

    %% Styles
    classDef external fill:#eee,stroke:#333,stroke-width:1px
    classDef config fill:#bbdefb,stroke:#0d47a1,stroke-width:1px
    classDef script fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px
    classDef service fill:#aed581,stroke:#558b2f,stroke-width:1px
    classDef ml fill:#ffcc80,stroke:#ef6c00,stroke-width:1px
    classDef test fill:#fafafa,stroke:#999,stroke-width:1px,stroke-dasharray:5 5
```
