How fabriziosalmi/caddy-adf works
Project type and purpose
• This is an open-source Caddy v2 middleware module written in Go, called “caddy-adf.”
• Its purpose is to act as a real-time, ML-enhanced Web Application Firewall (WAF) that plugs into the Caddy HTTP server.
File structure
• Top-level Go source files (admin.go, anomaly.go, caddyadf.go, caddyfile.go, history.go, ml.go, normalizer.go, sanitize.go) implement discrete subsystems:
• caddyadf.go: module registration and hooks into the Caddy middleware chain
• caddyfile.go: parsing of ml_waf directives in the Caddyfile
• anomaly.go: core scoring orchestration
• ml.go: ML score extraction using pre-trained.model and new_pre-trained.model
• history.go: time-windowed, sharded request history store per client IP
• normalizer.go: attribute normalization (linear/log)
• sanitize.go: redaction of sensitive headers and query parameters
• admin.go: HTTP admin endpoints (/ml_waf/update_model, /ml_waf/get_config)
• Configuration and scripts:
• Caddyfile: example integration with global and per-path settings
• go.mod / go.sum: module dependencies
• test.py, test.sh: functional and CI smoke tests
• CI metadata under .github/workflows (Go build/tests and Caddy integration).
README for architectural insights
• Describes a pipeline: Attribute Extraction → Normalization → Weighting → (Optionally) ML Scoring → History Correlation → Threshold Evaluation → Action (mark/block)
• Highlights dynamic thresholds, per-path configs, redaction, and admin API.
Main components and interactions
a. Components:
1. HTTP Client
2. Caddy Core HTTP Server
3. caddy-adf Middleware
– Attribute Extractor
– Normalizer
– Traditional Scoring Engine (weights + range checks)
– ML Scoring Engine (lookup in pre-trained.model)
– Request History Store (sharded in-memory cache)
– Redaction Engine
– Configuration Manager (global, per-path, dynamic thresholds)
4. File System (model files, config)
5. Admin HTTP API (/ml_waf)
6. Upstream Handler or Origin (the final “respond” or proxy)
b. Interactions / Data flow:
1. Client → Caddy Core → caddy-adf
2. caddy-adf: extract attributes → normalize → apply weights → (if enabled) ML lookup → combine scores
3. caddy-adf: query/update in-memory history store → calculate correlation score
4. caddy-adf: evaluate against anomaly_threshold and blocking_threshold (global or per-path)
5. caddy-adf: redact sensitive data → if blocked, return 403 + blocking headers → else pass to upstream
6. Admin tool or script → HTTP POST/GET to /ml_waf → update model file path or fetch current config
c. Architectural patterns and principles:
• Middleware pipeline (thin, composable stages)
• Sharded in-memory cache for concurrency and low latency
• Configuration-as-code via Caddyfile (declarative)
• Separation of concerns: scoring vs normalization vs persistence vs admin
d. Key technologies and libraries:
• Go modules and standard library (net/http, time, sync)
• Caddy v2 plugin SDK (module registration, Caddyfile parsing)
• xcaddy for custom build with the module
• Regular expressions for per-path matching and redaction
• Simple file-based ML model lookup