# aws-solutions/aws-waf-security-automations architecture

> Architecture diagram of the GitHub repository aws-solutions/aws-waf-security-automations, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aws-solutions/aws-waf-security-automations
- Repository: https://github.com/aws-solutions/aws-waf-security-automations

## Overview

##### Project type and purpose
- This is a serverless, microservices-based AWS solution (“Security Automations for AWS WAF”).
- It deploys multiple AWS Lambda functions (microservices) that integrate with AWS WAF, CloudFront/ALB, S3, Athena, CloudWatch Events, and third-party IP lists to automate threat detection and blocking.

##### File structure
- Top-level directories:
- deployment/ – CloudFormation templates, build and test scripts
- source/ – all Lambda microservices, helpers, custom resources, and shared libraries
- Under source/, you have:
- access_handler/ – Lambda for honeypot endpoint access (bad-bot detection)
- custom_resource/ – CloudFormation custom resources (provisioning support)
- helper/ – CFN dependency checks and auxiliary functions
- ip_retention_handler/ – scheduled IP set cleanup
- log_parser/ – processes access logs (S3/Athena) for suspicious traffic
- reputation_lists_parser/ – hourly ingestion of external IP reputation feeds
- timer/ – CloudWatch-triggered “sleep” function to pace resource creation
- lib/ – shared utilities (Boto3 wrappers, logging, metrics, DynamoDB/S3/SNS helpers)
- Configuration and test files show each microservice is packaged with Poetry (pyproject.toml) and covered by pytest.

##### README insights
- The solution groups protections into rule-group categories (AWS managed rules, manual IP lists, SQLi/XSS, HTTP flood, scanner & probe, IP reputation lists, bad bot honeypot).
- It triggers Lambda functions via CloudWatch Events (timers), S3 events (log arrivals), API Gateway (honeypot), and during CFN deployment (custom resources).
- CloudFormation templates and build scripts in deployment/ build assets, push to S3, then launch the stack.

##### Main components
- External / AWS services:
- Amazon CloudFront distribution or Application Load Balancer (entry point)
- AWS WAF Web ACL & IP Sets (rule groups, block/allow lists)
- Amazon S3 buckets:
– Access logs bucket (for CloudFront/ALB logs)
– Deployment asset buckets (global/regional)
- Amazon Athena / AWS Glue (for log queries and partitioning)
- Amazon CloudWatch Events / EventBridge (scheduled triggers)
- Third-party IP Reputation feeds (Spamhaus, ET, Tor)
- Microservice Lambdas:
- access_handler
- log_parser
- reputation_lists_parser
- ip_retention_handler
- custom_resource Lambdas (resource_manager, log_group_retention, operation handlers)
- timer function
- Build & deployment:
- Local CLI (Poetry, AWS CLI) → deployment/build-s3-dist.sh → S3 buckets → CloudFormation stack launch

##### Relationships & interactions
- Client traffic → CloudFront/ALB → inspected by AWS WAF Web ACL (rule groups)
- CloudFront/ALB logs → S3 → S3 event → log_parser Lambda → Analyze via Athena → update WAF IP Sets
- Scheduled CloudWatch Events →
- reputation_lists_parser → fetch external IP lists → update WAF IP Sets
- ip_retention_handler → remove expired IPs from allow/deny sets
- timer → pace custom_resource deployments
- API Gateway (or CloudFront function) → access_handler Lambda → add offending IP to WAF block list (honeypot)
- CloudFormation custom_resource Lambdas run during stack creation to configure buckets, WebACL, partitions, metrics, etc.
- All Lambdas utilize shared lib/ utilities for AWS calls, metrics collection, and CFN responses.

##### Architectural patterns & design principles
- Microservices: each Lambda focuses on a single domain of protection
- Event-driven: triggers via S3 events, schedules, API calls
- Infrastructure as code: CloudFormation with custom resources
- Shared library for common logic (DRY principle)
- Test-driven: pytest and coverage configurations per microservice

##### Final recommendations
- Essential AWS resources and Lambdas—avoid omitting any microservice.
- Both runtime flows (traffic, events) and deployment flows (build, deploy, configure).

## Diagram (Mermaid)

```mermaid
graph TD
    %% Entry Point and WAF
    subgraph "Entry Point"
        User(["User"]):::external
        CFALB(["CloudFront/ALB"]):::aws
        User --> CFALB
        CFALB --> WAF(["AWS WAF Web ACL"]):::aws
    end

    %% Rule Groups
    subgraph "WAF Rule Groups"
        RGA["A: AWS Managed Rules"]:::rule
        RGB["B/C: Manual IP Lists"]:::rule
        RGC["D: SQLi/XSS"]:::rule
        RGD["E: HTTP Flood"]:::rule
        RGE["F: Scanner & Probe"]:::rule
        RGF["G: IP Reputation Lists"]:::rule
        RGH["H: Bad Bot Honeypot"]:::rule
    end

    WAF --> RGA
    WAF --> RGB
    WAF --> RGC
    WAF --> RGD
    WAF --> RGE
    WAF --> RGF
    WAF --> RGH

    %% Data Ingestion and Processing
    subgraph "Data Sources"
        S3Logs(["Access Logs S3"]):::aws
        Athena(["Amazon Athena/Glue"]):::aws
        ThirdParty(["Third-Party IP Feeds"]):::external
    end

    S3Logs -->|"S3:ObjectCreated"| LP
    LP(["log_parser Lambda"]):::lambda -->|"Query via Athena"| Athena
    LP -->|"Update WAF IP Sets"| WAF

    ThirdParty -->|"Hourly fetch"| RP
    RP(["reputation_lists_parser Lambda"]):::lambda -->|"Update WAF IP Sets"| WAF

    %% Scheduled Lambdas
    subgraph "Scheduled Events"
        CW1(["CloudWatch Events: Schedule"]):::aws
        CW2(["CloudWatch Events: Schedule"]):::aws
    end

    CW1 --> IR
    CW2 --> TI
    IR(["ip_retention_handler Lambda"]):::lambda -->|"Update WAF IP Sets"| WAF
    TI(["timer Lambda"]):::lambda

    %% Honeypot API
    API(["API Gateway (Honeypot)"]):::aws --> AL
    AL(["access_handler Lambda"]):::lambda -->|"Add IP to block list"| WAF

    %% Deployment Pipeline
    subgraph "Deployment Pipeline"
        LocalCLI(["Local CLI (Poetry/AWS CLI)"]):::process
        BuildScript(["build-s3-dist.sh"]):::process
        DeployBucket(["Deployment S3 Buckets"]):::aws
        CFNStack(["CloudFormation Stack"]):::aws
        LocalCLI --> BuildScript
        BuildScript --> DeployBucket
        DeployBucket --> CFNStack
        CFNStack --> CR
        CR(["custom_resource Lambdas"]):::lambda -->|"Provision AWS resources"| AllAWS(["All AWS Resources"]):::aws
    end

    %% Click Events
    click AL "https://github.com/aws-solutions/aws-waf-security-automations/blob/main/source/access_handler/access_handler.py"
    click LP "https://github.com/aws-solutions/aws-waf-security-automations/blob/main/source/log_parser/lambda_log_parser.py"
    click RP "https://github.com/aws-solutions/aws-waf-security-automations/blob/main/source/reputation_lists_parser/reputation_lists.py"
    click IR "https://github.com/aws-solutions/aws-waf-security-automations/blob/main/source/ip_retention_handler/remove_expired_ip.py"
    click TI "https://github.com/aws-solutions/aws-waf-security-automations/blob/main/source/timer/timer.py"
    click CR "https://github.com/aws-solutions/aws-waf-security-automations/blob/main/source/custom_resource/custom_resource.py"

    %% Styles
    classDef aws fill:#ebf5ff,stroke:#3182bd,color:#1e3a8a
    classDef lambda fill:#fff7ed,stroke:#dd6b20,color:#c05621,stroke-width:2px
    classDef external fill:#f0fff4,stroke:#38a169,stroke-dasharray: 5 5,color:#2f855a
    classDef process fill:#f5f5f5,stroke:#718096,color:#2d3748
    classDef rule fill:#e6fffa,stroke:#38b2ac,color:#2c7a7b,stroke-width:2px
```
