How thtcsec/gcp-serverless-soar works
1. What this repo is
- A GCP-native, serverless SOAR platform for detecting and responding to cloud security incidents.
- The core design is event-driven: detections arrive, are normalized/correlated/scored, then routed into an incident-specific response playbook.
2. Main ingress paths
- Findings and alerts come from Security Command Center, Cloud Audit Logs, and other GCP telemetry sources.
- The repo’s architecture assumes GCP event routing through Eventarc / Pub/Sub into Python handlers, with src/handlers.py, src/main.py, and src/queue_processor.py acting as the primary processing entry points.
3. Event processing pipeline
- src/core/event_normalizer.py converts heterogeneous alerts into a shared incident shape.
- src/core/correlator.py groups related events into a single incident view.
- src/integrations/scoring.py applies risk scoring, and src/integrations/anomaly_detector.py / src/ml/* add behavioral and ML-based risk signals.
- This creates the “decision layer” before any remediation action is taken.
4. Decisioning and playbook selection
- src/playbooks/registry.py maps normalized incidents to response playbooks.
- The repo includes dedicated playbooks for GCE containment, ransomware, service account compromise, storage exfiltration, GKE pod isolation, Cloud SQL compromise, and API Gateway abuse.
- src/core/auto_remediation.py and the playbooks together form the automation boundary: they decide whether to act, request approval, or defer.
5. Response execution on GCP
- The actual remediation work is split between Python orchestration and GCP API calls, primarily through src/clients/gcp.py and workflow helpers in src/workflow/*.
- Actions include isolating instances, taking snapshots, terminating instances, and enforcing containment controls.
- src/core/process_containment.py covers host-level containment on Compute Engine when the response needs process/file-level action rather than just network isolation.
6. Long-running worker boundary
- The architecture separates short-lived orchestration from long-running investigation work.
- containers/isolation-worker and containers/forensics-worker are Cloud Run-style containerized workers used for heavier operations such as isolation workflows and forensic scanning.
- This boundary matters because the main event handler stays lightweight while workers handle slower tasks that do not fit well into a simple function invocation.
7. Forensics and evidence preservation
- The platform preserves evidence before fully stopping or quarantining a resource.
- src/core/report_generator.py, src/core/audit_logger.py, and the forensics worker are used to produce incident artifacts and audit trails.
- The response pattern is intentionally “snapshot first, stop second” for GCE-style incidents.
8. Cross-project and organization-level response
- src/cross_project/cross_project_responder.py and src/core/config.py support environments where security operations span multiple GCP projects.
- The Terraform layout also reflects this with modules for security, queues, workflows, containers, and integrations, indicating the platform is meant to be centrally operated across projects, not just inside one app project.
9. External integrations
- src/integrations/slack_notifier.py, jira.py, jira_manager.py, and siem_forwarder.py handle outbound notifications and ticketing.
- src/integrations/ai_summarizer.py adds GenAI-based incident summaries.
- src/integrations/intel.py brings in threat-intel enrichment, which feeds scoring and alert quality.
- These integrations sit on the outbound side of the architecture, after decisioning and response planning.
10. Observability, audit, and compliance
- src/core/logger.py, src/core/metrics.py, and src/core/audit_logger.py provide runtime observability and immutable action logging.
- The Terraform monitoring module and terraform/monitoring.tf indicate dashboards and alerting are first-class parts of the deployment, not an afterthought.
11. Infrastructure as Code and deployment shape
- The platform is deployed with Terraform, not manual console setup.
- terraform/modules/workflows, queues, containers, security, integrations, and monitoring define reusable subsystems; terraform/environments/dev and prod show environment-specific wiring.
- This means the architecture diagram should include both the runtime flow and the IaC layer that provisions it.
12. Testing and attack simulation
- attack_simulation/ is a built-in red-team harness used to generate realistic incidents such as miner/ransomware, storage exfiltration, and service account compromise.
- It is containerized and is meant to drive the same detection/response pipeline the production system uses, making it useful as a “source” in the architecture diagram for validation.
13. Overall data flow to diagram
- Source telemetry enters GCP detection services.
- Events are routed into Pub/Sub/Eventarc.
- Python orchestration normalizes, correlates, scores, and enriches.
- A playbook is selected and executed through GCP APIs and workflows.
- Long-running evidence collection and isolation move to Cloud Run workers.
- Results are pushed out to Slack/Jira/SIEM and recorded in audit logs.
14. Key architectural boundaries
- Detection vs. response: SCC/Audit Logs detect; Python services decide and act.
- Orchestration vs. heavy work: handlers/workflows coordinate; containers do forensic or isolation jobs.
- Internal control plane vs. external notifications: remediation stays in GCP; Slack/Jira/SIEM are outbound only.
- Policy/decisioning vs. execution: scoring/correlation choose the action; clients/workers perform it.
Main entry points: gcp.py GCP client · auto_remediation.py Auto-remediation · gce_containment.py GCE containment · scoring.py Scoring
Components 31
Runtime
Event-driven Python control plane
- Handlers · Primary event entrypoints
src/handlers.py - Main · Function bootstrap and wiring
src/main.py - Queue processor · Async queue processing boundary
src/queue_processor.py - Normalizer · Converts alerts into a common incident shape
src/core/event_normalizer.py - Correlator · Groups related events into incidents
src/core/correlator.py - Scoring · Applies risk scoring
src/integrations/scoring.py - Anomaly detector · Behavioral anomaly signals
src/integrations/anomaly_detector.py - Threat classifier · Incident classification model
src/ml/threat_classifier.py
Response
Decisioning and remediation playbooks
- Playbook registry · Maps incidents to response playbooks
src/playbooks/registry.py - Auto-remediation · Approval and execution boundary
src/core/auto_remediation.py - GCE containment · Compute Engine containment response
src/playbooks/gce_containment.py - Ransomware · Ransomware response playbook
src/playbooks/ransomware_response.py - SA compromise · Service account compromise response
src/playbooks/sa_compromise.py - Storage exfiltration · Bucket exfiltration response
src/playbooks/storage_exfiltration.py - GCP client · GCP API execution layer
src/clients/gcp.py - Snapshot · Preserve evidence first
src/workflow/create_snapshot.py - Isolate · Network or host isolation
src/workflow/isolate_instance.py - Terminate · Stop or remove compromised resources
src/workflow/terminate_instance.py - Process containment · Host-level process/file containment
src/core/process_containment.py
Integrations
Outbound and enrichment services
- Audit log · Immutable action logging
src/core/audit_logger.py - Slack · Chat notifications
src/integrations/slack_notifier.py - Jira · Ticket creation and management
src/integrations/jira_manager.py - SIEM · Outbound security event forwarding
src/integrations/siem_forwarder.py - AI summary · Incident summarization
src/integrations/ai_summarizer.py
Workers
Long-running Cloud Run jobs
- Forensics worker · Long-running evidence collection
containers/forensics-worker/src/forensics_worker.py - Isolation worker · Long-running isolation jobs
containers/isolation-worker/src/isolation_worker.py
Terraform
Provisioning and deployment layout
- Workflows · Workflow provisioning
terraform/modules/workflows/main.tf - Queues · Queue provisioning
terraform/modules/queues/main.tf - Containers · Worker container provisioning
terraform/modules/containers/main.tf - Security · Security and IAM provisioning
terraform/modules/security/main.tf
Simulation
Attack generation and validation
- Attack sim · Red-team incident generator
attack_simulation/scripts/simulate_event.py
Connections 35
- Attack sim → Handlers: injects incidents · Generates realistic detections into the same pipeline
- Handlers → Normalizer: normalize · Transforms heterogeneous events into a shared incident shape
- Queue processor → Correlator: correlate · Processes queued events into incident groups
- Normalizer → Correlator: group · Feeds normalized incidents into correlation
- Correlator → Scoring: score · Computes risk and priority
- Scoring → Anomaly detector: enrich · Adds behavioral anomaly signals
- Scoring → Threat classifier: classify · Applies ML-based threat classification
- Scoring → Playbook registry: route · Selects the applicable playbook
- Playbook registry → Auto-remediation: policy · Determines whether to act, ask approval, or defer
- Auto-remediation → GCE containment: dispatch · Routes GCE incidents to containment automation
- Auto-remediation → Ransomware: dispatch · Routes ransomware incidents to response automation
- Auto-remediation → SA compromise: dispatch · Routes service-account incidents to response automation
- Auto-remediation → Storage exfiltration: dispatch · Routes storage exfiltration incidents to response automation
- GCE containment → Snapshot: preserve · Captures evidence before disruption
- GCE containment → Isolate: contain · Isolates the impacted resource
- GCE containment → Terminate: stop · Terminates compromised resources when needed
- GCE containment → Process containment: deep contain · Uses host-level containment for process/file actions
- Ransomware → Snapshot: preserve · Snapshots before containment
- SA compromise → GCP client: revoke · Uses GCP APIs for credential and access response
- Storage exfiltration → GCP client: secure · Uses GCP APIs for bucket and access controls
- Snapshot → GCP client: api · Executes snapshot operations through GCP APIs
- Isolate → GCP client: api · Executes containment operations through GCP APIs
- Terminate → GCP client: api · Executes termination operations through GCP APIs
- GCP client → Audit log: log · Records every remediation action
- GCP client → Slack: notify · Sends outbound notifications
- GCP client → Jira: ticket · Creates or updates incident tickets
- GCP client → SIEM: forward · Streams response context to SIEM
- GCP client → AI summary: summarize · Generates incident summaries for operators
- Forensics worker → Audit log: evidence log · Produces forensic artifacts and audit trail
- Isolation worker → GCP client: execute · Runs heavy isolation workflows outside the main handler
- Queues → Queue processor: provisions · Defines the async queue boundary
- Workflows → Snapshot: provisions · Deploys response workflows and orchestration
- Containers → Forensics worker: provisions · Deploys long-running worker containers
- Containers → Isolation worker: provisions · Deploys long-running worker containers
- Security → GCP client: governs · Defines IAM and security posture for execution