How freelabz/secator works
Project Type and Purpose
• Secator is a Python-based CLI application and library that orchestrates security assessment tasks and workflows.
• It wraps dozens of external pentesting tools, normalizes inputs/outputs, and supports both local and distributed execution via Celery.
• It can be used as a standalone tool (CLI), as a Python library, in Docker containers, or deployed in Kubernetes via Helm.
File Structure Highlights
• Top-level:
• Dockerfiles for multiple OSes, docker-compose.yml, cloudbuild.yaml
• helm/ for Kubernetes deployment (Chart.yaml, templates for secator-manager, secator-worker, redis)
• scripts/ for installer and demos
• secator/ Python package (core logic, CLI entrypoint, tasks, runners, exporters, hooks, configs)
• tests/ covering unit, integration, performance
• secator/ subdirs:
• cli.py, config.py, celery.py & celery_utils.py (entrypoints and Celery integration)
• runners/ (synchronous runner, Celery runner, workflow orchestrator)
• tasks/ (plugins for each external security tool)
• workflows/ (YAML-driven compositions of tasks)
• exporters/ (console, csv, json, table, gdrive)
• hooks/ (drivers: gcs, mongodb), serializers, output_types
Core Components
a. CLI Layer
• secator executable (entrypoint in setup.py → cli.py)
• Argument parser dispatching to “scan”, “task”, or “workflow” commands
b. Core Library
• Configuration loader (profiles, scan/workflow YAMLs)
• Runner subsystem:
• Local runner (executes tasks as subprocesses)
• Celery runner (enqueues tasks via Celery broker)
• Workflow runner (chains tasks based on definitions)
• Task plugins: wrappers invoking external tools, parsing raw output
• Serializer + Output Types: normalize raw outputs into a unified schema
• Exporters: write or stream final results to console, files, cloud services
c. Distributed Execution
• Message broker (e.g., Redis) and result backend
• Celery workers (secator-worker) consuming task messages
• Signals and utilities for tracking and orchestrating distributed tasks
d. External Dependencies
• External pentesting tools invoked as child processes
• Optional services: Google Drive API, GCS, MongoDB (via hooks)
e. Deployment Artifacts
• Docker container(s) for client and worker
• Kubernetes Helm chart defining:
• Redis service
• secator-manager (CLI/HTTP gRPC gateway if any)
• secator-worker replica set
Relationships and Data Flow
1. User invokes secator CLI → CLI Layer
2. CLI parses command → loads config profiles and workflow/task definitions
3. CLI hands off to Runner:
• Local runner: spawns external tool processes directly
• Celery runner: pushes tasks into Redis broker
4. (Distributed) Celery workers pick up tasks → call Task plugin wrappers → run external tools → produce raw output
5. Raw output → Serializer → Output Type objects → sent back via Celery result backend or directly to export pipeline
6. Exporters serialize final objects into chosen formats/targets (console, JSON, CSV, GDrive, GCS, MongoDB)
7. Workflow runner coordinates multiple tasks in sequence or parallel, passing data between them
Architectural Patterns and Principles
• Plugin architecture: each tool integration is a separate module under secator/tasks, making it easy to extend.
• Separation of concerns:
• CLI vs Runner vs Task plugin vs Serializer vs Exporter
• Config-driven workflows: YAML profiles decouple orchestration from code logic.
• Optional distributed execution: Celery integration communicates via broker, isolating concurrency concerns.
• Declarative deployment: Dockerfiles and Helm chart encapsulate environment and scale concerns.
Essential Elements Checklist
• All components clearly labeled
• Entry points vs core vs plugins vs infra vs external
• Deployment layer: group Docker/Helm resources separately from the application code