# freelabz/secator architecture

> Architecture diagram of the GitHub repository freelabz/secator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/freelabz/secator
- Repository: https://github.com/freelabz/secator

## Overview

##### Project Type and Purpose
- Secator is a Python-based CLI application and library that orchestrates security assessment tasks and workflows.
- It wraps dozens of external pentesting tools, normalizes inputs/outputs, and supports both local and distributed execution via Celery.
- It can be used as a standalone tool (CLI), as a Python library, in Docker containers, or deployed in Kubernetes via Helm.

##### File Structure Highlights
- Top-level:
- Dockerfiles for multiple OSes, docker-compose.yml, cloudbuild.yaml
- helm/ for Kubernetes deployment (Chart.yaml, templates for secator-manager, secator-worker, redis)
- scripts/ for installer and demos
- secator/ Python package (core logic, CLI entrypoint, tasks, runners, exporters, hooks, configs)
- tests/ covering unit, integration, performance
- secator/ subdirs:
- cli.py, config.py, celery.py & celery_utils.py (entrypoints and Celery integration)
- runners/ (synchronous runner, Celery runner, workflow orchestrator)
- tasks/ (plugins for each external security tool)
- workflows/ (YAML-driven compositions of tasks)
- exporters/ (console, csv, json, table, gdrive)
- hooks/ (drivers: gcs, mongodb), serializers, output_types

##### Core Components
a. CLI Layer
- secator executable (entrypoint in setup.py → cli.py)
- Argument parser dispatching to “scan”, “task”, or “workflow” commands
b. Core Library
- Configuration loader (profiles, scan/workflow YAMLs)
- Runner subsystem:
- Local runner (executes tasks as subprocesses)
- Celery runner (enqueues tasks via Celery broker)
- Workflow runner (chains tasks based on definitions)
- Task plugins: wrappers invoking external tools, parsing raw output
- Serializer + Output Types: normalize raw outputs into a unified schema
- Exporters: write or stream final results to console, files, cloud services
c. Distributed Execution
- Message broker (e.g., Redis) and result backend
- Celery workers (secator-worker) consuming task messages
- Signals and utilities for tracking and orchestrating distributed tasks
d. External Dependencies
- External pentesting tools invoked as child processes
- Optional services: Google Drive API, GCS, MongoDB (via hooks)
e. Deployment Artifacts
- Docker container(s) for client and worker
- Kubernetes Helm chart defining:
- Redis service
- secator-manager (CLI/HTTP gRPC gateway if any)
- secator-worker replica set

##### Relationships and Data Flow
1. User invokes `secator` CLI → CLI Layer
2. CLI parses command → loads config profiles and workflow/task definitions
3. CLI hands off to Runner:
- Local runner: spawns external tool processes directly
- Celery runner: pushes tasks into Redis broker
4. (Distributed) Celery workers pick up tasks → call Task plugin wrappers → run external tools → produce raw output
5. Raw output → Serializer → Output Type objects → sent back via Celery result backend or directly to export pipeline
6. Exporters serialize final objects into chosen formats/targets (console, JSON, CSV, GDrive, GCS, MongoDB)
7. Workflow runner coordinates multiple tasks in sequence or parallel, passing data between them

##### Architectural Patterns and Principles
- Plugin architecture: each tool integration is a separate module under secator/tasks, making it easy to extend.
- Separation of concerns:
- CLI vs Runner vs Task plugin vs Serializer vs Exporter
- Config-driven workflows: YAML profiles decouple orchestration from code logic.
- Optional distributed execution: Celery integration communicates via broker, isolating concurrency concerns.
- Declarative deployment: Dockerfiles and Helm chart encapsulate environment and scale concerns.

##### Essential Elements Checklist
- All components clearly labeled
- Entry points vs core vs plugins vs infra vs external
- Deployment layer: group Docker/Helm resources separately from the application code

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Side
    subgraph "Client Side"
        EP["Entrypoint (pyproject.toml)"]:::user
        CLI["secator CLI"]:::user
        ConfigLoader["Configuration Loader"]:::user
        ConfigFiles["Config YAMLs (profiles/scans/workflows)"]:::user
    end

    %% Core Engine
    subgraph "Core Engine"
        RunnerLocal["Local Runner"]:::core
        RunnerCelery["Celery Runner"]:::core
        WorkflowRunner["Workflow Runner"]:::core
        CeleryInt["Celery Integration"]:::core
    end

    %% Plugin Zone
    subgraph "Plugin Zone"
        Tasks["Task Plugins"]:::plugin
        Serializers["Serializers"]:::plugin
        OutputTypes["Output Types"]:::plugin
        Exporters["Exporters"]:::plugin
        Hooks["Hooks (GCS, MongoDB)"]:::plugin
    end

    %% Infrastructure
    subgraph "Infrastructure"
        Redis["Redis Broker"]:::infra
        CeleryWorkers["Celery Workers"]:::infra
        ExtTools["External Tools"]:::external
        GDriveSvc["GDrive/GCS/MongoDB"]:::infra
    end

    %% Deployment
    subgraph "Deployment"
        Docker["Docker Containers"]:::deploy
        Kubernetes["Kubernetes (Helm)"]:::deploy
    end

    %% Data Flows
    EP -->|defines entrypoint| CLI
    CLI -->|parses args| ConfigLoader
    CLI -->|loads YAMLs| ConfigFiles
    CLI -->|dispatch scan/workflow| RunnerLocal
    CLI -->|dispatch scan/workflow| RunnerCelery
    CLI -->|dispatch workflows| WorkflowRunner

    WorkflowRunner -->|uses| RunnerLocal
    WorkflowRunner -->|uses| RunnerCelery

    RunnerLocal -->|spawns subprocess| Tasks
    RunnerCelery -->|push Celery message| Redis
    Redis -->|consumed by| CeleryWorkers
    CeleryWorkers -->|invoke| Tasks

    Tasks -->|invoke| ExtTools
    Tasks -->|stdout raw| Serializers
    Serializers -->|normalized objects| OutputTypes
    OutputTypes -->|to export| Exporters
    Exporters -->|write/stream| GDriveSvc

    %% Deployment Flows
    Docker --> CLI
    Docker --> CeleryWorkers
    Docker --> Redis
    Kubernetes --> Docker
    Kubernetes -->|deploy manager| CLI
    Kubernetes -->|deploy workers| CeleryWorkers
    Kubernetes -->|deploy redis| Redis

    %% Click Events
    click CLI "https://github.com/freelabz/secator/blob/main/secator/cli.py"
    click EP "https://github.com/freelabz/secator/blob/main/pyproject.toml"
    click ConfigLoader "https://github.com/freelabz/secator/blob/main/secator/config.py"
    click ConfigFiles "https://github.com/freelabz/secator/tree/main/secator/configs/"
    click CeleryInt "https://github.com/freelabz/secator/blob/main/secator/celery.py"
    click RunnerLocal "https://github.com/freelabz/secator/blob/main/secator/runners/command.py"
    click RunnerCelery "https://github.com/freelabz/secator/blob/main/secator/runners/celery.py"
    click WorkflowRunner "https://github.com/freelabz/secator/blob/main/secator/runners/workflow.py"
    click Tasks "https://github.com/freelabz/secator/tree/main/secator/tasks/"
    click Serializers "https://github.com/freelabz/secator/blob/main/secator/serializers/dataclass.py"
    click OutputTypes "https://github.com/freelabz/secator/blob/main/secator/output_types/_base.py"
    click Exporters "https://github.com/freelabz/secator/blob/main/secator/exporters/console.py"
    click Hooks "https://github.com/freelabz/secator/blob/main/secator/hooks/gcs.py"
    click Docker "https://github.com/freelabz/secator/tree/main/Dockerfile"
    click Kubernetes "https://github.com/freelabz/secator/blob/main/helm/Chart.yaml"

    %% Styles
    classDef user fill:#AED6F1,stroke:#1B4F72,color:#1B2631
    classDef core fill:#ABEBC6,stroke:#1D8348,color:#145A32
    classDef plugin fill:#F9E79F,stroke:#B7950B,color:#7D6608
    classDef infra fill:#F5CBA7,stroke:#BA4A00,color:#641E16
    classDef external fill:#D5DBDB,stroke:#565656,color:#1B2631
    classDef deploy fill:#D7BDE2,stroke:#6C3483,color:#4A235A
```
