# autumn-27/scopesentry-scan architecture

> Architecture diagram of the GitHub repository autumn-27/scopesentry-scan, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/autumn-27/scopesentry-scan
- Repository: https://github.com/autumn-27/scopesentry-scan

## Overview

##### Project type and purpose
- ScopeSentry-Scan is a Go-based, modular security scanner (CLI + microservices) that orchestrates multiple scanning engines (e.g. nuclei, gospider, subfinder, rustscan, etc.), caches and stores results, and exposes a server API.
- It is deployed via Docker Compose in multiple containers: server, scan workers, MongoDB, Redis, and auxiliary caches.

##### File structure for architectural clues
Top‐level:
- cmd/ScopeSentry – entrypoint for CLI/server process
- internal/… – core packages (config, context management, runner, handler, modules, notification, storage adapters, etc.)
- libs/… – vendored/open-source scanners (fingerprintx, gospider, nuclei, subfinder) used as libraries or executables
- modules/… – scanning modules plugged into the runner (asset mapping, port scan, URL scan, vulnerability scan, etc.)
- build/*.yml – Docker Compose definitions for services (scan, server, Redis, MongoDB)

##### README
- Confirms this is the “scan side” of the ScopeSentry platform.
- Points to a public website and main repo, indicating a two‐tiered architecture (frontend/server vs. scan backend).

A. CLI / Server Process (cmd/ScopeSentry)
- Reads ScopeSentryConfig.yaml
- Exposes HTTP (or gRPC) API for task submission & status
- Triggers configuration updates via internal/configupdater

B. Redis Queue (build/redis-docker-compose.yml)
- Acts as task broker between server and scan workers

C. Scan Worker Service (cmd/ScopeSentry invoked in “scan” container)
- Consumes tasks from Redis
- Uses internal/runner, internal/handler to dispatch modules
- Loads modules/plugins from modules/… and libs/…
- Applies contextmanagement, pooling, bigcache (in-memory), pebbledb (local embedded), etc.

D. MongoDB (build/mongodb-docker-compose.yml)
- Stores scan results, duplicates, management data
- internal/mongodb adapter

E. Notification Service
- internal/notification pushes alerts after scans complete
- Could integrate e-mail, Slack, webhooks

F. Local Caches
- BigCache (internal/bigcache) – hot in-memory caching
- PebbleDB (internal/pebbledb) – embedded key-value store for intermediate state

G. External Scanner Libraries / Executables
- libs/nuclei, libs/gospider, libs/subfinder, libs/fingerprintx as imported packages or forked CLI tools
- tools/… executables (ksubdomain, rustscan, etc.) invoked by modules

##### Relationships and data flows
1. User or upstream service calls the Server API (HTTP/gRPC).
2. Server publishes “Scan Task” to Redis queue.
3. Scan Worker subscribes to Redis queue, pulls tasks.
4. Worker loads configuration, context, and module plugins.
5. Modules orchestrate calls to external tools/libraries (nuclei, gospider, rustscan, etc.).
6. As modules produce results, Handler aggregates them and writes to MongoDB (and local PebbleDB for dedupe).
7. Notification component picks up final status and sends alerts.

##### Architectural patterns and principles
- Microservices/Containerized deployment – separate server vs. worker vs. data stores
- Producer-Consumer via Redis queue
- Plugin-based architecture – modules found under modules/…, loaded dynamically by internal/configupdater & internal/runner
- Separation of concerns – config, context, runner, handler, storage adapters, notification all in internal/…
- Cache layering – in-memory (bigcache), on-disk embedded (pebbledb), persistent (MongoDB)
- Use of open-source scanning libraries under libs/…

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External User
    User["User / API Client"]:::external

    %% Server Container
    subgraph "Server Container"
        direction TB
        ServerCLI["Server Process\n(cmd/ScopeSentry/main.go)"]:::service
        RestructureCmd["Restructure Cmd\n(cmd/ScopeSentry/restructure_cmd.go)"]:::service
        ConfigUpdater["Configuration Updater"]:::internal
    end

    %% Redis Queue
    Redis["Redis Queue"]:::db

    %% Scan Worker Container
    subgraph "Scan Worker Container"
        direction TB
        WorkerCLI["Scan Worker\n(cmd/ScopeSentry/main.go)"]:::service
        ContextManager["Context Manager"]:::internal
        RunnerCore["Runner Core"]:::internal
        TaskHandler["Task Handler"]:::internal
        BigCache["BigCache (In-Memory)"]:::internal
        PebbleDB["PebbleDB (Embedded)"]:::db
        MongoAdapter["MongoDB Adapter"]:::internal
        Notification["Notification Service"]:::internal
        PluginInterfaces["Plugin Interfaces"]:::internal
        ModulesDir["Modules Directory"]:::internal
    end

    %% Datastores
    MongoDB["MongoDB"]:::db

    %% External Tools
    Nuclei["nuclei"]:::external
    Gospider["gospider"]:::external
    Subfinder["subfinder"]:::external
    Fingerprintx["fingerprintx"]:::external
    ToolsDir["Tools Executables"]:::external

    %% Flows
    User -->|"API Call"| ServerCLI
    ServerCLI -->|"Enqueue Task"| Redis
    RestructureCmd --> ServerCLI
    ConfigUpdater --> ServerCLI

    Redis -->|"Consume Task"| WorkerCLI
    WorkerCLI --> ContextManager
    WorkerCLI --> RunnerCore
    RunnerCore --> PluginInterfaces
    PluginInterfaces --> ModulesDir
    ModulesDir -->|Invoke| Nuclei
    ModulesDir -->|Invoke| Gospider
    ModulesDir -->|Invoke| Subfinder
    ModulesDir -->|Invoke| Fingerprintx
    ModulesDir -->|Invoke| ToolsDir

    RunnerCore --> BigCache
    RunnerCore --> PebbleDB

    ModulesDir --> TaskHandler
    TaskHandler -->|"Write Results"| MongoAdapter
    MongoAdapter -->|Persist| MongoDB
    TaskHandler -->|"Dedupe"| PebbleDB
    TaskHandler --> Notification
    Notification -->|"Alert"| User

    %% Click Events
    click ServerCLI "https://github.com/autumn-27/scopesentry-scan/blob/main/cmd/ScopeSentry/main.go"
    click RestructureCmd "https://github.com/autumn-27/scopesentry-scan/blob/main/cmd/ScopeSentry/restructure_cmd.go"
    click ConfigUpdater "https://github.com/autumn-27/scopesentry-scan/blob/main/internal/configupdater/configupdater.go"
    click Redis "https://github.com/autumn-27/scopesentry-scan/blob/main/build/redis-docker-compose.yml"
    click WorkerCLI "https://github.com/autumn-27/scopesentry-scan/blob/main/cmd/ScopeSentry/main.go"
    click ContextManager "https://github.com/autumn-27/scopesentry-scan/blob/main/internal/contextmanager/contextmanager.go"
    click RunnerCore "https://github.com/autumn-27/scopesentry-scan/blob/main/internal/runner/runner.go"
    click TaskHandler "https://github.com/autumn-27/scopesentry-scan/blob/main/internal/handler/task.go"
    click BigCache "https://github.com/autumn-27/scopesentry-scan/blob/main/internal/bigcache/bigcache.go"
    click PebbleDB "https://github.com/autumn-27/scopesentry-scan/blob/main/internal/pebbledb/pebbledb.go"
    click MongoAdapter "https://github.com/autumn-27/scopesentry-scan/blob/main/internal/mongodb/mongodb.go"
    click Notification "https://github.com/autumn-27/scopesentry-scan/blob/main/internal/notification/notification.go"
    click PluginInterfaces "https://github.com/autumn-27/scopesentry-scan/blob/main/internal/interfaces/module.go"
    click ModulesDir "https://github.com/autumn-27/scopesentry-scan/tree/main/modules/"
    click MongoDB "https://github.com/autumn-27/scopesentry-scan/blob/main/build/mongodb-docker-compose.yml"
    click Nuclei "https://github.com/autumn-27/scopesentry-scan/tree/main/libs/nuclei"
    click Gospider "https://github.com/autumn-27/scopesentry-scan/tree/main/libs/gospider"
    click Subfinder "https://github.com/autumn-27/scopesentry-scan/tree/main/libs/subfinder"
    click Fingerprintx "https://github.com/autumn-27/scopesentry-scan/tree/main/libs/fingerprintx"
    click ToolsDir "https://github.com/autumn-27/scopesentry-scan/tree/main/tools/"

    %% Styles
    classDef service fill:#b2f2bb,stroke:#2f8f3e,stroke-width:1px
    classDef db fill:#cde2ff,stroke:#2a5dab,stroke-width:1px
    classDef internal fill:#ffd8a8,stroke:#d35400,stroke-width:1px
    classDef external fill:#ffccd5,stroke:#a8323e,stroke-width:1px

    %% Legend
    subgraph "Legend"
        direction TB
        Svc["Service"]:::service
        DB["Datastore"]:::db
        Int["Internal Component"]:::internal
        Ext["External Tool/User"]:::external
    end
```
