How autumn-27/scopesentry-scan works
Project type and purpose
• ScopeSentry-Scan is a Go-based, modular security scanner (CLI + microservices) that orchestrates multiple scanning engines (e.g. nuclei, gospider, subfinder, rustscan, etc.), caches and stores results, and exposes a server API.
• It is deployed via Docker Compose in multiple containers: server, scan workers, MongoDB, Redis, and auxiliary caches.
File structure for architectural clues
Top‐level:
• cmd/ScopeSentry – entrypoint for CLI/server process
• internal/… – core packages (config, context management, runner, handler, modules, notification, storage adapters, etc.)
• libs/… – vendored/open-source scanners (fingerprintx, gospider, nuclei, subfinder) used as libraries or executables
• modules/… – scanning modules plugged into the runner (asset mapping, port scan, URL scan, vulnerability scan, etc.)
• build/*.yml – Docker Compose definitions for services (scan, server, Redis, MongoDB)
README
• Confirms this is the “scan side” of the ScopeSentry platform.
• Points to a public website and main repo, indicating a two‐tiered architecture (frontend/server vs. scan backend).
A. CLI / Server Process (cmd/ScopeSentry)
• Reads ScopeSentryConfig.yaml
• Exposes HTTP (or gRPC) API for task submission & status
• Triggers configuration updates via internal/configupdater
B. Redis Queue (build/redis-docker-compose.yml)
• Acts as task broker between server and scan workers
C. Scan Worker Service (cmd/ScopeSentry invoked in “scan” container)
• Consumes tasks from Redis
• Uses internal/runner, internal/handler to dispatch modules
• Loads modules/plugins from modules/… and libs/…
• Applies contextmanagement, pooling, bigcache (in-memory), pebbledb (local embedded), etc.
D. MongoDB (build/mongodb-docker-compose.yml)
• Stores scan results, duplicates, management data
• internal/mongodb adapter
E. Notification Service
• internal/notification pushes alerts after scans complete
• Could integrate e-mail, Slack, webhooks
F. Local Caches
• BigCache (internal/bigcache) – hot in-memory caching
• PebbleDB (internal/pebbledb) – embedded key-value store for intermediate state
G. External Scanner Libraries / Executables
• libs/nuclei, libs/gospider, libs/subfinder, libs/fingerprintx as imported packages or forked CLI tools
• tools/… executables (ksubdomain, rustscan, etc.) invoked by modules
Relationships and data flows
1. User or upstream service calls the Server API (HTTP/gRPC).
2. Server publishes “Scan Task” to Redis queue.
3. Scan Worker subscribes to Redis queue, pulls tasks.
4. Worker loads configuration, context, and module plugins.
5. Modules orchestrate calls to external tools/libraries (nuclei, gospider, rustscan, etc.).
6. As modules produce results, Handler aggregates them and writes to MongoDB (and local PebbleDB for dedupe).
7. Notification component picks up final status and sends alerts.
Architectural patterns and principles
• Microservices/Containerized deployment – separate server vs. worker vs. data stores
• Producer-Consumer via Redis queue
• Plugin-based architecture – modules found under modules/…, loaded dynamically by internal/configupdater & internal/runner
• Separation of concerns – config, context, runner, handler, storage adapters, notification all in internal/…
• Cache layering – in-memory (bigcache), on-disk embedded (pebbledb), persistent (MongoDB)
• Use of open-source scanning libraries under libs/…