How intruder-sec/reconswift works
Project Type and Purpose
• Project Type: Backend service for an automated security reconnaissance framework.
• Purpose: Receive scan requests, enqueue them in AWS SQS, execute Bash‐based scans (quick/advanced/full), store results in MongoDB, and expose REST endpoints to a NextJS frontend.
File Structure
• Top-Level Files:
– Dockerfile, Jenkinsfile, .gitignore, LEARN.md – deployment & CI/CD
– index.js – application entrypoint
– routes.js – route definitions
– package.json / package-lock.json – Node dependencies & scripts
• Controllers (controllers/*.js): API logic for startScan, addScan, completeScan, deleteQueue
• Models (model/*.js): Mongoose schemas (UserSchema, UserScans)
• Utils:
– modules/*.sh – scan scripts (quick_scan, full_scan, advanced_scan)
– tools/Sublist3r – external subdomain enumeration binary
– wordlists/dir/common.txt – directory brute-forcing wordlist
– script.sh – orchestrator wrapper
• Public: static resources (TXT proofs)
README Insights
– API Layer (Node/Express)
– Message Queue (AWS SQS)
– Worker processes (Bash scripts)
– Database (MongoDB Atlas)
– Cloud services (Cloudinary / Azure)
• Env requirements: SQS URL, AWS keys, Mongo URL, Cloudinary creds
Main Components & Relationships
a. Components:
1. Client (NextJS Frontend)
2. API Server (Node.js + Express: index.js, routes.js, controllers/)
3. Message Queue (AWS SQS)
4. Worker Node(s) (Bash scripts under utils/modules/)
5. External Scanning Tools (Sublist3r, wordlists)
6. Database (MongoDB Atlas)
7. Cloudinary / Cloud Storage (for screenshots or assets)
8. CI/CD (Jenkins, Docker)
b. Interactions:
• Frontend → HTTP POST “/startScan” → API Server
• API Server: validates, writes UserScans record, enqueues message to SQS
• Worker (polls SQS): receives job → executes appropriate .sh script → calls Sublist3r/tools → outputs results
• Worker: updates scan status & results in MongoDB, uploads assets to Cloudinary (if needed)
• Frontend polls or WebSocket for status via API Server → reads from MongoDB
c. Architectural Patterns:
• Queue-based decoupling (producer/consumer) for long-running scans
• MVC-style separation in API (routes → controllers → models)
• Microservices-like workers (Bash scripts) invoked outside Node runtime
d. Key Technologies:
• Node.js, Express
• Mongoose (MongoDB)
• AWS SQS
• Bash scripting & external tools (Sublist3r)
• Docker, Jenkins for CI/CD
• Cloudinary / Azure for media