How smicallef/spiderfoot works
SpiderFoot automates OSINT reconnaissance through either its web UI or CLI. A user defines a target and scan configuration; scan control initializes selected plugins, which gather and publish findings for other plugins to consume. Findings are persisted for review, correlation, visualization and export. The graph preserves documented capability branches such as threat intelligence, discovery, dark-web search and cloud-bucket checks. Some branches are represented by their module directory because their implementations were not sampled; those relationships are documented at the capability level rather than inferred from module names.
Main entry points: plugin.py Plugin runtime · sfwebui.py Web interface · sfscan.py Scan control · db.py Scan database
Components 17
User interfaces
- Web interface
sfwebui.py - Command line
sfcli.py - Application entry
sf.py
Scan execution
- Scan control
sfscan.py - SpiderFoot services
sflib.py - Plugin runtime
spiderfoot/plugin.py - Event model
spiderfoot/event.py - Thread pool
spiderfoot/threadpool.py
Discovery modules
- Discovery and enrichment
modules - Threat intelligence
modules - Web and social discovery
modules - Dark-web search
modules - Cloud bucket checks
modules
Analysis and results
- Scan database
spiderfoot/db.py - Correlation engine
spiderfoot/correlation.py - Results and visualization
spiderfoot/static/js/viz.js
Other components
- Analyst
Connections 23
- Analyst → Web interface: configures scan
- Analyst → Command line: runs scans
- Web interface → Scan control: starts scan
sfwebui.py - Application entry → Web interface: starts web UI
sf.py - Application entry → Scan control: starts scanning
sf.py - Scan control → SpiderFoot services: uses services
sfscan.py - Scan control → Plugin runtime: initializes plugins
sfscan.py - Plugin runtime → Thread pool: submits tasks
spiderfoot/plugin.py - Plugin runtime → Event model: publishes events
spiderfoot/plugin.py - Plugin runtime → Discovery and enrichment: dispatches events
- Plugin runtime → Threat intelligence: dispatches events
- Plugin runtime → Web and social discovery: dispatches events
- Plugin runtime → Dark-web search: dispatches events
- Plugin runtime → Cloud bucket checks: dispatches events
- Discovery and enrichment → Plugin runtime: publishes findings
- Threat intelligence → Plugin runtime: publishes findings
- Web and social discovery → Plugin runtime: publishes findings
- Dark-web search → Plugin runtime: publishes findings
- Cloud bucket checks → Plugin runtime: publishes findings
- Scan control → Scan database: stores scan data
- Correlation engine → Scan database: queries events
spiderfoot/correlation.py - Web interface → Scan database: reads scan data
- Web interface → Results and visualization: shows visualizations