How pedramamini/threatingestor works
ThreatIngestor polls configured threat-intelligence sources, extracts and normalizes indicators, applies filtering/state, then dispatches artifacts to configured operators. The README documents a broad plugin catalog; source and operator families are grouped where their individual implementations were not sampled. Edges reflect the main orchestration and the explicitly shown extraction, persistence, and integration behavior.
Main entry points: threatingestor/__init__.py Ingestor · sources Feed Sources · sources/__init__.py IOC Extraction · operators Operator Plugins
Components 24
Orchestration
- CLI Entry
threatingestor/__init__.py - Configuration
threatingestor/config.py - Ingestor
threatingestor/__init__.py - Source State
threatingestor/state.py - Whitelist Filter
threatingestor/whitelist.py
Source Plugins
- Feed Sources
threatingestor/sources - RSS & Sitemap
threatingestor/sources/rss.py - Web & Image
threatingestor/sources/web.py - Git & GitHub
threatingestor/sources/github.py - Twitter & VirusTotal
threatingestor/sources/twitter.py - SQS Source
threatingestor/sources/sqs.py
Artifact Processing
- IOC Extraction
threatingestor/sources/__init__.py - Artifact Types
threatingestor/artifacts.py
Operators & Extras
- Operator Plugins
threatingestor/operators - CSV & SQL
threatingestor/operators/csv.py - MISP & ThreatKB
threatingestor/operators/misp.py - Queue Operators
threatingestor/operators/sqs.py - Queue Worker
threatingestor/extras/queueworker.py - Artifact Browser
threatingestor/extras/webapp.py
Other components
- Operator
- Threat Feeds
- External Systems
- Artifact SQLite DB
- SQS / Beanstalk
Connections 17
- Operator → CLI Entry: starts
- CLI Entry → Ingestor: creates
- Ingestor → Configuration: loads
- Ingestor → Feed Sources: runs plugins
- Ingestor → Source State: reads and saves
- Ingestor → Whitelist Filter: filters artifacts
- Feed Sources → Threat Feeds: fetches content
- RSS & Sitemap → IOC Extraction: processes entries
- IOC Extraction → Artifact Types: creates
- Ingestor → Operator Plugins: dispatches artifacts
- Operator Plugins → Artifact Types: handles
- CSV & SQL → External Systems: writes records
- MISP & ThreatKB → External Systems: updates events
- Queue Operators → SQS / Beanstalk: sends messages
- SQS Source → SQS / Beanstalk: reads messages
- Queue Worker → SQS / Beanstalk: reads and writes
- Artifact Browser → Artifact SQLite DB: queries