How ahmdngi/shipcrawler works
ShipCrawler is a Flask dashboard and background-worker platform for agent-driven maritime and people OSINT investigations. Users submit a target and optional agent profile; the API queues a task, and the worker runs the documented investigation phases through Hermes, streaming progress for the dashboard while producing reports for later viewing and history lookup. The README documents vessel identity, AIS, exposure, vulnerability, threat-intelligence, and report stages, plus analyst, red-team, and detection outputs. Person research and structured report parsing are also represented. Some subsystem links rely on README descriptions because the relevant source excerpts are incomplete.
Main entry points: api.py API routes · worker.py Queue worker · shipcrawler-core.js Dashboard client · shipcrawler-sse.js SSE client
Components 17
Dashboard and API
- Dashboard client
static/js/shipcrawler-core.js - SSE client
static/js/shipcrawler-sse.js - Flask app
app.py - API routes
routes/api.py
Task execution
- Queue worker
worker.py - Hermes agent
Investigation and reports
- OSINT methodology
skill/SKILL.md - Investigation phases
worker.py - Analysis and tiers
services/analysis.py - Report templates
templates - Report parser
parsers.py
Progress and results
- Task queue
worker.py - Progress logger
worker_progress.py - Progress logs
worker_progress.py - Report archive
Other components
- Analyst
- Maritime OSINT sources
Connections 21
- Analyst → Dashboard client: submits target
- Dashboard client → API routes: requests scan
- Flask app → API routes: registers routes
- API routes → Task queue: writes task
- Queue worker → Task queue: claims task
- Queue worker → Hermes agent: runs profile
- Queue worker → OSINT methodology: loads methodology
- Hermes agent → Investigation phases: executes investigation
- Investigation phases → Maritime OSINT sources: researches sources
- Queue worker → Progress logger: writes events
- Progress logger → Progress logs: appends JSONL
- API routes → Progress logger: reads events
- SSE client → API routes: opens stream
- API routes → SSE client: streams events
- Dashboard client → SSE client: connects updates
- Queue worker → Analysis and tiers: uses analysis
- Queue worker → Report templates: generates outputs
- Queue worker → Report archive: saves reports
- API routes → Report archive: loads history
- API routes → Report parser: parses report
- API routes → Dashboard client: returns report