How nexvisionlab/url-and-website-safety-threat-intelligence-scanner works
url-safety-investigator is a Python CLI for assessing URL and website risk, with a specialized online-shop analyzer and shareable reports. The main workflow normalizes a target, checks cache, gathers local and live signals, aggregates them into an explainable verdict, and presents or stores the result. The shop workflow reuses URL investigation before adding shop-specific checks and a risk band. Edges follow the pipeline and analyzer excerpts, the SOURCE INDEX, and README; unsampled detector and shop internals are represented at subsystem level where their exact wiring is not shown.
Main entry points: pipeline.py Investigation pipeline · analyzer.py Shop analyzer · report.py Report CLI · cli.py Investigation CLI
Components 26
User entry points
- Investigation CLI
usi/cli.py - Shop CLI
usi/shop/__main__.py - Report CLI
report.py
URL investigation
- Investigation pipeline
usi/pipeline.py - Static URL heuristics
usi/heuristics - WHOIS and RDAP
usi/lookups/whois_lookup.py - TLS checks
usi/lookups/tls_cert.py - Certificate search
usi/lookups/crtsh.py - Page fetching
usi/content/fetcher.py - Page extraction
usi/content/extractor.py - Content threat checks
usi/content - Reputation APIs
usi/reputation
Shop assessment
- Shop analyzer
usi/shop/analyzer.py - Shop page checks
usi/shop/pages.py - Shop fingerprinting
usi/shop/fingerprint.py - Shop risk rules
usi/shop/rules.py - Marketplace guidance
usi/shop/platforms.py
Results and state
- Verdict aggregator
usi/verdict/aggregator.py - Result cache
usi/cache.py - Result formatter
usi/output/formatter.py - Report builder
usi/output/report.py
Other components
- User
- Public data sources
- Reputation services
- Verdict and findings
- Markdown or HTML report
Connections 30
- User → Investigation CLI: submits URL
- Investigation CLI → Investigation pipeline: runs investigation
usi/cli.py - Investigation pipeline → Result cache: checks and stores
usi/pipeline.py - Investigation pipeline → Static URL heuristics: runs checks
usi/pipeline.py - Investigation pipeline → WHOIS and RDAP: looks up domain
usi/pipeline.py - WHOIS and RDAP → Public data sources: queries registration data
usi/lookups/whois_lookup.py - Investigation pipeline → TLS checks: checks certificate
usi/pipeline.py - Investigation pipeline → Certificate search: checks lookalike certificates
usi/pipeline.py - Investigation pipeline → Page fetching: fetches target
usi/pipeline.py - Page fetching → Page extraction: provides page content
usi/pipeline.py - Investigation pipeline → Content threat checks: checks extracted content
usi/pipeline.py - Investigation pipeline → Reputation APIs: runs enabled checks
usi/pipeline.py - Reputation APIs → Reputation services: queries APIs
usi/reputation/base.py - Investigation pipeline → Verdict aggregator: aggregates signals
usi/pipeline.py - Verdict aggregator → Verdict and findings: produces verdict
- Investigation pipeline → Result formatter: formats result
usi/pipeline.py - Investigation CLI → Result formatter: displays result
usi/cli.py - User → Shop CLI: submits shop
- Shop CLI → Shop analyzer: runs assessment
- Shop analyzer → Marketplace guidance: recognizes marketplace
usi/shop/analyzer.py - Shop analyzer → Investigation pipeline: runs URL investigation
usi/shop/analyzer.py - Shop analyzer → Shop page checks: checks shop pages
usi/shop/analyzer.py - Shop analyzer → Shop fingerprinting: compares shop signals
usi/shop/analyzer.py - Shop analyzer → Shop risk rules: rates shop risk
usi/shop/analyzer.py - Shop analyzer → Verdict and findings: returns shop assessment
- User → Report CLI: requests report
- Report CLI → Investigation pipeline: investigates URLs
report.py - Report CLI → Result cache: reads cached results
report.py - Report CLI → Report builder: builds report
report.py - Report builder → Markdown or HTML report: renders report