How gorkemguler/metascout works
MetaScout discovers publicly exposed documents, downloads and analyzes them for metadata and optional content leaks, then produces reports. The main workflow is coordinated by the scan pipeline; users can initiate it through the CLI, local web UI, or job-based REST API. Discovery includes crawling, sitemaps, archive/search sources, subdomains, and critical-file checks. The README documents HTML, JSON, and optional PDF reports, plus scan history/diffs. Some discovery and pipeline details are unsampled, so only documented or indexed relationships are shown.
Main entry points: pipeline.py Scan pipeline · web.py Local web UI · __init__.py Content scanning · cli.py CLI
Components 26
User interfaces
- CLI
src/metascout/cli.py - Local web UI
src/metascout/web.py - REST API
src/metascout/api/app.py - Background jobs
src/metascout/api/jobs.py - Scan settings
src/metascout/config.py
Discovery
- Crawl and sitemap
src/metascout/discovery/crawler.py - Wayback discovery
src/metascout/discovery/wayback.py - DDGS search
src/metascout/discovery/ddgs_search.py - Search APIs
src/metascout/discovery/search_engines.py - Subdomain enumeration
src/metascout/discovery/subdomains.py
Document analysis
- Scan pipeline
src/metascout/pipeline.py - Document downloader
src/metascout/downloader.py - Metadata extraction
src/metascout/metadata/exiftool_wrapper.py - Metadata analysis
src/metascout/metadata/analyzer.py - Content scanning
src/metascout/content_scan/__init__.py - Text and OCR
src/metascout/content_scan/text_extract.py - PII patterns
src/metascout/content_scan/pii_patterns.py - Visual signatures
src/metascout/content_scan/visual_signature.py - Scan result models
src/metascout/models.py
Results and reports
- HTML and JSON reports
src/metascout/report/html_report.py - PDF report
src/metascout/report/pdf_report.py - Report comparison
src/metascout/diff.py
Other components
- Scanner
- Search providers
- Target websites
- ExifTool
Connections 28
- Scanner → CLI: runs scan
README.md - Scanner → Local web UI: starts scan
README.md - Scanner → REST API: submits job
src/metascout/api/app.py - CLI → Scan pipeline: runs scan
src/metascout/cli.py - Local web UI → Scan pipeline: runs scan
src/metascout/web.py - REST API → Background jobs: submits job
src/metascout/api/app.py - Background jobs → Scan pipeline: runs scan
src/metascout/api/app.py - Scan pipeline → Scan settings: uses settings
src/metascout/pipeline.py - Scan pipeline → Crawl and sitemap: discovers documents
src/metascout/pipeline.py - Scan pipeline → Wayback discovery: queries archive
README.md - Scan pipeline → DDGS search: queries DDGS
src/metascout/pipeline.py - Scan pipeline → Search APIs: queries providers
src/metascout/pipeline.py - Scan pipeline → Subdomain enumeration: enumerates subdomains
README.md - Scan pipeline → Document downloader: downloads files
src/metascout/pipeline.py - Document downloader → Target websites: fetches documents
README.md - Scan pipeline → Metadata extraction: extracts metadata
src/metascout/pipeline.py - Metadata extraction → ExifTool: invokes
src/metascout/metadata/exiftool_wrapper.py - Scan pipeline → Metadata analysis: analyzes metadata
src/metascout/pipeline.py - Metadata analysis → PII patterns: checks internal hosts
src/metascout/metadata/analyzer.py - Scan pipeline → Content scanning: scans content
src/metascout/pipeline.py - Content scanning → Text and OCR: extracts text
src/metascout/content_scan/__init__.py - Text and OCR → Visual signatures: checks signatures
src/metascout/content_scan/__init__.py - Content scanning → PII patterns: detects patterns
src/metascout/content_scan/__init__.py - Scan pipeline → Scan result models: builds findings
src/metascout/pipeline.py - Scan pipeline → HTML and JSON reports: renders reports
README.md - Scan pipeline → PDF report: renders PDF
README.md - Local web UI → Report comparison: compares runs
src/metascout/web.py - Search APIs → Search providers: queries