How nyn-05/lead_scraper works
LEAD_SCRAPER is a Python toolkit for discovering Google Maps businesses, public LinkedIn/Instagram profiles, and photography-service prospects in public posts. Users initiate collection through the CLI or Streamlit dashboard; collection results are processed, persisted in SQLite, and exported as CSV, JSON, or Excel. The graph separates the three discovery workflows from shared configuration, processing, and storage. Some subsystem links rely on README descriptions because the relevant implementations were not fully excerpted.
Main entry points: main.py CLI dispatch · storage.py Database and exports · dashboard.py Streamlit dashboard · scrapers.py Maps listings
Components 16
User interfaces
- CLI dispatch
main.py - Streamlit dashboard
lead_scraper/dashboard.py
Lead discovery
- Maps listings
lead_scraper/scrapers.py - Social profiles
lead_scraper/scrapers.py - Photography collectors
lead_scraper/scrapers.py
Processing and configuration
- Configuration
lead_scraper/config.py - Post normalization
lead_scraper/processing.py - Lead classification
lead_scraper/processing.py - Lead scoring
lead_scraper/processing.py
Persistence and exports
- Database and exports
lead_scraper/storage.py - SQLite database
data/leads.db - CSV JSON Excel
lead_scraper/storage.py
Other components
- User
- Public sources
- Search providers
- Google Maps
Connections 24
- User → CLI dispatch: runs command
README.md - User → Streamlit dashboard: uses dashboard
README.md - CLI dispatch → Maps listings: dispatches Maps
main.py - CLI dispatch → Social profiles: dispatches social
main.py - CLI dispatch → Photography collectors: runs collectors
main.py - Streamlit dashboard → Maps listings: starts collection
lead_scraper/dashboard.py - Streamlit dashboard → Social profiles: starts collection
lead_scraper/dashboard.py - Streamlit dashboard → Photography collectors: starts collection
lead_scraper/dashboard.py - Maps listings → Google Maps: collects listings
README.md - Social profiles → Search providers: searches profiles
lead_scraper/scrapers.py - Photography collectors → Public sources: collects posts
README.md - CLI dispatch → Configuration: loads settings
main.py - CLI dispatch → Post normalization: normalizes posts
main.py - CLI dispatch → Lead classification: classifies posts
main.py - CLI dispatch → Lead scoring: scores leads
main.py - Photography collectors → Post normalization: submits posts
README.md - Post normalization → Lead classification: feeds normalized posts
README.md - Lead classification → Lead scoring: provides classification
lead_scraper/processing.py - CLI dispatch → Database and exports: persists results
main.py - Maps listings → Database and exports: stores listings
README.md - Social profiles → Database and exports: stores profiles
README.md - Streamlit dashboard → Database and exports: reads stored data
lead_scraper/dashboard.py - Database and exports → SQLite database: reads and writes
lead_scraper/storage.py - Database and exports → CSV JSON Excel: writes exports
lead_scraper/storage.py