How sovai-0/2gis-niche-search works
2GIS Parser is a configurable organization-data collector with CLI and desktop entry points. Its main workflow plans region and rubric/search tasks, discovers organization IDs, fetches each organization’s full card through a real Chrome browser, and persists resumable queue state plus raw responses. A separate export workflow normalizes and deduplicates those responses into reports and CSV/JSONL/XLSX files. The README also documents region refresh, machine diagnostics, and ZIP-based team handoff. Several internal modules are indexed but not excerpted, so their standalone behavior and some connections are omitted.
Main entry points: orchestrator.py Task planner · run.py CLI commands · store.py SQLite and raw JSONL · desktop_run.py Desktop workflow
Components 24
Configuration and planning
- CLI commands
run.py - Run configuration
src/venues2gis/config.py - Task planner
src/venues2gis/orchestrator.py - Search queries
src/venues2gis/queries.py - Rubric sets
src/venues2gis/rubrics.py - Region catalog
src/venues2gis/regions.py
Browser collection
- Worker pool
src/venues2gis/worker.py - Chrome CDP
src/venues2gis/browser.py - Result discovery
src/venues2gis/discover.py - Card extraction
src/venues2gis/cards.py
Persistence and transfer
- SQLite and raw JSONL
src/venues2gis/store.py - ZIP handoff
src/venues2gis/transfer.py
Enrichment and export
- Record normalization
src/venues2gis/normalize.py - Capacity parsing
src/venues2gis/capacity.py - Venue classification
src/venues2gis/classify.py - Duplicate detection
src/venues2gis/dedupe.py - Data exports and reports
src/venues2gis/export.py
Desktop and diagnostics
- Desktop workflow
src/venues2gis/desktop_run.py - Lead checks
src/venues2gis/lead_checks.py - Machine diagnostics
src/venues2gis/doctor.py - 2GIS probe
src/venues2gis/probe.py - Region status
src/venues2gis/report_regions.py
Other components
- Operator
- 2GIS services
Connections 35
- Operator → CLI commands: runs commands
README.md - CLI commands → Run configuration: loads settings
run.py - CLI commands → Task planner: starts collection
run.py - CLI commands → Data exports and reports: runs export
run.py - CLI commands → Machine diagnostics: runs diagnostics
run.py - CLI commands → 2GIS probe: runs probe
run.py - CLI commands → ZIP handoff: packs or merges
run.py - Task planner → Run configuration: reads settings
src/venues2gis/orchestrator.py - Task planner → Search queries: builds query sources
src/venues2gis/orchestrator.py - Task planner → Rubric sets: builds rubric sources
src/venues2gis/orchestrator.py - Task planner → Region catalog: loads regions
src/venues2gis/orchestrator.py - Task planner → SQLite and raw JSONL: queues tasks
src/venues2gis/orchestrator.py - Task planner → Worker pool: starts workers
README.md - Worker pool → Result discovery: runs discovery
src/venues2gis/worker.py - Worker pool → Card extraction: fetches cards
src/venues2gis/worker.py - Worker pool → SQLite and raw JSONL: claims and saves
src/venues2gis/worker.py - Worker pool → Chrome CDP: opens session
src/venues2gis/worker.py - Result discovery → Chrome CDP: browses results
README.md - Result discovery → SQLite and raw JSONL: records discoveries
README.md - Card extraction → Chrome CDP: captures card response
README.md - Card extraction → 2GIS services: receives API data
README.md - Region catalog → 2GIS services: refreshes region list
src/venues2gis/regions.py - Data exports and reports → SQLite and raw JSONL: reads raw records
src/venues2gis/export.py - Data exports and reports → Record normalization: normalizes items
src/venues2gis/export.py - Record normalization → Capacity parsing: parses capacity
src/venues2gis/normalize.py - Record normalization → Venue classification: classifies venues
src/venues2gis/normalize.py - Data exports and reports → Duplicate detection: finds similar records
src/venues2gis/export.py - ZIP handoff → SQLite and raw JSONL: moves collection state
README.md - Desktop workflow → Run configuration: reads settings
src/venues2gis/desktop_run.py - Desktop workflow → Task planner: starts collection
src/venues2gis/desktop_run.py - Desktop workflow → SQLite and raw JSONL: uses collection data
src/venues2gis/desktop_run.py - Desktop workflow → Data exports and reports: writes workbooks
src/venues2gis/desktop_run.py - Desktop workflow → Lead checks: runs lead checks
src/venues2gis/desktop_run.py - Machine diagnostics → Chrome CDP: checks browser setup
src/venues2gis/doctor.py - Region status → Region catalog: summarizes regions
src/venues2gis/report_regions.py