# sovai-0/2gis-niche-search architecture

> Architecture diagram of the GitHub repository sovai-0/2gis-niche-search, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sovai-0/2gis-niche-search
- Repository: https://github.com/sovai-0/2gis-niche-search

## Overview

2GIS Parser is a configurable organization-data collector with CLI and desktop entry points. Its main workflow plans region and rubric/search tasks, discovers organization IDs, fetches each organization’s full card through a real Chrome browser, and persists resumable queue state plus raw responses. A separate export workflow normalizes and deduplicates those responses into reports and CSV/JSONL/XLSX files. The README also documents region refresh, machine diagnostics, and ZIP-based team handoff. Several internal modules are indexed but not excerpted, so their standalone behavior and some connections are omitted.

Main entry points: [`src/venues2gis/orchestrator.py`](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/orchestrator.py) (Task planner), [`run.py`](https://github.com/sovai-0/2gis-niche-search/blob/main/run.py) (CLI commands), [`src/venues2gis/store.py`](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/store.py) (SQLite and raw JSONL), [`src/venues2gis/desktop_run.py`](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/desktop_run.py) (Desktop workflow)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_planning["Configuration and planning"]
  node_cli["CLI commands<br/>[run.py]"]
  node_config["Run configuration<br/>[config.py]"]
  node_orchestrator["Task planner<br/>[orchestrator.py]"]
  node_queries["Search queries<br/>[queries.py]"]
  node_rubrics["Rubric sets<br/>[rubrics.py]"]
  node_regions["Region catalog<br/>[regions.py]"]
end

subgraph group_collection["Browser collection"]
  node_worker["Worker pool<br/>[worker.py]"]
  node_browser["Chrome CDP<br/>[browser.py]"]
  node_discover["Result discovery<br/>[discover.py]"]
  node_cards["Card extraction<br/>[cards.py]"]
end

subgraph group_data["Persistence and transfer"]
  node_store[("SQLite and raw JSONL<br/>[store.py]")]
  node_transfer["ZIP handoff<br/>[transfer.py]"]
end

subgraph group_enrichment["Enrichment and export"]
  node_normalize["Record normalization<br/>[normalize.py]"]
  node_capacity["Capacity parsing<br/>[capacity.py]"]
  node_classify["Venue classification<br/>[classify.py]"]
  node_dedupe["Duplicate detection<br/>[dedupe.py]"]
  node_export["Data exports and reports<br/>[export.py]"]
end

subgraph group_operations["Desktop and diagnostics"]
  node_desktop["Desktop workflow<br/>[desktop_run.py]"]
  node_leadchecks["Lead checks<br/>[lead_checks.py]"]
  node_doctor["Machine diagnostics<br/>[doctor.py]"]
  node_probe["2GIS probe<br/>[probe.py]"]
  node_regionreport["Region status<br/>[report_regions.py]"]
end

node_user(("Operator"))
node_twogis{{"2GIS services"}}

node_user -->|"runs commands"| node_cli
node_cli -->|"loads settings"| node_config
node_cli -->|"starts collection"| node_orchestrator
node_cli -->|"runs export"| node_export
node_cli -->|"runs diagnostics"| node_doctor
node_cli -->|"runs probe"| node_probe
node_cli -->|"packs or merges"| node_transfer
node_orchestrator -->|"reads settings"| node_config
node_orchestrator -->|"builds query sources"| node_queries
node_orchestrator -->|"builds rubric sources"| node_rubrics
node_orchestrator -->|"loads regions"| node_regions
node_orchestrator -->|"queues tasks"| node_store
node_orchestrator -->|"starts workers"| node_worker
node_worker -->|"runs discovery"| node_discover
node_worker -->|"fetches cards"| node_cards
node_worker -->|"claims and saves"| node_store
node_worker -->|"opens session"| node_browser
node_discover -->|"browses results"| node_browser
node_discover -->|"records discoveries"| node_store
node_cards -->|"captures card response"| node_browser
node_cards -->|"receives API data"| node_twogis
node_regions -.->|"refreshes region list"| node_twogis
node_export -->|"reads raw records"| node_store
node_export -->|"normalizes items"| node_normalize
node_normalize -->|"parses capacity"| node_capacity
node_normalize -->|"classifies venues"| node_classify
node_export -->|"finds similar records"| node_dedupe
node_transfer -->|"moves collection state"| node_store
node_desktop -->|"reads settings"| node_config
node_desktop -->|"starts collection"| node_orchestrator
node_desktop -->|"uses collection data"| node_store
node_desktop -->|"writes workbooks"| node_export
node_desktop -->|"runs lead checks"| node_leadchecks
node_doctor -->|"checks browser setup"| node_browser
node_regionreport -->|"summarizes regions"| node_regions

click node_cli "https://github.com/sovai-0/2gis-niche-search/blob/main/run.py"
click node_config "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/config.py"
click node_orchestrator "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/orchestrator.py"
click node_queries "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/queries.py"
click node_rubrics "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/rubrics.py"
click node_regions "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/regions.py"
click node_worker "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/worker.py"
click node_browser "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/browser.py"
click node_discover "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/discover.py"
click node_cards "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/cards.py"
click node_store "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/store.py"
click node_transfer "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/transfer.py"
click node_normalize "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/normalize.py"
click node_capacity "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/capacity.py"
click node_classify "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/classify.py"
click node_dedupe "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/dedupe.py"
click node_export "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/export.py"
click node_desktop "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/desktop_run.py"
click node_leadchecks "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/lead_checks.py"
click node_doctor "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/doctor.py"
click node_probe "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/probe.py"
click node_regionreport "https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/report_regions.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_config,node_orchestrator,node_queries,node_rubrics,node_regions toneBlue
class node_worker,node_browser,node_discover,node_cards toneAmber
class node_store,node_transfer toneMint
class node_normalize,node_capacity,node_classify,node_dedupe,node_export toneRose
class node_desktop,node_leadchecks,node_doctor,node_probe,node_regionreport,node_user,node_twogis toneIndigo
```

## Components

### Configuration and planning

- [CLI commands](https://github.com/sovai-0/2gis-niche-search/blob/main/run.py): `run.py`
- [Run configuration](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/config.py): `src/venues2gis/config.py`
- [Task planner](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/orchestrator.py): `src/venues2gis/orchestrator.py`
- [Search queries](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/queries.py): `src/venues2gis/queries.py`
- [Rubric sets](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/rubrics.py): `src/venues2gis/rubrics.py`
- [Region catalog](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/regions.py): `src/venues2gis/regions.py`

### Browser collection

- [Worker pool](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/worker.py): `src/venues2gis/worker.py`
- [Chrome CDP](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/browser.py): `src/venues2gis/browser.py`
- [Result discovery](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/discover.py): `src/venues2gis/discover.py`
- [Card extraction](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/cards.py): `src/venues2gis/cards.py`

### Persistence and transfer

- [SQLite and raw JSONL](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/store.py): `src/venues2gis/store.py`
- [ZIP handoff](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/transfer.py): `src/venues2gis/transfer.py`

### Enrichment and export

- [Record normalization](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/normalize.py): `src/venues2gis/normalize.py`
- [Capacity parsing](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/capacity.py): `src/venues2gis/capacity.py`
- [Venue classification](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/classify.py): `src/venues2gis/classify.py`
- [Duplicate detection](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/dedupe.py): `src/venues2gis/dedupe.py`
- [Data exports and reports](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/export.py): `src/venues2gis/export.py`

### Desktop and diagnostics

- [Desktop workflow](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/desktop_run.py): `src/venues2gis/desktop_run.py`
- [Lead checks](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/lead_checks.py): `src/venues2gis/lead_checks.py`
- [Machine diagnostics](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/doctor.py): `src/venues2gis/doctor.py`
- [2GIS probe](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/probe.py): `src/venues2gis/probe.py`
- [Region status](https://github.com/sovai-0/2gis-niche-search/blob/main/src/venues2gis/report_regions.py): `src/venues2gis/report_regions.py`

### Other components

- **Operator**
- **2GIS services**

## Connections

- Operator → CLI commands: runs commands (evidence: `README.md`)
- CLI commands → Run configuration: loads settings (evidence: `run.py`)
- CLI commands → Task planner: starts collection (evidence: `run.py`)
- CLI commands → Data exports and reports: runs export (evidence: `run.py`)
- CLI commands → Machine diagnostics: runs diagnostics (evidence: `run.py`)
- CLI commands → 2GIS probe: runs probe (evidence: `run.py`)
- CLI commands → ZIP handoff: packs or merges (evidence: `run.py`)
- Task planner → Run configuration: reads settings (evidence: `src/venues2gis/orchestrator.py`)
- Task planner → Search queries: builds query sources (evidence: `src/venues2gis/orchestrator.py`)
- Task planner → Rubric sets: builds rubric sources (evidence: `src/venues2gis/orchestrator.py`)
- Task planner → Region catalog: loads regions (evidence: `src/venues2gis/orchestrator.py`)
- Task planner → SQLite and raw JSONL: queues tasks (evidence: `src/venues2gis/orchestrator.py`)
- Task planner → Worker pool: starts workers (evidence: `README.md`)
- Worker pool → Result discovery: runs discovery (evidence: `src/venues2gis/worker.py`)
- Worker pool → Card extraction: fetches cards (evidence: `src/venues2gis/worker.py`)
- Worker pool → SQLite and raw JSONL: claims and saves (evidence: `src/venues2gis/worker.py`)
- Worker pool → Chrome CDP: opens session (evidence: `src/venues2gis/worker.py`)
- Result discovery → Chrome CDP: browses results (evidence: `README.md`)
- Result discovery → SQLite and raw JSONL: records discoveries (evidence: `README.md`)
- Card extraction → Chrome CDP: captures card response (evidence: `README.md`)
- Card extraction → 2GIS services: receives API data (evidence: `README.md`)
- Region catalog → 2GIS services: refreshes region list (evidence: `src/venues2gis/regions.py`)
- Data exports and reports → SQLite and raw JSONL: reads raw records (evidence: `src/venues2gis/export.py`)
- Data exports and reports → Record normalization: normalizes items (evidence: `src/venues2gis/export.py`)
- Record normalization → Capacity parsing: parses capacity (evidence: `src/venues2gis/normalize.py`)
- Record normalization → Venue classification: classifies venues (evidence: `src/venues2gis/normalize.py`)
- Data exports and reports → Duplicate detection: finds similar records (evidence: `src/venues2gis/export.py`)
- ZIP handoff → SQLite and raw JSONL: moves collection state (evidence: `README.md`)
- Desktop workflow → Run configuration: reads settings (evidence: `src/venues2gis/desktop_run.py`)
- Desktop workflow → Task planner: starts collection (evidence: `src/venues2gis/desktop_run.py`)
- Desktop workflow → SQLite and raw JSONL: uses collection data (evidence: `src/venues2gis/desktop_run.py`)
- Desktop workflow → Data exports and reports: writes workbooks (evidence: `src/venues2gis/desktop_run.py`)
- Desktop workflow → Lead checks: runs lead checks (evidence: `src/venues2gis/desktop_run.py`)
- Machine diagnostics → Chrome CDP: checks browser setup (evidence: `src/venues2gis/doctor.py`)
- Region status → Region catalog: summarizes regions (evidence: `src/venues2gis/report_regions.py`)
