# artemsteshenko/parser_maps architecture

> Architecture diagram of the GitHub repository artemsteshenko/parser_maps, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/artemsteshenko/parser_maps
- Repository: https://github.com/artemsteshenko/parser_maps

## Overview

This is a command-line scraper for Yandex Maps. Its main workflow searches a city, district and organization category to save organization URLs, then reads those URL files, visits each organization page, extracts fields, shapes the records and writes CSV output. Category mappings and JSON link files support repeated runs; additional extracted fields include ratings, contact details, hours and goods, while review scraping exists as a helper but is not wired into the main parse flow. The sampled source establishes the two-stage flow; other category folders are visible in the tree, but their contents were not inspected.

Main entry points: [`info_parser.py`](https://github.com/artemsteshenko/parser_maps/blob/master/info_parser.py) (Parser), [`link_parser.py`](https://github.com/artemsteshenko/parser_maps/blob/master/link_parser.py) (Link command), [`soup_parser.py`](https://github.com/artemsteshenko/parser_maps/blob/master/soup_parser.py) (Field extraction), [`links`](https://github.com/artemsteshenko/parser_maps/tree/master/links) (Organization URLs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Link collection"]
  node_linkcli["Link command<br/>[link_parser.py]"]
  node_category["Category mapping<br/>[constants.py]"]
  node_collector["LinksCollector<br/>[link_parser.py]"]
  node_linkfiles["Organization URLs"]
end

subgraph group_enrichment["Page enrichment"]
  node_infocli["Info command<br/>[info_parser.py]"]
  node_parser["Parser<br/>[info_parser.py]"]
  node_soup["Field extraction<br/>[soup_parser.py]"]
  node_reviews["Review helper<br/>[soup_parser.py]"]
end

subgraph group_data["Scraped data"]
  node_record["Record formatter<br/>[json_pattern.py]"]
  node_csv["CSV output"]
end

node_operator(("Operator"))
node_maps["Yandex Maps"]

node_operator -->|"runs"| node_linkcli
node_linkcli -->|"looks up category"| node_category
node_linkcli -->|"invokes"| node_collector
node_collector -->|"searches listings"| node_maps
node_collector -->|"writes URLs"| node_linkfiles
node_operator -->|"runs"| node_infocli
node_infocli -->|"reads URLs"| node_linkfiles
node_infocli -->|"passes URLs"| node_parser
node_parser -->|"visits organizations"| node_maps
node_parser -->|"extracts fields"| node_soup
node_soup -->|"provides values"| node_record
node_parser -->|"formats organization"| node_record
node_parser -->|"writes batches"| node_csv

click node_linkcli "https://github.com/artemsteshenko/parser_maps/blob/master/link_parser.py"
click node_category "https://github.com/artemsteshenko/parser_maps/blob/master/utils/constants.py"
click node_collector "https://github.com/artemsteshenko/parser_maps/blob/master/link_parser.py"
click node_linkfiles "https://github.com/artemsteshenko/parser_maps/tree/master/links"
click node_infocli "https://github.com/artemsteshenko/parser_maps/blob/master/info_parser.py"
click node_parser "https://github.com/artemsteshenko/parser_maps/blob/master/info_parser.py"
click node_soup "https://github.com/artemsteshenko/parser_maps/blob/master/soup_parser.py"
click node_reviews "https://github.com/artemsteshenko/parser_maps/blob/master/soup_parser.py"
click node_record "https://github.com/artemsteshenko/parser_maps/blob/master/utils/json_pattern.py"
click node_csv "https://github.com/artemsteshenko/parser_maps/tree/master/result_output"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_linkcli,node_category,node_collector,node_linkfiles toneBlue
class node_infocli,node_parser,node_soup,node_reviews toneAmber
class node_record,node_csv toneMint
class node_operator,node_maps toneIndigo
```

## Components

### Link collection

- [Link command](https://github.com/artemsteshenko/parser_maps/blob/master/link_parser.py): `link_parser.py`
- [Category mapping](https://github.com/artemsteshenko/parser_maps/blob/master/utils/constants.py): `utils/constants.py`
- [LinksCollector](https://github.com/artemsteshenko/parser_maps/blob/master/link_parser.py): `link_parser.py`
- [Organization URLs](https://github.com/artemsteshenko/parser_maps/tree/master/links): `links`

### Page enrichment

- [Info command](https://github.com/artemsteshenko/parser_maps/blob/master/info_parser.py): `info_parser.py`
- [Parser](https://github.com/artemsteshenko/parser_maps/blob/master/info_parser.py): `info_parser.py`
- [Field extraction](https://github.com/artemsteshenko/parser_maps/blob/master/soup_parser.py): `soup_parser.py`
- [Review helper](https://github.com/artemsteshenko/parser_maps/blob/master/soup_parser.py): `soup_parser.py`

### Scraped data

- [Record formatter](https://github.com/artemsteshenko/parser_maps/blob/master/utils/json_pattern.py): `utils/json_pattern.py`
- [CSV output](https://github.com/artemsteshenko/parser_maps/tree/master/result_output): `result_output`

### Other components

- **Operator**
- **Yandex Maps**

## Connections

- Operator → Link command: runs
- Link command → Category mapping: looks up category
- Link command → LinksCollector: invokes
- LinksCollector → Yandex Maps: searches listings
- LinksCollector → Organization URLs: writes URLs
- Operator → Info command: runs
- Info command → Organization URLs: reads URLs
- Info command → Parser: passes URLs
- Parser → Yandex Maps: visits organizations
- Parser → Field extraction: extracts fields
- Field extraction → Record formatter: provides values
- Parser → Record formatter: formats organization
- Parser → CSV output: writes batches