# gagota/doctolib-scrapper architecture

> Architecture diagram of the GitHub repository gagota/doctolib-scrapper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gagota/doctolib-scrapper
- Repository: https://github.com/gagota/doctolib-scrapper

## Overview

Doctolib Scraper is a local Python application that collects practitioner data from a Doctolib search and saves it as YAML. The user configures the search and run options; the main entry point creates configuration and logging, then runs link collection and practitioner-detail collection, reusing saved links when link collection is disabled. Browser automation and parsing are implemented in the scraper. The README also describes future Maps and Sheets steps, but says they are not included. Connections reflect the shown entry-point calls and documented behavior; unsampled scraper persistence details are omitted.

Main entry points: [`src/_main.py`](https://github.com/gagota/doctolib-scrapper/blob/main/src/_main.py) (Run coordinator), [`src/config.py`](https://github.com/gagota/doctolib-scrapper/blob/main/src/config.py) (Run configuration), [`src/scraper.py`](https://github.com/gagota/doctolib-scrapper/blob/main/src/scraper.py) (Browser session), [`src/data_manager.py`](https://github.com/gagota/doctolib-scrapper/blob/main/src/data_manager.py) (YAML data manager)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_run["Run setup"]
  node_main["Run coordinator<br/>[_main.py]"]
  node_config["Run configuration<br/>[config.py]"]
  node_logger["Logging<br/>[logger_wrapper.py]"]
end

subgraph group_scrape["Scraping"]
  node_browser["Browser session<br/>[scraper.py]"]
  node_links["Search links<br/>[scraper.py]"]
  node_details["Practitioner details<br/>[scraper.py]"]
end

subgraph group_storage["Local data"]
  node_manager["YAML data manager<br/>[data_manager.py]"]
  node_yaml["YAML data files"]
end

node_user(("Local user"))
node_doctolib["Doctolib"]

node_user -->|"configures and runs"| node_main
node_main -->|"creates"| node_config
node_main -->|"creates"| node_logger
node_main -->|"runs"| node_links
node_main -->|"runs"| node_details
node_main -->|"loads saved links"| node_manager
node_main -->|"opens scraper"| node_browser
node_links -->|"requests search data"| node_doctolib
node_details -->|"requests practitioner data"| node_doctolib
node_browser -->|"navigates pages"| node_doctolib
node_manager -->|"reads and writes"| node_yaml
node_config -->|"defines data paths"| node_yaml

click node_main "https://github.com/gagota/doctolib-scrapper/blob/main/src/_main.py"
click node_config "https://github.com/gagota/doctolib-scrapper/blob/main/src/config.py"
click node_logger "https://github.com/gagota/doctolib-scrapper/blob/main/src/logger_wrapper.py"
click node_browser "https://github.com/gagota/doctolib-scrapper/blob/main/src/scraper.py"
click node_links "https://github.com/gagota/doctolib-scrapper/blob/main/src/scraper.py"
click node_details "https://github.com/gagota/doctolib-scrapper/blob/main/src/scraper.py"
click node_manager "https://github.com/gagota/doctolib-scrapper/blob/main/src/data_manager.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_config,node_logger,node_user toneBlue
class node_browser,node_links,node_details toneAmber
class node_manager,node_yaml toneMint
class node_doctolib toneIndigo
```

## Components

### Run setup

- [Run coordinator](https://github.com/gagota/doctolib-scrapper/blob/main/src/_main.py): `src/_main.py`
- [Run configuration](https://github.com/gagota/doctolib-scrapper/blob/main/src/config.py): `src/config.py`
- [Logging](https://github.com/gagota/doctolib-scrapper/blob/main/src/logger_wrapper.py): `src/logger_wrapper.py`

### Scraping

- [Browser session](https://github.com/gagota/doctolib-scrapper/blob/main/src/scraper.py): `src/scraper.py`
- [Search links](https://github.com/gagota/doctolib-scrapper/blob/main/src/scraper.py): `src/scraper.py`
- [Practitioner details](https://github.com/gagota/doctolib-scrapper/blob/main/src/scraper.py): `src/scraper.py`

### Local data

- [YAML data manager](https://github.com/gagota/doctolib-scrapper/blob/main/src/data_manager.py): `src/data_manager.py`
- **YAML data files**

### Other components

- **Local user**
- **Doctolib**

## Connections

- Local user → Run coordinator: configures and runs (evidence: `README.md`)
- Run coordinator → Run configuration: creates (evidence: `src/_main.py`)
- Run coordinator → Logging: creates (evidence: `src/_main.py`)
- Run coordinator → Search links: runs (evidence: `src/_main.py`)
- Run coordinator → Practitioner details: runs (evidence: `src/_main.py`)
- Run coordinator → YAML data manager: loads saved links (evidence: `src/_main.py`)
- Run coordinator → Browser session: opens scraper (evidence: `src/_main.py`)
- Search links → Doctolib: requests search data (evidence: `src/scraper.py`)
- Practitioner details → Doctolib: requests practitioner data (evidence: `src/scraper.py`)
- Browser session → Doctolib: navigates pages (evidence: `src/scraper.py`)
- YAML data manager → YAML data files: reads and writes (evidence: `src/data_manager.py`)
- Run configuration → YAML data files: defines data paths (evidence: `src/config.py`)
