How gagota/doctolib-scrapper works
Doctolib Scraper is a local Python application that collects practitioner data from a Doctolib search and saves it as YAML. The user configures the search and run options; the main entry point creates configuration and logging, then runs link collection and practitioner-detail collection, reusing saved links when link collection is disabled. Browser automation and parsing are implemented in the scraper. The README also describes future Maps and Sheets steps, but says they are not included. Connections reflect the shown entry-point calls and documented behavior; unsampled scraper persistence details are omitted.
Main entry points: _main.py Run coordinator · config.py Run configuration · scraper.py Browser session · data_manager.py YAML data manager
Components 10
Run setup
- Run coordinator
src/_main.py - Run configuration
src/config.py - Logging
src/logger_wrapper.py
Scraping
- Browser session
src/scraper.py - Search links
src/scraper.py - Practitioner details
src/scraper.py
Local data
- YAML data manager
src/data_manager.py - YAML data files
Other components
- Local user
- Doctolib
Connections 12
- Local user → Run coordinator: configures and runs
README.md - Run coordinator → Run configuration: creates
src/_main.py - Run coordinator → Logging: creates
src/_main.py - Run coordinator → Search links: runs
src/_main.py - Run coordinator → Practitioner details: runs
src/_main.py - Run coordinator → YAML data manager: loads saved links
src/_main.py - Run coordinator → Browser session: opens scraper
src/_main.py - Search links → Doctolib: requests search data
src/scraper.py - Practitioner details → Doctolib: requests practitioner data
src/scraper.py - Browser session → Doctolib: navigates pages
src/scraper.py - YAML data manager → YAML data files: reads and writes
src/data_manager.py - Run configuration → YAML data files: defines data paths
src/config.py