# bdelion/web-scraper architecture

> Architecture diagram of the GitHub repository bdelion/web-scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bdelion/web-scraper
- Repository: https://github.com/bdelion/web-scraper

## Overview

The repository’s active product is a Node.js weather-scraping CLI. Its main workflow reads date ranges from Excel or accepts a custom range, resolves a station, fetches and formats observations, then writes Excel or prints JSON. README also describes scraping tutorials for other sites and Excel examples, represented as a separate exploratory branch. The map is based on the supplied source excerpts and tree; wiring inside unsampled exploratory files is not inferred.

Main entry points: [`src/meteo-scraping-file-input.js`](https://github.com/bdelion/web-scraper/blob/main/src/meteo-scraping-file-input.js) (Command entry), [`src/scrapers/weatherScraper.js`](https://github.com/bdelion/web-scraper/blob/main/src/scrapers/weatherScraper.js) (Range retrieval), [`src/utils/excelUtils.js`](https://github.com/bdelion/web-scraper/blob/main/src/utils/excelUtils.js) (Excel I/O), [`src/utils/dateHourUtils.js`](https://github.com/bdelion/web-scraper/blob/main/src/utils/dateHourUtils.js) (Date conversions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cli["CLI workflow"]
  node_entry["Command entry"]
  node_input["Input date ranges"]
  node_output["Excel or JSON output"]
end

subgraph group_scraping["Weather scraping"]
  node_format["Range formatting<br/>[weatherScraper.js]"]
  node_station["Station lookup<br/>[weatherScraper.js]"]
  node_ranges["Range retrieval<br/>[weatherScraper.js]"]
  node_observations["Daily observations<br/>[weatherScraper.js]"]
  node_results["Weather results"]
end

subgraph group_support["Date and file utilities"]
  node_excelutils["Excel I/O<br/>[excelUtils.js]"]
  node_dateutils["Date conversions<br/>[dateHourUtils.js]"]
  node_dayjs["Date configuration<br/>[dayjsConfig.js]"]
  node_math["Temperature statistics<br/>[mathUtils.js]"]
  node_logging["Logging<br/>[logUtils.js]"]
  node_errors["Scraping errors<br/>[customErrors.js]"]
end

subgraph group_examples["Exploratory examples"]
  node_tutorials["Scraping tutorials"]
end

node_user(("CLI user"))
node_excel["Excel files"]
node_meteociel["Meteociel"]
node_tutorialsites["Tutorial target sites"]

node_user -->|"runs commands"| node_entry
node_entry -->|"reads input"| node_excelutils
node_excelutils -->|"reads or writes"| node_excel
node_entry -->|"formats ranges"| node_format
node_entry -->|"resolves station"| node_station
node_entry -->|"fetches ranges"| node_ranges
node_entry -->|"converts dates"| node_dateutils
node_dateutils -->|"uses configuration"| node_dayjs
node_ranges -->|"fetches days"| node_observations
node_station -->|"requests station ID"| node_meteociel
node_observations -->|"requests observations"| node_meteociel
node_ranges -->|"returns weather data"| node_results
node_observations -->|"produces observations"| node_results
node_entry -->|"writes results"| node_excelutils
node_entry -->|"prints JSON"| node_output
node_results -->|"supplies results"| node_output
node_tutorials -.->|"demonstrate scraping"| node_tutorialsites

click node_entry "https://github.com/bdelion/web-scraper/blob/main/src/meteo-scraping-file-input.js"
click node_format "https://github.com/bdelion/web-scraper/blob/main/src/scrapers/weatherScraper.js"
click node_station "https://github.com/bdelion/web-scraper/blob/main/src/scrapers/weatherScraper.js"
click node_ranges "https://github.com/bdelion/web-scraper/blob/main/src/scrapers/weatherScraper.js"
click node_observations "https://github.com/bdelion/web-scraper/blob/main/src/scrapers/weatherScraper.js"
click node_excelutils "https://github.com/bdelion/web-scraper/blob/main/src/utils/excelUtils.js"
click node_dateutils "https://github.com/bdelion/web-scraper/blob/main/src/utils/dateHourUtils.js"
click node_dayjs "https://github.com/bdelion/web-scraper/blob/main/src/config/dayjsConfig.js"
click node_math "https://github.com/bdelion/web-scraper/blob/main/src/utils/mathUtils.js"
click node_logging "https://github.com/bdelion/web-scraper/blob/main/src/utils/logUtils.js"
click node_errors "https://github.com/bdelion/web-scraper/blob/main/src/errors/customErrors.js"
click node_tutorials "https://github.com/bdelion/web-scraper/tree/main/src/explorations"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_input,node_output,node_user toneBlue
class node_format,node_station,node_ranges,node_observations,node_results toneAmber
class node_excelutils,node_dateutils,node_dayjs,node_math,node_logging,node_errors toneMint
class node_tutorials toneRose
class node_excel,node_meteociel,node_tutorialsites toneIndigo
```

## Components

### CLI workflow

- [Command entry](https://github.com/bdelion/web-scraper/blob/main/src/meteo-scraping-file-input.js): `src/meteo-scraping-file-input.js`
- **Input date ranges**
- **Excel or JSON output**

### Weather scraping

- [Range formatting](https://github.com/bdelion/web-scraper/blob/main/src/scrapers/weatherScraper.js): `src/scrapers/weatherScraper.js`
- [Station lookup](https://github.com/bdelion/web-scraper/blob/main/src/scrapers/weatherScraper.js): `src/scrapers/weatherScraper.js`
- [Range retrieval](https://github.com/bdelion/web-scraper/blob/main/src/scrapers/weatherScraper.js): `src/scrapers/weatherScraper.js`
- [Daily observations](https://github.com/bdelion/web-scraper/blob/main/src/scrapers/weatherScraper.js): `src/scrapers/weatherScraper.js`
- **Weather results**

### Date and file utilities

- [Excel I/O](https://github.com/bdelion/web-scraper/blob/main/src/utils/excelUtils.js): `src/utils/excelUtils.js`
- [Date conversions](https://github.com/bdelion/web-scraper/blob/main/src/utils/dateHourUtils.js): `src/utils/dateHourUtils.js`
- [Date configuration](https://github.com/bdelion/web-scraper/blob/main/src/config/dayjsConfig.js): `src/config/dayjsConfig.js`
- [Temperature statistics](https://github.com/bdelion/web-scraper/blob/main/src/utils/mathUtils.js): `src/utils/mathUtils.js`
- [Logging](https://github.com/bdelion/web-scraper/blob/main/src/utils/logUtils.js): `src/utils/logUtils.js`
- [Scraping errors](https://github.com/bdelion/web-scraper/blob/main/src/errors/customErrors.js): `src/errors/customErrors.js`

### Exploratory examples

- [Scraping tutorials](https://github.com/bdelion/web-scraper/tree/main/src/explorations): `src/explorations`

### Other components

- **CLI user**
- **Excel files**
- **Meteociel**
- **Tutorial target sites**

## Connections

- CLI user → Command entry: runs commands
- Command entry → Excel I/O: reads input
- Excel I/O → Excel files: reads or writes
- Command entry → Range formatting: formats ranges
- Command entry → Station lookup: resolves station
- Command entry → Range retrieval: fetches ranges
- Command entry → Date conversions: converts dates
- Date conversions → Date configuration: uses configuration
- Range retrieval → Daily observations: fetches days
- Station lookup → Meteociel: requests station ID
- Daily observations → Meteociel: requests observations
- Range retrieval → Weather results: returns weather data
- Daily observations → Weather results: produces observations
- Command entry → Excel I/O: writes results
- Command entry → Excel or JSON output: prints JSON
- Weather results → Excel or JSON output: supplies results
- Scraping tutorials → Tutorial target sites: demonstrate scraping