How rupam-1/python-project-etl-weather-data-into-azure-blob works
This repository is a scheduled weather-data ETL script. Its main workflow reads configuration, fetches current weather, optionally writes records locally as CSV, uploads files to Azure Blob Storage, then builds and publishes Markdown previews to GitHub. Logging is a shared internal layer used by the clients and file writer. The README documents the schedule and ETL stages; source excerpts show the concrete calls. The configured CSV path is the local staging area, and the excerpts do not show a separate transformation module.
Main entry points: weather-etl.py ETL runner · logging_client.py Logger manager · weather_client.py Weather client · azure_blob_client.py Blob client
Components 11
ETL workflow
- ETL runner
weather-etl.py - Runtime config
config.ini - CSV writer
helpers/create_file_helper.py
External integrations
- Weather client
helpers/weather_client.py - Blob client
helpers/azure_blob_client.py
Shared services
- Logger manager
helpers/logging_client.py
Preview publishing
- Overview publisher
overview_md_github.py
Other components
- Cron scheduler
- Weather API
- Azure Blob
- GitHub repository
Connections 13
- Cron scheduler → ETL runner: triggers
README.md - ETL runner → Runtime config: reads
weather-etl.py - ETL runner → Weather client: fetches data
weather-etl.py - Weather client → Weather API: requests data
helpers/weather_client.py - ETL runner → CSV writer: writes records
weather-etl.py - ETL runner → Blob client: uploads files
weather-etl.py - Blob client → Azure Blob: stores files
helpers/azure_blob_client.py - ETL runner → Overview publisher: publishes preview
weather-etl.py - Overview publisher → GitHub repository: updates files
overview_md_github.py - ETL runner → Logger manager: creates logger
weather-etl.py - Weather client → Logger manager: creates logger
helpers/weather_client.py - CSV writer → Logger manager: logs writes
helpers/create_file_helper.py - Blob client → Logger manager: creates logger
helpers/azure_blob_client.py