# rupam-1/python-project-etl-weather-data-into-azure-blob architecture

> Architecture diagram of the GitHub repository rupam-1/python-project-etl-weather-data-into-azure-blob, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rupam-1/python-project-etl-weather-data-into-azure-blob
- Repository: https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob

## Overview

This repository is a scheduled weather-data ETL script. Its main workflow reads configuration, fetches current weather, optionally writes records locally as CSV, uploads files to Azure Blob Storage, then builds and publishes Markdown previews to GitHub. Logging is a shared internal layer used by the clients and file writer. The README documents the schedule and ETL stages; source excerpts show the concrete calls. The configured CSV path is the local staging area, and the excerpts do not show a separate transformation module.

Main entry points: [`weather-etl.py`](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/weather-etl.py) (ETL runner), [`helpers/logging_client.py`](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/logging_client.py) (Logger manager), [`helpers/weather_client.py`](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/weather_client.py) (Weather client), [`helpers/azure_blob_client.py`](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/azure_blob_client.py) (Blob client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["ETL workflow"]
  node_etl["ETL runner<br/>[weather-etl.py]"]
  node_config["Runtime config<br/>[config.ini]"]
  node_csv_writer["CSV writer"]
end

subgraph group_integrations["External integrations"]
  node_weather["Weather client<br/>[weather_client.py]"]
  node_blob_client["Blob client"]
end

subgraph group_internal["Shared services"]
  node_logger["Logger manager<br/>[logging_client.py]"]
end

subgraph group_publishing["Preview publishing"]
  node_overview["Overview publisher"]
end

node_scheduler(("Cron scheduler"))
node_weather_api["Weather API"]
node_azure[("Azure Blob")]
node_github["GitHub repository"]

node_scheduler -->|"triggers"| node_etl
node_etl -->|"reads"| node_config
node_etl -->|"fetches data"| node_weather
node_weather -->|"requests data"| node_weather_api
node_etl -->|"writes records"| node_csv_writer
node_etl -->|"uploads files"| node_blob_client
node_blob_client -->|"stores files"| node_azure
node_etl -->|"publishes preview"| node_overview
node_overview -->|"updates files"| node_github
node_etl -->|"creates logger"| node_logger
node_weather -->|"creates logger"| node_logger
node_csv_writer -->|"logs writes"| node_logger
node_blob_client -->|"creates logger"| node_logger

click node_etl "https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/weather-etl.py"
click node_config "https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/config.ini"
click node_weather "https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/weather_client.py"
click node_csv_writer "https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/create_file_helper.py"
click node_blob_client "https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/azure_blob_client.py"
click node_logger "https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/logging_client.py"
click node_overview "https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/overview_md_github.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_etl,node_config,node_csv_writer toneBlue
class node_weather,node_blob_client,node_azure toneAmber
class node_logger,node_weather_api toneMint
class node_overview,node_scheduler toneRose
class node_github toneIndigo
```

## Components

### ETL workflow

- [ETL runner](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/weather-etl.py): `weather-etl.py`
- [Runtime config](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/config.ini): `config.ini`
- [CSV writer](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/create_file_helper.py): `helpers/create_file_helper.py`

### External integrations

- [Weather client](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/weather_client.py): `helpers/weather_client.py`
- [Blob client](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/azure_blob_client.py): `helpers/azure_blob_client.py`

### Shared services

- [Logger manager](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/helpers/logging_client.py): `helpers/logging_client.py`

### Preview publishing

- [Overview publisher](https://github.com/rupam-1/python-project-etl-weather-data-into-azure-blob/blob/main/overview_md_github.py): `overview_md_github.py`

### Other components

- **Cron scheduler**
- **Weather API**
- **Azure Blob**
- **GitHub repository**

## Connections

- Cron scheduler → ETL runner: triggers (evidence: `README.md`)
- ETL runner → Runtime config: reads (evidence: `weather-etl.py`)
- ETL runner → Weather client: fetches data (evidence: `weather-etl.py`)
- Weather client → Weather API: requests data (evidence: `helpers/weather_client.py`)
- ETL runner → CSV writer: writes records (evidence: `weather-etl.py`)
- ETL runner → Blob client: uploads files (evidence: `weather-etl.py`)
- Blob client → Azure Blob: stores files (evidence: `helpers/azure_blob_client.py`)
- ETL runner → Overview publisher: publishes preview (evidence: `weather-etl.py`)
- Overview publisher → GitHub repository: updates files (evidence: `overview_md_github.py`)
- ETL runner → Logger manager: creates logger (evidence: `weather-etl.py`)
- Weather client → Logger manager: creates logger (evidence: `helpers/weather_client.py`)
- CSV writer → Logger manager: logs writes (evidence: `helpers/create_file_helper.py`)
- Blob client → Logger manager: creates logger (evidence: `helpers/azure_blob_client.py`)