# peddiaravind/weather_api_etl architecture

> Architecture diagram of the GitHub repository peddiaravind/weather_api_etl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/peddiaravind/weather_api_etl
- Repository: https://github.com/peddiaravind/weather_api_etl

## Overview

This repository is a weather-data ETL application. Running `main.py` creates the MySQL table, fetches current conditions for the default city, transforms the API JSON into a one-row Pandas DataFrame, loads it into MySQL, and exports the DataFrame to CSV. Configuration and logging support the runtime stages. The workflow is based on `main.py` and the sampled stage implementations; README-listed capabilities such as CSV export are included, while unsampled behavior is not inferred.

Main entry points: [`main.py`](https://github.com/peddiaravind/weather_api_etl/blob/main/main.py) (Pipeline runner), [`extract.py`](https://github.com/peddiaravind/weather_api_etl/blob/main/extract.py) (Weather extraction), [`load.py`](https://github.com/peddiaravind/weather_api_etl/blob/main/load.py) (MySQL loading), [`db/connection.py`](https://github.com/peddiaravind/weather_api_etl/blob/main/db/connection.py) (Database connection)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["ETL orchestration"]
  node_main["Pipeline runner<br/>[main.py]"]
  node_csv["CSV export<br/>[main.py]"]
end

subgraph group_stages["Weather data stages"]
  node_extract["Weather extraction<br/>[extract.py]"]
  node_transform["Data transformation<br/>[transform.py]"]
  node_load["MySQL loading<br/>[load.py]"]
end

subgraph group_storage["Persistence"]
  node_table_setup["Table setup<br/>[create_tables.py]"]
  node_connection["Database connection<br/>[connection.py]"]
end

subgraph group_shared["Shared services"]
  node_config["Runtime configuration<br/>[config.py]"]
  node_logger["ETL logger<br/>[logger.py]"]
end

node_operator(("Pipeline operator"))
node_weather_api["WeatherAPI"]
node_mysql[("MySQL database")]
node_environment["Environment variables"]

node_operator -->|"runs"| node_main
node_main -->|"creates table"| node_table_setup
node_main -->|"requests data"| node_extract
node_extract -->|"fetches JSON"| node_weather_api
node_extract --> node_transform
node_main -->|"transforms data"| node_transform
node_main -->|"loads DataFrame"| node_load
node_main -->|"exports CSV"| node_csv
node_table_setup -->|"requests connection"| node_connection
node_load -->|"requests connection"| node_connection
node_connection -->|"reads settings"| node_config
node_config -->|"reads variables"| node_environment
node_connection -->|"connects"| node_mysql
node_table_setup -->|"creates table"| node_mysql
node_load -->|"inserts rows"| node_mysql
node_extract -->|"reads API key"| node_config
node_extract -->|"logs events"| node_logger
node_transform -->|"logs events"| node_logger
node_load -->|"logs events"| node_logger

click node_main "https://github.com/peddiaravind/weather_api_etl/blob/main/main.py"
click node_extract "https://github.com/peddiaravind/weather_api_etl/blob/main/extract.py"
click node_transform "https://github.com/peddiaravind/weather_api_etl/blob/main/transform.py"
click node_load "https://github.com/peddiaravind/weather_api_etl/blob/main/load.py"
click node_csv "https://github.com/peddiaravind/weather_api_etl/blob/main/main.py"
click node_table_setup "https://github.com/peddiaravind/weather_api_etl/blob/main/db/create_tables.py"
click node_connection "https://github.com/peddiaravind/weather_api_etl/blob/main/db/connection.py"
click node_config "https://github.com/peddiaravind/weather_api_etl/blob/main/config/config.py"
click node_logger "https://github.com/peddiaravind/weather_api_etl/blob/main/utils/logger.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_csv toneBlue
class node_extract,node_transform,node_load,node_mysql toneAmber
class node_table_setup,node_connection,node_weather_api toneMint
class node_config,node_logger toneRose
class node_operator,node_environment toneIndigo
```

## Components

### ETL orchestration

- [Pipeline runner](https://github.com/peddiaravind/weather_api_etl/blob/main/main.py): `main.py`
- [CSV export](https://github.com/peddiaravind/weather_api_etl/blob/main/main.py): `main.py`

### Weather data stages

- [Weather extraction](https://github.com/peddiaravind/weather_api_etl/blob/main/extract.py): `extract.py`
- [Data transformation](https://github.com/peddiaravind/weather_api_etl/blob/main/transform.py): `transform.py`
- [MySQL loading](https://github.com/peddiaravind/weather_api_etl/blob/main/load.py): `load.py`

### Persistence

- [Table setup](https://github.com/peddiaravind/weather_api_etl/blob/main/db/create_tables.py): `db/create_tables.py`
- [Database connection](https://github.com/peddiaravind/weather_api_etl/blob/main/db/connection.py): `db/connection.py`

### Shared services

- [Runtime configuration](https://github.com/peddiaravind/weather_api_etl/blob/main/config/config.py): `config/config.py`
- [ETL logger](https://github.com/peddiaravind/weather_api_etl/blob/main/utils/logger.py): `utils/logger.py`

### Other components

- **Pipeline operator**
- **WeatherAPI**
- **MySQL database**
- **Environment variables**

## Connections

- Pipeline operator → Pipeline runner: runs
- Pipeline runner → Table setup: creates table
- Pipeline runner → Weather extraction: requests data
- Weather extraction → WeatherAPI: fetches JSON
- Weather extraction → Data transformation
- Pipeline runner → Data transformation: transforms data
- Pipeline runner → MySQL loading: loads DataFrame
- Pipeline runner → CSV export: exports CSV
- Table setup → Database connection: requests connection
- MySQL loading → Database connection: requests connection
- Database connection → Runtime configuration: reads settings
- Runtime configuration → Environment variables: reads variables
- Database connection → MySQL database: connects
- Table setup → MySQL database: creates table
- MySQL loading → MySQL database: inserts rows
- Weather extraction → Runtime configuration: reads API key
- Weather extraction → ETL logger: logs events
- Data transformation → ETL logger: logs events
- MySQL loading → ETL logger: logs events