# ahmed-abdelwahed1/hevy-flow architecture

> Architecture diagram of the GitHub repository ahmed-abdelwahed1/hevy-flow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ahmed-abdelwahed1/hevy-flow
- Repository: https://github.com/ahmed-abdelwahed1/hevy-flow

## Overview

1. **What this repo is**
   - Hevy Flow is a Python-based analytics pipeline for workout logs.
   - It has two main runtime paths: an ETL job that prepares and loads data, and a Streamlit dashboard that reads the loaded data for visualization.

2. **Primary data source**
   - The raw input is a CSV export from Hevy stored in `data/workouts.csv`.
   - This is the ingestion boundary: everything upstream of the database starts from that local file.

3. **ETL subsystem**
   - The `etl/` package contains the data pipeline logic:
     - `extract.py` reads and validates the CSV input.
     - `transform.py` cleans and reshapes the workout records.
     - `load.py` writes the processed data into the database.
   - This is a staged pipeline, so data flows linearly from file ingestion to transformation to persistence.

4. **Pipeline entrypoint**
   - `main.py` is the orchestration layer for the ETL job.
   - It likely coordinates the three ETL phases rather than containing business logic itself, which keeps execution flow separate from transformation details.

5. **Storage boundary**
   - The destination is Supabase, used here as a PostgreSQL-backed warehouse.
   - The loader in `etl/load.py` is the only component that writes to this store, making it the main persistence boundary in the system.

6. **Analytics/UI subsystem**
   - `dashboard.py` is the presentation layer.
   - It uses Streamlit for the app runtime and Plotly for charts, so the dashboard is a consumer of the database rather than part of the ETL pipeline.

7. **Configuration and runtime setup**
   - `config.py` centralizes environment-dependent settings.
   - `.env.example` documents the required secrets/config values, and `python-dotenv` is used to load them at runtime.
   - `environment.yml` defines the Conda-based Python environment, including the main data and database libraries.

8. **Core dependency model**
   - `pandas` is the main in-memory data processing library across extract/transform/load.
   - `psycopg2-binary` is the database adapter for PostgreSQL/Supabase access.
   - Streamlit and Plotly are only used in the dashboard layer, not in ETL.

9. **Data flow across the repo**
   - The end-to-end path is: CSV export → ETL extraction → cleansing/transformation → database load → dashboard reads from database.
   - The dashboard does not appear to consume the CSV directly, which keeps reporting decoupled from raw ingestion.

10. **Testing boundaries**
   - `tests/` contains focused tests for extract, transform, load, and main orchestration.
   - This suggests the repo treats each ETL stage as a testable unit, with separate coverage for the pipeline entrypoint.

11. **CI / delivery**
   - `.github/workflows/ci.yml` indicates automated CI is in place.
   - Architecturally, this supports regression checks around the ETL pipeline and its interfaces, especially useful because the system depends on data shape and database connectivity.

12. **How to draw the architecture**
   - Draw three major zones: local raw data, ETL processing, and Supabase storage, with the dashboard as a separate consumer layer.
   - Emphasize that `main.py` drives the batch pipeline, while `dashboard.py` is an independent read path into the same database.

Main entry points: [`etl/extract.py`](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/etl/extract.py) (extract), [`main.py`](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/main.py) (main), [`dashboard.py`](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/dashboard.py) (dashboard), [`etl/transform.py`](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/etl/transform.py) (transform)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_raw["Raw data"]
  node_n_raw_csv["workouts.csv<br/>csv export<br/>[workouts.csv]"]
end

subgraph group_g_etl["ETL pipeline"]
  node_n_main["main<br/>etl orchestrator<br/>[main.py]"]
  node_n_extract["extract<br/>csv ingest<br/>[extract.py]"]
  node_n_transform["transform<br/>data shaping<br/>[transform.py]"]
  node_n_load["load<br/>db writer<br/>[load.py]"]
  node_n_pandas(("pandas<br/>dataframe library"))
  node_n_supabase[("Supabase<br/>warehouse")]
end

subgraph group_g_app["Analytics UI"]
  node_n_dashboard["dashboard<br/>streamlit app<br/>[dashboard.py]"]
  node_n_streamlit(("Streamlit<br/>ui runtime"))
  node_n_plotly(("Plotly<br/>charting"))
end

subgraph group_g_ops["Ops & config"]
  node_n_config["config<br/>[config.py]"]
  node_n_dotenv(("dotenv<br/>env loader"))
  node_n_env["environment.yml<br/>conda env<br/>[environment.yml]"]
  node_n_ci["CI workflow<br/>automation<br/>[ci.yml]"]
  node_n_tests["tests<br/>test suite"]
end

node_n_raw_csv -->|"ingests"| node_n_extract
node_n_main -.->|"runs"| node_n_extract
node_n_main -.->|"runs"| node_n_transform
node_n_main -.->|"runs"| node_n_load
node_n_extract -->|"uses"| node_n_pandas
node_n_extract -->|"passes data"| node_n_transform
node_n_transform -->|"uses"| node_n_pandas
node_n_transform -->|"produces"| node_n_load
node_n_load -->|"writes"| node_n_supabase
node_n_dashboard -->|"reads"| node_n_supabase
node_n_dashboard -->|"built on"| node_n_streamlit
node_n_dashboard -->|"charts with"| node_n_plotly
node_n_config -.->|"configures"| node_n_extract
node_n_config -.->|"configures"| node_n_load
node_n_config -.->|"configures"| node_n_dashboard
node_n_dotenv -->|"loads"| node_n_config
node_n_env -.->|"includes"| node_n_pandas
node_n_env -.->|"includes"| node_n_streamlit
node_n_ci -.->|"runs"| node_n_tests
node_n_ci -.->|"checks"| node_n_main
node_n_ci -.->|"checks"| node_n_dashboard
node_n_tests -.->|"covers"| node_n_extract
node_n_tests -.->|"covers"| node_n_transform
node_n_tests -.->|"covers"| node_n_load
node_n_tests -.->|"covers"| node_n_main

click node_n_raw_csv "https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/data/workouts.csv"
click node_n_main "https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/main.py"
click node_n_extract "https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/etl/extract.py"
click node_n_transform "https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/etl/transform.py"
click node_n_load "https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/etl/load.py"
click node_n_dashboard "https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/dashboard.py"
click node_n_config "https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/config.py"
click node_n_env "https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/environment.yml"
click node_n_ci "https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/.github/workflows/ci.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_raw_csv toneBlue
class node_n_main,node_n_extract,node_n_transform,node_n_load,node_n_pandas,node_n_supabase toneAmber
class node_n_dashboard,node_n_streamlit,node_n_plotly toneMint
class node_n_config,node_n_dotenv,node_n_env,node_n_ci,node_n_tests toneRose
```

## Components

### Raw data

Local workout export ingested by the pipeline.

- [workouts.csv](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/data/workouts.csv): csv export · Raw Hevy workout export consumed by the ETL job. · `data/workouts.csv`

### ETL pipeline

Batch processing stages and orchestration.

- [main](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/main.py): etl orchestrator · Entry point that coordinates the batch pipeline. · `main.py`
- [extract](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/etl/extract.py): csv ingest · Reads and validates the raw workout CSV. · `etl/extract.py`
- [transform](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/etl/transform.py): data shaping · Cleans and reshapes workout records for loading. · `etl/transform.py`
- [load](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/etl/load.py): db writer · Persists processed workout data into Supabase/PostgreSQL. · `etl/load.py`
- **pandas**: dataframe library · Primary in-memory data processing dependency for ETL.
- **Supabase**: warehouse · PostgreSQL-backed storage layer for processed analytics data.

### Analytics UI

Read-only dashboard and its presentation stack.

- [dashboard](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/dashboard.py): streamlit app · Interactive analytics UI that reads from the database. · `dashboard.py`
- **Streamlit**: ui runtime · Application framework used to run the dashboard.
- **Plotly**: charting · Visualization library used for dashboard charts.

### Ops & config

Environment, secrets, and CI support.

- [config](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/config.py): runtime config · Centralized environment-dependent settings. · `config.py`
- **dotenv**: env loader · Loads secrets and configuration from environment files.
- [environment.yml](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/environment.yml): conda env · Defines the Python environment and core dependencies. · `environment.yml`
- [CI workflow](https://github.com/ahmed-abdelwahed1/hevy-flow/blob/main/.github/workflows/ci.yml): automation · Automated checks for pipeline and repository health. · `.github/workflows/ci.yml`
- **tests**: test suite · Focused coverage for extraction, transformation, loading, and orchestration.

## Connections

- workouts.csv → extract: ingests · Raw CSV is the ETL input boundary.
- main → extract: runs · Main entrypoint starts extraction.
- main → transform: runs · Main entrypoint coordinates transformation.
- main → load: runs · Main entrypoint coordinates loading.
- extract → pandas: uses · Extraction works with tabular data frames.
- extract → transform: passes data · Validated records flow into transformation.
- transform → pandas: uses · Transformation relies on pandas operations.
- transform → load: produces · Cleaned records are handed to loading.
- load → Supabase: writes · Loader persists processed data to Supabase/PostgreSQL.
- dashboard → Supabase: reads · Dashboard queries the loaded analytics store.
- dashboard → Streamlit: built on · Streamlit provides the app runtime.
- dashboard → Plotly: charts with · Plotly renders interactive visualizations.
- config → extract: configures · Shared settings affect pipeline behavior.
- config → load: configures · Connection and load settings feed the writer.
- config → dashboard: configures · Dashboard runtime uses shared environment settings.
- dotenv → config: loads · Environment variables are loaded into config.
- environment.yml → pandas: includes · Environment specification bundles core data libraries.
- environment.yml → Streamlit: includes · Environment specification includes the dashboard runtime.
- CI workflow → tests: runs · CI likely executes the repository test suite.
- CI workflow → main: checks · CI helps protect the batch pipeline entrypoint.
- CI workflow → dashboard: checks · CI helps protect the dashboard app boundary.
- tests → extract: covers · Tests validate extraction behavior.
- tests → transform: covers · Tests validate transformation behavior.
- tests → load: covers · Tests validate loading behavior.
- tests → main: covers · Tests validate pipeline orchestration.