# cywhale/woa23 architecture

> Architecture diagram of the GitHub repository cywhale/woa23, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cywhale/woa23
- Repository: https://github.com/cywhale/woa23

## Overview

##### Project type and purpose
- This is a Python-based microservice that exposes a REST API (“woa23_app.py”) for querying World Ocean Atlas 2023 (WOA23) data.
- It uses Dask for parallel data slicing, serves JSON or CSV, and provides Swagger/OpenAPI docs.

##### File structure
- Top level
- woa23_app.py – the main Flask/FastAPI entry point
- setup.py, Pipfile, requirements.txt – packaging and dependency management
- conf/ – deployment scripts (ecosystem.config.js for PM2, start_app.sh)
- src/ – core modules: config, dask_client_manager, woa23_utils
- dev/ – data-preparation notebooks and utilities (zarr conversion, PostGIS ingestion)
- figs/ – example output images
- Architectural hints
- “src/dask_client_manager.py”: the project spins up or connects to a Dask cluster for compute
- “src/woa23_utils.py”: encapsulates data-access logic (reading from Zarr stores)
- Deployment scripts imply a managed process (PM2 or shell) hosting the Python API

##### README for additional insights
- Confirms it’s an HTTP API with two endpoints (JSON and CSV)
- References Swagger UI (auto-generated from FastAPI/Flask + OpenAPI)
- Data is pre-downloaded or fetched from NOAA WOA23 and stored as Zarr or PostGIS

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Systems
    subgraph "External Systems"
        direction TB
        Clients["Clients (curl, Web UI, Swagger UI)"]:::external
        NOAA["NOAA WOA23 Archives"]:::external
    end

    %% Deployment Layer
    subgraph "Deployment Layer"
        direction TB
        Nginx["Nginx / Load Balancer"]:::deploy
        PM2["PM2 / Process Manager"]:::deploy
    end

    Clients -->|"HTTP GET JSON/CSV"| Nginx
    Nginx --> PM2

    %% API Service
    subgraph "API Service"
        direction TB
        Router["Router / HTTP Server"]:::compute
        Config["Config Module"]:::compute
        DaskMgr["Dask Client Manager"]:::compute
        BizLogic["woa23_utils (Data Access)"]:::compute
        Router -->|"loads config"| Config
        Router -->|"schedules tasks"| DaskMgr
        Router -->|"invokes business logic"| BizLogic
    end

    PM2 --> Router

    %% Dask Cluster
    subgraph "Dask Cluster"
        direction TB
        Scheduler["Scheduler"]:::compute
        Workers["Workers"]:::compute
    end

    DaskMgr -->|"Dask RPC"| Scheduler
    Scheduler --> Workers

    %% Storage Layer
    subgraph "Storage Layer"
        direction TB
        Zarr["Zarr Store"]:::storage
        PostGIS["PostGIS DB"]:::storage
    end

    Workers -->|"reads/writes"| Zarr
    Workers -->|"reads/writes"| PostGIS
    BizLogic -->|"reads"| Zarr
    BizLogic -->|"reads"| PostGIS
    Router -->|"returns JSON/CSV"| Clients

    %% Batch Data Preparation
    NOAA -.->|"batch download"| Batch["Batch Data Preparation"]:::compute
    subgraph "Batch Data Preparation"
        direction TB
        Download["woa23_download_batch.py"]:::compute
        ZarrWrite["zarr_parallel_write_woa23.py"]:::compute
        PostIngest["zarr2postgis.ipynb"]:::compute
        NB["*.ipynb (dev notebooks)"]:::compute
        Download --> ZarrWrite --> PostIngest
        NB
        Batch --> Download
    end

    %% Project Artifacts
    subgraph "Project Artifacts"
        direction TB
        SetupPy["setup.py"]:::deploy
        Pipfile["Pipfile"]:::deploy
        Req["requirements.txt"]:::deploy
        Docs["README.md"]:::deploy
        Figures["Example Outputs (figs/)"]:::external
    end

    %% Click Events
    click Router "https://github.com/cywhale/woa23/blob/main/woa23_app.py"
    click Config "https://github.com/cywhale/woa23/blob/main/src/config.py"
    click DaskMgr "https://github.com/cywhale/woa23/blob/main/src/dask_client_manager.py"
    click BizLogic "https://github.com/cywhale/woa23/blob/main/src/woa23_utils.py"
    click PM2 "https://github.com/cywhale/woa23/blob/main/conf/ecosystem.config.js"
    click PM2 "https://github.com/cywhale/woa23/blob/main/conf/start_app.sh"
    click PM2 "https://github.com/cywhale/woa23/blob/main/conf/simu.sh"
    click SetupPy "https://github.com/cywhale/woa23/blob/main/setup.py"
    click Pipfile "https://github.com/cywhale/woa23/tree/main/Pipfile"
    click Req "https://github.com/cywhale/woa23/blob/main/requirements.txt"
    click Docs "https://github.com/cywhale/woa23/blob/main/README.md"
    click Figures "https://github.com/cywhale/woa23/tree/main/figs/"
    click Download "https://github.com/cywhale/woa23/blob/main/dev/woa23_download_batch.py"
    click ZarrWrite "https://github.com/cywhale/woa23/blob/main/dev/zarr_parallel_write_woa23.py"
    click PostIngest "https://github.com/cywhale/woa23/blob/main/dev/zarr2postgis.ipynb"
    click NB "https://github.com/cywhale/woa23/blob/main/dev/*.ipynb"

    %% Styles
    classDef compute fill:#8ecae6,stroke:#219ebc,color:#023047;
    classDef storage fill:#a7c957,stroke:#588157,color:#3a5a40;
    classDef deploy fill:#dee2e6,stroke:#adb5bd,color:#495057;
    classDef external fill:#ffb703,stroke:#fb8500,color:#662506;
```
