# wateraccounting/wa architecture

> Architecture diagram of the GitHub repository wateraccounting/wa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wateraccounting/wa
- Repository: https://github.com/wateraccounting/wa

## Overview

##### Project Type and Purpose
- WA+ is a monolithic Python toolbox for automating the retrieval, processing, modeling, and reporting of hydrologic and remote-sensing data.
- It consists of •Data Collection•, •Data Processing/Utilities•, •Hydrologic Modeling•, •Product Aggregation•, and •Reporting/Export• layers, all orchestrated via a central entry point.

##### Break Down the Major Architectural Components
A. Entry Point / Orchestration
– WebAccounts.py (CLI script that drives the entire pipeline)
B. Data-Collection Layer
– Collect/… subpackages (ALEXI, ASCAT, CFSR, CHIRPS, …)
– Each subpackage contains DataAccess*.py plus per-cadence scripts (daily.py, monthly.py, etc.)
– Pulls raw data from external services (e.g. HTTP downloads, FTP) and writes NetCDF/CSV
C. Pre-Processing & Utility Layer
– Functions/… (flux splitting, dictionary filling, time aggregators)
– General/… (data conversions, raster handling)
– These modules transform raw files into standardized inputs for models
D. Modeling Layer
– Models/SurfWAT (channel routing, reservoir, withdrawals)
– Models/waterpix (GIS-based water-pixel modeling, Arcpy & GDAL variants)
– Each model reads processed data, runs simulations, outputs result files
E. Product Aggregation Layer
– Products/ETens (ensemble evapotranspiration products)
– Products/ETref (reference ET workflows)
– Aggregates model outputs into higher-level products (daily/monthly)
F. Reporting/Export Layer
– Generator/Sheet2, Sheet3, Sheet4, Sheet5 (CSV + PDF generators)
– Sheets/*.py (scripts to assemble sample sheets)
– Final output: PDF reports, CSV tables for end users
G. Documentation
– docs/ManualWA.md + docs/figs for user guidance (not in run-time path)

##### Component Interactions
1. User runs WebAccounts.py with parameters →
2. WebAccounts calls Collect → for each data source:
→ External service (HTTP/FTP)
→ DataAccess → writes raw files to disk/storage
3. Functions & General modules read raw files → produce cleaned, aggregated input files
4. Models (SurfWAT / waterpix) ingest cleaned inputs → output simulation NetCDFs
5. Products modules pick up simulation outputs → compute ETens / ETref products
6. Generator & Sheets consume final products → produce PDF/CSV deliverables
7. User collects deliverables; repeat pipeline as needed

##### Architectural Patterns & Design Principles
- Pipeline Data-flow: strictly layered—data moves one way from collection to reporting
- Modular Packaging: each data source, function group, and model lives in its own subpackage
- Separation of Concerns:
– Collectors know only about fetching data
– Utility modules know only about data transformations
– Models focus exclusively on simulation logic
– Generators focus only on presentation
- Extensibility points: add a new collector by dropping a new subfolder under Collect/, add a new model under Models/

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Entry Point / Orchestration"
        WP["WebAccounts.py"]:::orchestr
    end

    subgraph "Data-Collection Layer"
        CT["Collect_Tools.py"]:::datacollection
        COL["Collect/"]:::datacollection
        HTTP["HTTP Service"]:::external
        FTP["FTP Service"]:::external
    end

    subgraph "Pre-Processing & Utility Layer"
        FUNC["Functions/"]:::preprocessing
        GEN["General/"]:::preprocessing
    end

    subgraph "Hydrologic Modeling Layer"
        SW["SurfWAT"]:::modeling
        WPX["waterpix"]:::modeling
    end

    subgraph "Product Aggregation Layer"
        ETENS["ETens"]:::products
        ETREF["ETref"]:::products
    end

    subgraph "Reporting/Export Layer"
        S2["Generator/Sheet2"]:::reporting
        S3["Generator/Sheet3"]:::reporting
        S4["Generator/Sheet4"]:::reporting
        S5["Generator/Sheet5"]:::reporting
        SH["Sheets/"]:::reporting
    end

    subgraph "Documentation"
        DOC["docs/"]:::doc
    end

    WP -->|invokes| COL
    WP -->|uses| CT

    HTTP -->|provides raw NetCDF| COL
    FTP -->|provides raw NetCDF| COL

    COL -->|raw NetCDF| FUNC
    COL -->|raw NetCDF| GEN

    FUNC -->|cleaned CSV| SW
    GEN -->|cleaned CSV| SW
    FUNC -->|cleaned CSV| WPX
    GEN -->|cleaned CSV| WPX

    SW -->|model outputs| ETENS
    SW -->|model outputs| ETREF
    WPX -->|model outputs| ETENS
    WPX -->|model outputs| ETREF

    ETENS -->|ET products| S2
    ETENS -->|ET products| S3
    ETENS -->|ET products| S4
    ETENS -->|ET products| S5
    ETREF -->|ET products| S2
    ETREF -->|ET products| S3
    ETREF -->|ET products| S4
    ETREF -->|ET products| S5

    S2 -->|sample sheets| SH
    S3 -->|sample sheets| SH
    S4 -->|sample sheets| SH
    S5 -->|sample sheets| SH

    WP -->|docs reference| DOC

    classDef datacollection fill:#d0e7ff,stroke:#3291ff,color:#000
    classDef preprocessing fill:#fff3bf,stroke:#f0c000,color:#000
    classDef modeling fill:#d3f9d8,stroke:#37b24d,color:#000
    classDef products fill:#e5dbff,stroke:#7048e8,color:#000
    classDef reporting fill:#ffe3e3,stroke:#fa5252,color:#000
    classDef external fill:#f1f3f5,stroke:#748ffc,color:#000,stroke-dasharray: 5 5
    classDef orchestr fill:#ffecd1,stroke:#d9480f,color:#000
    classDef doc fill:#f8f9fa,stroke:#adb5bd,color:#000,stroke-dasharray: 2 2

    click WP "https://github.com/wateraccounting/wa/blob/master/WebAccounts.py"
    click COL "https://github.com/wateraccounting/wa/tree/master/Collect/"
    click CT "https://github.com/wateraccounting/wa/blob/master/Collect_Tools.py"
    click FUNC "https://github.com/wateraccounting/wa/tree/master/Functions/"
    click GEN "https://github.com/wateraccounting/wa/tree/master/General/"
    click SW "https://github.com/wateraccounting/wa/tree/master/Models/SurfWAT/"
    click WPX "https://github.com/wateraccounting/wa/tree/master/Models/waterpix/"
    click ETENS "https://github.com/wateraccounting/wa/tree/master/Products/ETens/"
    click ETREF "https://github.com/wateraccounting/wa/tree/master/Products/ETref/"
    click S2 "https://github.com/wateraccounting/wa/tree/master/Generator/Sheet2/"
    click S3 "https://github.com/wateraccounting/wa/tree/master/Generator/Sheet3/"
    click S4 "https://github.com/wateraccounting/wa/tree/master/Generator/Sheet4/"
    click S5 "https://github.com/wateraccounting/wa/tree/master/Generator/Sheet5/"
    click SH "https://github.com/wateraccounting/wa/tree/master/Sheets/"
    click DOC "https://github.com/wateraccounting/wa/tree/master/docs/"
```
