How wateraccounting/wa works
Project Type and Purpose
• WA+ is a monolithic Python toolbox for automating the retrieval, processing, modeling, and reporting of hydrologic and remote-sensing data.
• It consists of •Data Collection•, •Data Processing/Utilities•, •Hydrologic Modeling•, •Product Aggregation•, and •Reporting/Export• layers, all orchestrated via a central entry point.
Break Down the Major Architectural Components
A. Entry Point / Orchestration
– WebAccounts.py (CLI script that drives the entire pipeline)
B. Data-Collection Layer
– Collect/… subpackages (ALEXI, ASCAT, CFSR, CHIRPS, …)
– Each subpackage contains DataAccess*.py plus per-cadence scripts (daily.py, monthly.py, etc.)
– Pulls raw data from external services (e.g. HTTP downloads, FTP) and writes NetCDF/CSV
C. Pre-Processing & Utility Layer
– Functions/… (flux splitting, dictionary filling, time aggregators)
– General/… (data conversions, raster handling)
– These modules transform raw files into standardized inputs for models
D. Modeling Layer
– Models/SurfWAT (channel routing, reservoir, withdrawals)
– Models/waterpix (GIS-based water-pixel modeling, Arcpy & GDAL variants)
– Each model reads processed data, runs simulations, outputs result files
E. Product Aggregation Layer
– Products/ETens (ensemble evapotranspiration products)
– Products/ETref (reference ET workflows)
– Aggregates model outputs into higher-level products (daily/monthly)
F. Reporting/Export Layer
– Generator/Sheet2, Sheet3, Sheet4, Sheet5 (CSV + PDF generators)
– Sheets/*.py (scripts to assemble sample sheets)
– Final output: PDF reports, CSV tables for end users
G. Documentation
– docs/ManualWA.md + docs/figs for user guidance (not in run-time path)
Component Interactions
1. User runs WebAccounts.py with parameters →
2. WebAccounts calls Collect → for each data source:
→ External service (HTTP/FTP)
→ DataAccess → writes raw files to disk/storage
3. Functions & General modules read raw files → produce cleaned, aggregated input files
4. Models (SurfWAT / waterpix) ingest cleaned inputs → output simulation NetCDFs
5. Products modules pick up simulation outputs → compute ETens / ETref products
6. Generator & Sheets consume final products → produce PDF/CSV deliverables
7. User collects deliverables; repeat pipeline as needed
Architectural Patterns & Design Principles
• Pipeline Data-flow: strictly layered—data moves one way from collection to reporting
• Modular Packaging: each data source, function group, and model lives in its own subpackage
• Separation of Concerns:
– Collectors know only about fetching data
– Utility modules know only about data transformations
– Models focus exclusively on simulation logic
– Generators focus only on presentation
• Extensibility points: add a new collector by dropping a new subfolder under Collect/, add a new model under Models/