How dewberry/hydroshift works
Project Type and Purpose
• This is a full-stack, data-analytics web application built in Python (Streamlit) with an auxiliary R service for advanced analytics.
• Purpose: Let end users explore non-stationarity analysis via an interactive Streamlit UI; heavy computations (e.g., changepoint detection, FFA) are performed in Python or offloaded to an R “microservice.”
File Structure
Top-level files & infra:
– Dockerfile, docker-compose.yml, deploy.sh → containerization & deployment
– .devcontainer/ → VS Code dev container config
– .github/workflows/ → CI/CD pipelines (build, test, version, push)
Core application package: hydroshift/
– _pages/ → Streamlit “page” modules (homepage, summary, changepoint)
– streamlit_app.py → central Streamlit runner
– rserver/ → R service (Plumber API scripts + Python bootstrap start_r_server.py)
– utils/, add_analytics.py → data-processing, changepoint, plotting helpers
– templates/ → Jinja2 + Markdown templates for dynamic reports
– data/ → static assets (e.g., skewmap_4326.tif)
Testing & config:
– tests/ → unit tests for statistical modules
– pyproject.toml, requirements.txt → Python dependencies
– renv.lock → R package dependencies
README and Runtime Flow
• README shows “streamlit run code/app.py,” but actual entrypoint is hydroshift/streamlit_app.py.
• On startup:
a. Python virtualenv activated → installs requirements.txt
b. streamlit_app spins up UI on port 8501
c. start_r_server.py optionally launches R Plumber API (port 8000)
d. Logging initialized; templates loaded
Main Components & Their Interactions
a. User & Browser (Client)
b. Streamlit Server (Python Frontend)
c. R Microservice (Plumber API)
d. Data Layer
– Static files (GeoTIFF in hydroshift/data)
– Any external data sources fetched via utils/data_retrieval.py
e. Template Engine (Jinja2/Markdown → HTML panels)
f. Logging & Session Management
g. CI/CD Pipeline & Container Registry
h. Local Dev Environment (.devcontainer) & Deployment Scripts (Docker, docker-compose)
Architectural Patterns & Design Principles
• Microservice pattern: Python UI separate from R analytics service
• Modular design: “pages” modules implement view layer; utils implement model/business logic
• Template-based reporting: Jinja2 for dynamic Markdown → HTML injection into Streamlit
• Containerized dev/prod parity: Dockerfile + docker-compose for reproducible environments
Final Checklist
[ ] All components labeled clearly
[ ] Notes on ports (8501, 8000), config files, and deployment scripts