# dewberry/hydroshift architecture

> Architecture diagram of the GitHub repository dewberry/hydroshift, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dewberry/hydroshift
- Repository: https://github.com/dewberry/hydroshift

## Overview

##### Project Type and Purpose
- This is a full-stack, data-analytics web application built in Python (Streamlit) with an auxiliary R service for advanced analytics.
- Purpose: Let end users explore non-stationarity analysis via an interactive Streamlit UI; heavy computations (e.g., changepoint detection, FFA) are performed in Python or offloaded to an R “microservice.”

##### File Structure
Top-level files & infra:
– Dockerfile, docker-compose.yml, deploy.sh → containerization & deployment
– .devcontainer/ → VS Code dev container config
– .github/workflows/ → CI/CD pipelines (build, test, version, push)
Core application package: hydroshift/
– _pages/ → Streamlit “page” modules (homepage, summary, changepoint)
– streamlit_app.py → central Streamlit runner
– rserver/ → R service (Plumber API scripts + Python bootstrap start_r_server.py)
– utils/, add_analytics.py → data-processing, changepoint, plotting helpers
– templates/ → Jinja2 + Markdown templates for dynamic reports
– data/ → static assets (e.g., skewmap_4326.tif)
Testing & config:
– tests/ → unit tests for statistical modules
– pyproject.toml, requirements.txt → Python dependencies
– renv.lock → R package dependencies

##### README and Runtime Flow
- README shows “streamlit run code/app.py,” but actual entrypoint is hydroshift/streamlit_app.py.
- On startup:
a. Python virtualenv activated → installs requirements.txt
b. streamlit_app spins up UI on port 8501
c. start_r_server.py optionally launches R Plumber API (port 8000)
d. Logging initialized; templates loaded

##### Main Components & Their Interactions
a. User & Browser (Client)
b. Streamlit Server (Python Frontend)
c. R Microservice (Plumber API)
d. Data Layer
– Static files (GeoTIFF in hydroshift/data)
– Any external data sources fetched via utils/data_retrieval.py
e. Template Engine (Jinja2/Markdown → HTML panels)
f. Logging & Session Management
g. CI/CD Pipeline & Container Registry
h. Local Dev Environment (.devcontainer) & Deployment Scripts (Docker, docker-compose)

##### Architectural Patterns & Design Principles
- Microservice pattern: Python UI separate from R analytics service
- Modular design: “pages” modules implement view layer; utils implement model/business logic
- Template-based reporting: Jinja2 for dynamic Markdown → HTML injection into Streamlit
- Containerized dev/prod parity: Dockerfile + docker-compose for reproducible environments

##### Final Checklist
[ ] All components labeled clearly
[ ] Notes on ports (8501, 8000), config files, and deployment scripts

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client"
        Browser[/"Browser (Client)"/]:::frontend
    end

    %% Frontend Layer
    subgraph "Frontend Layer"
        Streamlit(("Streamlit Server")):::frontend
        Pages[/"UI Pages (_pages)"/]:::frontend
        TemplateEngine[[Jinja2 Templates]]:::backend
    end

    %% Application Logic
    subgraph "App Logic"
        CP["Changepoint"]:::backend
        FFA["FFA"]:::backend
        AddAnalytics["Analytics Entry"]:::backend
        Common["Common Utils"]:::backend
        Plots["Plotting Utils"]:::backend
        DataRetrieval["External Data Fetch"]:::backend
        Logging["Logging"]:::backend
        Session["Session Management"]:::backend
    end

    %% Analytics Service
    subgraph "Analytics Service"
        RService{"R Microservice (Plumber API)"}:::backend
        RBootstrap(("R Bootstrap")):::backend
    end

    %% Data Store
    subgraph "Data Store"
        StaticData[(GeoTIFF Static Assets)]:::data
    end

    %% Infrastructure & CI/CD
    subgraph "Infrastructure"
        DevContainer[".devcontainer"]:::infra
        CICD(("CI/CD Pipelines")):::infra
        Docker("Docker & Deployment"):::infra
    end

    %% Connections
    Browser -->|"HTTP UI Events"| Streamlit
    Streamlit -->|"Load Pages"| Pages
    Streamlit -->|"use templates"| TemplateEngine
    TemplateEngine -->|"render HTML"| Browser

    Streamlit -->|"calls REST/JSON"| RService
    Streamlit -->|"launch via"| RBootstrap
    RBootstrap -->|"bootstraps"| RService

    Streamlit -->|"file I/O"| StaticData
    Streamlit -->|"fetch external data"| DataRetrieval
    DataRetrieval -->|"returns data"| Streamlit

    Streamlit -->|"invoke"| CP
    Streamlit -->|"invoke"| FFA
    Streamlit -->|"invoke"| AddAnalytics

    CP -->|"uses"| Common
    FFA -->|"uses"| Common
    AddAnalytics -->|"uses"| CP
    AddAnalytics -->|"uses"| FFA

    Streamlit -->|"init"| Logging
    Logging -->|"manages"| Session

    DevContainer -->|"configures"| Streamlit
    CICD -->|"build/test"| Docker
    CICD -->|"deploy scripts"| Docker
    Docker -->|"runs container"| Streamlit
    Docker -->|"runs container"| RService

    %% Click Events
    click Streamlit "https://github.com/dewberry/hydroshift/blob/main/hydroshift/streamlit_app.py"
    click Pages "https://github.com/dewberry/hydroshift/tree/main/hydroshift/_pages/"
    click TemplateEngine "https://github.com/dewberry/hydroshift/tree/main/hydroshift/templates/"
    click RService "https://github.com/dewberry/hydroshift/blob/main/hydroshift/rserver/server.r"
    click RBootstrap "https://github.com/dewberry/hydroshift/blob/main/hydroshift/rserver/start_r_server.py"
    click StaticData "https://github.com/dewberry/hydroshift/blob/main/hydroshift/data/skewmap_4326.tif"
    click DataRetrieval "https://github.com/dewberry/hydroshift/blob/main/hydroshift/utils/data_retrieval.py"
    click CP "https://github.com/dewberry/hydroshift/blob/main/hydroshift/utils/changepoint.py"
    click FFA "https://github.com/dewberry/hydroshift/blob/main/hydroshift/utils/ffa.py"
    click AddAnalytics "https://github.com/dewberry/hydroshift/blob/main/hydroshift/add_analytics.py"
    click Common "https://github.com/dewberry/hydroshift/blob/main/hydroshift/utils/common.py"
    click Plots "https://github.com/dewberry/hydroshift/blob/main/hydroshift/utils/plots.py"
    click Logging "https://github.com/dewberry/hydroshift/blob/main/hydroshift/app_logging.py"
    click Session "https://github.com/dewberry/hydroshift/blob/main/hydroshift/session.py"
    click DevContainer "https://github.com/dewberry/hydroshift/blob/main/.devcontainer/devcontainer.json"
    click CICD "https://github.com/dewberry/hydroshift/blob/main/.github/workflows/docker-build.yaml"
    click Docker "https://github.com/dewberry/hydroshift/blob/main/docker-compose.yml"

    %% Styles
    classDef frontend fill:#cce5ff,stroke:#2b7bb9,color:#003366;
    classDef backend fill:#d4edda,stroke:#28a745,color:#155724;
    classDef data fill:#fff3cd,stroke:#ffcd39,color:#856404;
    classDef infra fill:#e2e3e5,stroke:#6c757d,color:#343a40;
```
