# ua-snap/data-api architecture

> Architecture diagram of the GitHub repository ua-snap/data-api, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ua-snap/data-api
- Repository: https://github.com/ua-snap/data-api

## Overview

##### Main Components
- Web Server Layer:
- The core application is defined in application.py (the Flask app) along with the configuration in config.py. It’s responsible for starting the API service.
- The routes directory houses a set of controllers/endpoints (e.g., route files like permafrost.py, fire.py, etc.) that serve as entry points for different API endpoints.
- Data Processing Modules:
- Modules such as csv_functions.py, fetch_data.py, generate_requests.py, generate_urls.py, postprocessing.py, zonal_stats.py, and validate_data.py provide supporting functionality (e.g., data queries, processing geospatial raster data from the geotiffs folder, and validating requests).
- The geotiffs folder contains the underlying raster data files used for calculating various environmental indicators.
- Presentation/Response Templates:
- The templates directory holds HTML templates for error pages (400, 403, 404, 422, 500) and documentation alongside a base template. This indicates that while the API is primarily data-driven, it offers templated responses for errors and documentation pages.
- The static folder includes assets like CSS and static configuration files (e.g., robots.txt).
- External Services & Infrastructure:
- Environment variables and configuration (as mentioned in the README and in config.py) specify external endpoints: GeoServer (for map data) and Rasdaman (for raster data querying).
- The .platform/nginx and varnish-configuration directories reveal that there is a reverse proxy (NGINX) and caching layer (Varnish) that fronts the Flask application, ensuring proper routing, caching, and load management.
- Pipfile/Pipfile.lock and requirements.txt denote the dependency management setup.

- That within these route handlers, the application calls various helper modules (e.g., fetch_data.py, generate_requests.py) to prepare data queries.
- A data flow from these modules to two different types of data sources:
- The local geotiffs (raster data files) folder — representing internal data assets.

##### Important Architectural Patterns & Technologies
- A layered architecture that cleanly separates:
- Presentation layer (templates/static),
- Business logic / routing (the Flask API and its various route handlers),
- Data access and processing layers (helper/change functions and external API calls).
- Usage of reverse-proxies and caching (NGINX and Varnish), which helps in routing, load balancing, and performance optimization.
- Detailed environmental and deployment configurations which denote how external dependencies are incorporated (via environment variables and config.py).
- The modular structure in the routes folder suggests adherence to a RESTful API design pattern with clearly defined endpoints for each data type.

## Diagram (Mermaid)

```mermaid
flowchart TD
    Client["Client (Browser/API Consumer)"]:::client

    subgraph "Proxy & Caching Layer"
        NGINX["NGINX Reverse Proxy"]:::infra
        Varnish["Varnish Cache"]:::infra
    end

    subgraph "Flask Application"
        FlaskApp["Flask App (application.py)"]:::application
        Config["Configuration (config.py)"]:::application
        APIRoutes["API Routes (routes/)"]:::routes
    end

    subgraph "Data Processing Modules"
        CSVFunc["CSV Functions (csv_functions.py)"]:::datamod
        FetchData["Fetch Data (fetch_data.py)"]:::datamod
        GenReq["Generate Requests (generate_requests.py)"]:::datamod
        GenURL["Generate URLs (generate_urls.py)"]:::datamod
        PostProc["Post Processing (postprocessing.py)"]:::datamod
        ZonalStats["Zonal Stats (zonal_stats.py)"]:::datamod
        ValidateData["Validate Data (validate_data.py)"]:::datamod
        ValidateReq["Validate Request (validate_request.py)"]:::datamod
    end

    RasterData["Raster Data (geotiffs/)"]:::datamod

    subgraph "Presentation / Response Templates"
        ErrorTemplates["Error Templates (templates/)"]:::presentation
        Static["Static Assets (static/)"]:::presentation
    end

    subgraph "External Services"
        GeoServer["GeoServer"]:::external
        Rasdaman["Rasdaman"]:::external
    end

    subgraph "Dependency Management"
        Pipfile["Pipfile"]:::infra
        PipfileLock["Pipfile.lock"]:::infra
        Requirements["requirements.txt"]:::infra
    end

    %% Relationships
    Client -->|"HTTPRequest"| NGINX
    NGINX -->|"Forwards"| Varnish
    Varnish -->|"Routes"| FlaskApp
    FlaskApp -->|"Uses"| Config
    FlaskApp -->|"Handles"| APIRoutes

    %% API Routes invoking data processing modules
    APIRoutes -->|"Calls"| CSVFunc
    APIRoutes -->|"Calls"| FetchData
    APIRoutes -->|"Calls"| GenReq
    APIRoutes -->|"Calls"| GenURL
    APIRoutes -->|"Calls"| PostProc
    APIRoutes -->|"Calls"| ZonalStats
    APIRoutes -->|"Calls"| ValidateData
    APIRoutes -->|"Calls"| ValidateReq

    %% Data Processing Modules access raster data
    CSVFunc -->|"Reads"| RasterData
    FetchData -->|"Reads"| RasterData
    GenReq -->|"Reads"| RasterData
    GenURL -->|"Reads"| RasterData
    PostProc -->|"Reads"| RasterData
    ZonalStats -->|"Reads"| RasterData
    ValidateData -->|"Reads"| RasterData
    ValidateReq -->|"Reads"| RasterData

    %% Data Processing Modules interact with External Services
    CSVFunc -->|"Integrates"| GeoServer
    FetchData -->|"Integrates"| GeoServer
    GenReq -->|"Integrates"| Rasdaman
    GenURL -->|"Integrates"| Rasdaman
    PostProc -->|"Integrates"| GeoServer
    ZonalStats -->|"Integrates"| Rasdaman
    ValidateData -->|"Integrates"| Rasdaman
    ValidateReq -->|"Integrates"| GeoServer

    %% Error handling paths via templates and static assets
    APIRoutes -->|"ErrorHandling"| ErrorTemplates
    APIRoutes -->|"ErrorHandling"| Static

    %% Dependency management (dashed relationships)
    FlaskApp ---|"Dependencies"| Pipfile
    FlaskApp ---|"Dependencies"| PipfileLock
    FlaskApp ---|"Dependencies"| Requirements

    %% Click Events
    click FlaskApp "https://github.com/ua-snap/data-api/blob/main/application.py"
    click Config "https://github.com/ua-snap/data-api/blob/main/config.py"
    click APIRoutes "https://github.com/ua-snap/data-api/tree/main/routes/"
    click CSVFunc "https://github.com/ua-snap/data-api/blob/main/csv_functions.py"
    click FetchData "https://github.com/ua-snap/data-api/blob/main/fetch_data.py"
    click GenReq "https://github.com/ua-snap/data-api/blob/main/generate_requests.py"
    click GenURL "https://github.com/ua-snap/data-api/blob/main/generate_urls.py"
    click PostProc "https://github.com/ua-snap/data-api/blob/main/postprocessing.py"
    click ZonalStats "https://github.com/ua-snap/data-api/blob/main/zonal_stats.py"
    click ValidateData "https://github.com/ua-snap/data-api/blob/main/validate_data.py"
    click ValidateReq "https://github.com/ua-snap/data-api/blob/main/validate_request.py"
    click RasterData "https://github.com/ua-snap/data-api/tree/main/geotiffs/"
    click ErrorTemplates "https://github.com/ua-snap/data-api/tree/main/templates/"
    click Static "https://github.com/ua-snap/data-api/tree/main/static/"
    click NGINX "https://github.com/ua-snap/data-api/tree/main/.platform/nginx/"
    click Varnish "https://github.com/ua-snap/data-api/tree/main/varnish-configuration/"
    click Pipfile "https://github.com/ua-snap/data-api/tree/main/Pipfile"
    click PipfileLock "https://github.com/ua-snap/data-api/blob/main/Pipfile.lock"
    click Requirements "https://github.com/ua-snap/data-api/blob/main/requirements.txt"

    %% Styling classes
    classDef client fill:#E6E6FA,stroke:#000,stroke-width:1px;
    classDef infra fill:#FFD700,stroke:#000,stroke-width:1px;
    classDef application fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef routes fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef datamod fill:#FFA07A,stroke:#000,stroke-width:1px;
    classDef presentation fill:#FFB6C1,stroke:#000,stroke-width:1px;
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px;
```
