# rs-works/hive-udfs architecture

> Architecture diagram of the GitHub repository rs-works/hive-udfs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rs-works/hive-udfs
- Repository: https://github.com/rs-works/hive-udfs

## Overview

##### Project Type and Purpose
- This is a Java-based Hive UDF (User Defined Function) library.
- Its purpose is to package a set of custom Hive functions (yearweek, yearweekdiff, yearmonth, yearmonthdiff, fetch_min, fetch_max, map_sum, geoip, urldecode) into a single JAR that can be loaded into Hive (or another Hadoop-compatible SQL engine) at runtime.

##### File Structure Analysis
- Top level:
- build.gradle, gradlew/gradlew.bat, settings.gradle – Gradle build and wrapper scripts.
- .circleci/config.yml – CI pipeline definition.
- README.md – documentation of each UDF.
- src/main/java/rsworks/hive – contains one Java class per Hive UDF.
- src/main/resources – placeholder for any runtime resources (e.g., GeoIP database file).
- src/test/java/rsworks/hive – unit tests covering key UDFs.

##### README Insights
- Describes each function’s signature, use case examples, and edge-cases.

##### Components
a. Core Components
1. UDF Library Module (Gradle project)
– Source code (src/main/java/rsworks/hive/*.java)
– Unit tests (src/test/java/rsworks/hive)
– Build artifacts: hive-udfs.jar
2. Build & CI Pipeline
– Gradle (compile, test, jar)
– CircleCI (checkout → build → test → publish)
3. Hive Execution Environment
– HiveServer2 or Hive CLI
– ClassLoader / ADD JAR mechanism
– GeoIP Database file (external resource loaded with ADD FILE)
4. End-User / Client
– SQL clients (Beeline, Spark SQL, etc.)
– Queries invoking the custom UDFs

b. Relationships & Data Flows
1. Developer commits code → CircleCI pulls repository → Gradle build + unit tests → produces hive-udfs.jar
2. Admin or user copies hive-udfs.jar (and GeoIP MMDB) to HDFS or local path
3. In Hive CLI/Beeline:
- JAR hive-udfs.jar
- FILE GeoLite2-City.mmdb
- CREATE TEMPORARY FUNCTION <fn> AS 'rsworks.hive.<ClassName>'
4. Hive execution engine loads the class from the JAR, invokes UDF methods at query runtime
5. geoip UDF may read the MMDB file via the resource path

c. Architectural Patterns & Principles
- Plugin architecture: Each UDF is a standalone Java class implementing Hive’s UDF interface, discovered at runtime via CREATE FUNCTION.
- Build/test automation: Gradle for dependency management & compilation, JUnit for unit tests, CircleCI for continuous integration.
- Separation of concerns:
- Source files under src/main/java
- Tests under src/test/java
- Build definitions at root

d. Key Technologies & Frameworks
- Java 8+
- Apache Hive UDF API
- Gradle
- JUnit (unit testing)
- CircleCI (CI/CD)
- GeoLite2 (external binary resource)

## Diagram (Mermaid)

```mermaid
graph TB
    %% External Entities
    Developer(["Developer"]):::external
    SQLClient(["SQL Client<br/>(Beeline, Spark SQL)"]):::external

    %% UDF Library Module
    subgraph "UDF Library Module"
        direction TB
        subgraph "Source Code" 
            direction TB
            Yearweek["Yearweek.java"]:::code
            YearweekDiff["YearweekDiff.java"]:::code
            Yearmonth["Yearmonth.java"]:::code
            YearmonthDiff["YearmonthDiff.java"]:::code
            FetchMin["FetchMin.java"]:::code
            FetchMax["FetchMax.java"]:::code
            MapSum["MapSum.java"]:::code
            GeoIP["GeoIP.java"]:::code
            URLDecode["URLDecode.java"]:::code
        end
        subgraph "Unit Tests"
            direction TB
            YWTest["YearweekTest.java"]:::code
            YWDiffTest["YearweekDiffTest.java"]:::code
            YMTest["YearmonthTest.java"]:::code
        end
        subgraph "Build Scripts & Definitions"
            direction TB
            BuildGradle["build.gradle"]:::artifact
            SettingsGradle["settings.gradle"]:::artifact
            Gradlew["gradlew"]:::artifact
            GradlewBat["gradlew.bat"]:::artifact
            GradleWrapper["gradle/wrapper"]:::artifact
        end
        subgraph "Documentation"
            README["README.md"]:::artifact
        end
        subgraph "Resources"
            MainRes["src/main/resources"]:::resource
            TestRes["src/test/resources"]:::resource
        end
    end

    %% Build & CI Pipeline
    subgraph "Build & CI Pipeline"
        direction TB
        CircleCI["CircleCI"]:::buildCD
        Gradle["Gradle Build"]:::buildCD
        JarArtifact["hive-udfs.jar"]:::artifact
    end

    %% Hive Environment & Runtime
    subgraph "Hive Execution Environment"
        direction TB
        HiveServer2["HiveServer2"]:::runtime
        ClassLoader["ClassLoader/ADD JAR"]:::runtime
        GeoIPDB["GeoLite2-City.mmdb"]:::resource
        UDFExec["UDF Execution"]:::runtime
    end

    %% Flow
    Developer -->|"commits code"| CircleCI
    CircleCI -->|"runs"| Gradle
    Gradle -->|"compiles & tests"| JarArtifact
    JarArtifact -->|"artifact"| HiveServer2
    MainRes -->|"package resource"| HiveServer2
    SQLClient -->|"ADD JAR & FILE"| ClassLoader
    ClassLoader -->|"loads JAR & resource"| HiveServer2
    HiveServer2 -->|"invokes"| UDFExec
    GeoIPDB -->|"loaded with ADD FILE"| ClassLoader
    UDFExec -->|"returns results"| SQLClient

    %% Click Events
    click Yearweek "https://github.com/rs-works/hive-udfs/blob/master/src/main/java/rsworks/hive/Yearweek.java"
    click YearweekDiff "https://github.com/rs-works/hive-udfs/blob/master/src/main/java/rsworks/hive/YearweekDiff.java"
    click Yearmonth "https://github.com/rs-works/hive-udfs/blob/master/src/main/java/rsworks/hive/Yearmonth.java"
    click YearmonthDiff "https://github.com/rs-works/hive-udfs/blob/master/src/main/java/rsworks/hive/YearmonthDiff.java"
    click FetchMin "https://github.com/rs-works/hive-udfs/blob/master/src/main/java/rsworks/hive/FetchMin.java"
    click FetchMax "https://github.com/rs-works/hive-udfs/blob/master/src/main/java/rsworks/hive/FetchMax.java"
    click MapSum "https://github.com/rs-works/hive-udfs/blob/master/src/main/java/rsworks/hive/MapSum.java"
    click GeoIP "https://github.com/rs-works/hive-udfs/blob/master/src/main/java/rsworks/hive/GeoIP.java"
    click URLDecode "https://github.com/rs-works/hive-udfs/blob/master/src/main/java/rsworks/hive/URLDecode.java"
    click YWTest "https://github.com/rs-works/hive-udfs/blob/master/src/test/java/rsworks/hive/YearweekTest.java"
    click YWDiffTest "https://github.com/rs-works/hive-udfs/blob/master/src/test/java/rsworks/hive/YearweekDiffTest.java"
    click YMTest "https://github.com/rs-works/hive-udfs/blob/master/src/test/java/rsworks/hive/YearmonthTest.java"
    click BuildGradle "https://github.com/rs-works/hive-udfs/blob/master/build.gradle"
    click SettingsGradle "https://github.com/rs-works/hive-udfs/blob/master/settings.gradle"
    click Gradlew "https://github.com/rs-works/hive-udfs/tree/master/gradlew"
    click GradlewBat "https://github.com/rs-works/hive-udfs/blob/master/gradlew.bat"
    click GradleWrapper "https://github.com/rs-works/hive-udfs/tree/master/gradle/wrapper"
    click README "https://github.com/rs-works/hive-udfs/blob/master/README.md"
    click MainRes "https://github.com/rs-works/hive-udfs/blob/master/src/main/resources/.gitkeep"
    click TestRes "https://github.com/rs-works/hive-udfs/blob/master/src/test/resources/.gitkeep"

    %% Styles
    classDef buildCD fill:#AED6F1,stroke:#3498DB,color:#1B4F72;
    classDef code fill:#ABEBC6,stroke:#28B463,color:#1D8348;
    classDef artifact fill:#A9DFBF,stroke:#229954,color:#186A3B;
    classDef runtime fill:#F9E79F,stroke:#F4D03F,color:#7D6608;
    classDef resource fill:#D5DBDB,stroke:#85929E,color:#1B2631;
    classDef external fill:#E8DAEF,stroke:#BB8FCE,color:#4A235A;
```
