# iliedorobat/qoli-framework architecture

> Architecture diagram of the GitHub repository iliedorobat/qoli-framework, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iliedorobat/qoli-framework
- Repository: https://github.com/iliedorobat/qoli-framework

## Overview

##### Project type and purpose
- This is a full-stack Java framework and service (“eLIF – European Life Index Framework”) that:
- Collects raw QoL (Quality of Life) data from external Eurostat APIs or downloaded JSON datasets
- Aggregates and calculates QoL dimensions and indicators via CLI commands
- Serves aggregated data (JSON/CSV) over REST endpoints
- Exposes configuration and statistics APIs, plus static prepared CSV/JSON files

##### File structure
Top-level directories and files:
- build.gradle, gradlew, settings.gradle → Gradle build system
- src/main/bash → Deployment/automation scripts (certbot hooks, startup, build/run wrappers)
- src/main/java
- no.ssb.jsonstat → JSON-Stat v1/v2 serialization/deserialization library
- ro.webdata.qoli.aggr → Data collection, parsing, aggregation of metrics by dimension
- ro.webdata.qoli.server → Embedded HTTP server, CORS/Auth filters, REST endpoints for stats/config/collector/geo
- ro.webdata.qoli.Main (CLI entry point) → supports flags: --collect, --calculate, --print
- src/main/resources/static →
- raw/json → raw input datasets
- prepared/{csv,json} → output published datasets organized by country/region and dimension
- Gradle wrapper, licensing, CI change history, README

##### README for additional insights
- Describes command-line usage to collect, calculate, print data via java -jar elife.jar
- Describes REST API calls over HTTPS on port 8443 served by an embedded (or Tomcat) server
- Mentions environment variables for authentication, keystore, host address, production/test flags
- Shows topics: dimensions (education, health, environment, etc.), indicators, data sources

##### Main components and their interactions
a. External systems/services
- Eurostat API (HTTPS JSON endpoint)
- File system (local JSON/CSV raw files in resources/raw)
- pm2 process manager (to launch bash startup script in background)
b. CLI Application (Jar)
- Entry point: Main.java
- DataCollector module (fetch raw data via HTTP or local raw files)
- Aggregation modules under ro.webdata.qoli.aggr.sta ts/dimensions → compute statistics per dimension
- JSON-Stat library modules (no.ssb.jsonstat) → parse, serialize JSON-Stat v1/v2
- Renderer modules (CsvRenderer, Table) → produce CSV tables
- Print modules → output to console or generate prepared resources
c. Embedded HTTP Server
- Server.java instantiates HTTP(S) listener on configurable port (8443)
- CORSFilter, AuthFilter → pre-process incoming requests
- AuthService → validates credentials for protected endpoints (e.g., /stats/collect)
- Endpoints under ro.webdata.qoli.server.endpoints
- StatsEndpoint → returns aggregated data on demand
- StatsCollectorEndpoint → triggers data collection when called with credentials
- StatsConfigEndpoint → returns metadata/config for available dimensions/indicators
- GeoEndpoint → returns geographic metadata if needed
d. Static Resource Server
- Serves files under src/main/resources/static/prepared and /raw
- Clients can fetch pre-generated CSV/JSON directly

Interactions (data flow):
1. At startup (via bash elife_startup.sh under pm2), the jar launches the HTTP server.
2. On an API request `/stats/collect`, AuthFilter→AuthService permits → StatsCollectorEndpoint
→ Aggregation modules fetch fresh data → write output into prepared folder.
3. On a `/stats` or `/stats/config` request, Controller reads from prepared resources (or in-memory service) → JSON/CSV response.
4. CLI modes (`--collect`, `--calculate`, `--print`) directly invoke DataCollector and Aggregator modules without HTTP.

##### Architectural patterns and principles
- Layered architecture (Controller → Service → Data/Utilities)
- Separation of concerns:
- Data fetching/parsing (Fetcher, LocalParser)
- Statistical computation (dimension-specific classes under aggr.stats.dimensions)
- Serialization/Rendering (JacksonStatParser, CsvRenderer, JsonStatSerializer)
- HTTP API layer (Filters, Endpoints)
- Modularization by dimension encourages extension (add new QoL dimension by implementing AggrParams/Stats classes)
- Use of external configuration via environment variables for deploy flexibility
- CLI + REST dual interface to the same core services

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Sources
    subgraph "External Data Sources" 
        direction TB
        Eurostat_API["Eurostat API (HTTPS JSON)"]:::external
        Local_Raw["Local Raw JSON Files"]:::external
    end

    %% Infrastructure
    subgraph "Infrastructure" 
        direction TB
        PM2["pm2 Process Manager"]:::infra
        Bash["elife_startup.sh"]:::infra
    end

    %% Configuration
    Env["Environment Variables"]:::config

    %% CLI Frontend
    CLI["CLI Frontend (Main.java)"]:::service
    click CLI "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/Main.java"

    %% Core Services & Engines
    subgraph "Core Services & Engines"
        direction TB
        DataCollector["DataCollector Service"]:::service
        click DataCollector "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/aggr/data/fetch/DataCollector.java"

        AggregationEngine["Aggregation Engine\n(ro.webdata.qoli.aggr.stats)"]:::service
        click AggregationEngine "https://github.com/iliedorobat/qoli-framework/tree/master/src/main/java/ro/webdata/qoli/aggr/stats"

        subgraph "Dimension Modules"
            direction TB
            Dimensions["QoL Dimensions\n(ro.webdata.qoli.aggr.stats.dimensions)"]:::service
            click Dimensions "https://github.com/iliedorobat/qoli-framework/tree/master/src/main/java/ro/webdata/qoli/aggr/stats/dimensions"
        end

        JSONStatLib["JSON-Stat Library\n(no.ssb.jsonstat)"]:::lib
        click JSONStatLib "https://github.com/iliedorobat/qoli-framework/tree/master/src/main/java/no/ssb/jsonstat"

        CsvRenderer["CSV Renderer (Json-Stat v1)"]:::utility
        click CsvRenderer "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/no/ssb/jsonstat/v1/table/CsvRenderer.java"
    end

    %% Storage
    subgraph "File System Storage" 
        direction TB
        RawStore[/"Static Raw Data Store"/]:::storage
        click RawStore "https://github.com/iliedorobat/qoli-framework/tree/master/src/main/resources/static/raw"
        PreparedStore[/"Static Prepared Data Store"/]:::storage
        click PreparedStore "https://github.com/iliedorobat/qoli-framework/tree/master/src/main/resources/static/prepared"
    end

    %% HTTP Server & API Layer
    subgraph "HTTP Server" 
        direction TB
        Server["Embedded HTTP Server\n(Server.java)"]:::api
        click Server "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/server/Server.java"
        CORSFilter["CORS Filter"]:::api
        click CORSFilter "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/server/CORSFilter.java"
        AuthFilter["Authentication Filter"]:::api
        click AuthFilter "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/server/auth/AuthFilter.java"
        AuthService["Authentication Service"]:::api
        click AuthService "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/server/auth/AuthService.java"
        subgraph "Endpoints"
            direction TB
            StatsEP["Stats Endpoint"]:::api
            click StatsEP "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/server/endpoints/stats/StatsEndpoint.java"
            StatsCollectorEP["Stats-Collector Endpoint"]:::api
            click StatsCollectorEP "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/server/endpoints/stats/collector/StatsCollectorEndpoint.java"
            StatsConfigEP["Stats-Config Endpoint"]:::api
            click StatsConfigEP "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/server/endpoints/stats/config/StatsConfigEndpoint.java"
            GeoEP["Geo Endpoint"]:::api
            click GeoEP "https://github.com/iliedorobat/qoli-framework/blob/master/src/main/java/ro/webdata/qoli/server/endpoints/geo/GeoEndpoint.java"
        end
    end

    %% Process Flows
    PM2 -->|launches| Bash
    Bash -->|starts with Env Vars| Server
    Bash -->|starts CLI on flags| CLI
    Env --> Server
    Env --> CLI

    %% CLI Data Flow
    CLI -->|invokes| DataCollector
    CLI -->|invokes| AggregationEngine
    DataCollector -->|reads JSON| Eurostat_API
    DataCollector -->|reads JSON| Local_Raw
    DataCollector -->|provides data| AggregationEngine
    AggregationEngine -->|uses| Dimensions
    AggregationEngine -->|uses| JSONStatLib
    AggregationEngine -->|renders via| CsvRenderer
    CsvRenderer -->|writes CSV/JSON| PreparedStore
    AggregationEngine -->|writes JSON| PreparedStore
    DataCollector -->|writes raw| RawStore

    %% HTTP API Flow
    Server --> CORSFilter
    CORSFilter --> AuthFilter
    AuthFilter --> AuthService
    AuthFilter --> StatsCollectorEP
    StatsCollectorEP --> DataCollector
    StatsCollectorEP --> AggregationEngine
    StatsCollectorEP -->|writes| PreparedStore
    Server --> StatsEP
    StatsEP -->|reads| PreparedStore
    Server --> StatsConfigEP
    StatsConfigEP -->|reads| PreparedStore
    Server --> GeoEP
    GeoEP -->|reads| PreparedStore

    classDef external fill:#f9f,stroke:#333,stroke-width:1px
    classDef infra fill:#bbf,stroke:#333,stroke-width:1px
    classDef config fill:#ffd700,stroke:#333,stroke-width:1px
    classDef service fill:#afa,stroke:#333,stroke-width:1px
    classDef lib fill:#fcf,stroke:#333,stroke-width:1px
    classDef utility fill:#ffb,stroke:#333,stroke-width:1px
    classDef storage fill:#ccf,stroke:#333,stroke-width:1px
    classDef api fill:#fdd,stroke:#333,stroke-width:1px
```
