How iliedorobat/qoli-framework works
Project type and purpose
• This is a full-stack Java framework and service (“eLIF – European Life Index Framework”) that:
• Collects raw QoL (Quality of Life) data from external Eurostat APIs or downloaded JSON datasets
• Aggregates and calculates QoL dimensions and indicators via CLI commands
• Serves aggregated data (JSON/CSV) over REST endpoints
• Exposes configuration and statistics APIs, plus static prepared CSV/JSON files
File structure
Top-level directories and files:
• build.gradle, gradlew, settings.gradle → Gradle build system
• src/main/bash → Deployment/automation scripts (certbot hooks, startup, build/run wrappers)
• src/main/java
• no.ssb.jsonstat → JSON-Stat v1/v2 serialization/deserialization library
• ro.webdata.qoli.aggr → Data collection, parsing, aggregation of metrics by dimension
• ro.webdata.qoli.server → Embedded HTTP server, CORS/Auth filters, REST endpoints for stats/config/collector/geo
• ro.webdata.qoli.Main (CLI entry point) → supports flags: --collect, --calculate, --print
• src/main/resources/static →
• raw/json → raw input datasets
• prepared/{csv,json} → output published datasets organized by country/region and dimension
• Gradle wrapper, licensing, CI change history, README
README for additional insights
• Describes command-line usage to collect, calculate, print data via java -jar elife.jar
• Describes REST API calls over HTTPS on port 8443 served by an embedded (or Tomcat) server
• Mentions environment variables for authentication, keystore, host address, production/test flags
• Shows topics: dimensions (education, health, environment, etc.), indicators, data sources
Main components and their interactions
a. External systems/services
• Eurostat API (HTTPS JSON endpoint)
• File system (local JSON/CSV raw files in resources/raw)
• pm2 process manager (to launch bash startup script in background)
b. CLI Application (Jar)
• Entry point: Main.java
• DataCollector module (fetch raw data via HTTP or local raw files)
• Aggregation modules under ro.webdata.qoli.aggr.sta ts/dimensions → compute statistics per dimension
• JSON-Stat library modules (no.ssb.jsonstat) → parse, serialize JSON-Stat v1/v2
• Renderer modules (CsvRenderer, Table) → produce CSV tables
• Print modules → output to console or generate prepared resources
c. Embedded HTTP Server
• Server.java instantiates HTTP(S) listener on configurable port (8443)
• CORSFilter, AuthFilter → pre-process incoming requests
• AuthService → validates credentials for protected endpoints (e.g., /stats/collect)
• Endpoints under ro.webdata.qoli.server.endpoints
• StatsEndpoint → returns aggregated data on demand
• StatsCollectorEndpoint → triggers data collection when called with credentials
• StatsConfigEndpoint → returns metadata/config for available dimensions/indicators
• GeoEndpoint → returns geographic metadata if needed
d. Static Resource Server
• Serves files under src/main/resources/static/prepared and /raw
• Clients can fetch pre-generated CSV/JSON directly
Interactions (data flow):
1. At startup (via bash elife_startup.sh under pm2), the jar launches the HTTP server.
2. On an API request /stats/collect, AuthFilter→AuthService permits → StatsCollectorEndpoint
→ Aggregation modules fetch fresh data → write output into prepared folder.
3. On a /stats or /stats/config request, Controller reads from prepared resources (or in-memory service) → JSON/CSV response.
4. CLI modes (--collect, --calculate, --print) directly invoke DataCollector and Aggregator modules without HTTP.
Architectural patterns and principles
• Layered architecture (Controller → Service → Data/Utilities)
• Separation of concerns:
• Data fetching/parsing (Fetcher, LocalParser)
• Statistical computation (dimension-specific classes under aggr.stats.dimensions)
• Serialization/Rendering (JacksonStatParser, CsvRenderer, JsonStatSerializer)
• HTTP API layer (Filters, Endpoints)
• Modularization by dimension encourages extension (add new QoL dimension by implementing AggrParams/Stats classes)
• Use of external configuration via environment variables for deploy flexibility
• CLI + REST dual interface to the same core services