# asridk/eurodata architecture

> Architecture diagram of the GitHub repository asridk/eurodata, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/asridk/eurodata
- Repository: https://github.com/asridk/eurodata

## Overview

Eurodata is a European data-intelligence platform: official-source statistics and curated events are ingested into DuckDB, then exposed through a Python API and exploratory frontends. Its principal refresh workflow fetches registered sources, loads validated records, rebuilds the structural graph, runs blocking quality checks, and optionally exports Parquet. Users explore series, country profiles, events, correlations and propagation through Streamlit or the web API; the web app also offers an AI analyst. Edges follow the README and shown references; unsampled frontend/API internals are represented conservatively.

Main entry points: [`flows/ingestion.py`](https://github.com/asridk/eurodata/blob/main/flows/ingestion.py) (Refresh orchestration), [`src/eurodata/db.py`](https://github.com/asridk/eurodata/blob/main/src/eurodata/db.py) (DuckDB dataset), [`src/eurodata/api.py`](https://github.com/asridk/eurodata/blob/main/src/eurodata/api.py) (Python data API), [`src/eurodata/model/seed.py`](https://github.com/asridk/eurodata/blob/main/src/eurodata/model/seed.py) (Reference seeding)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data foundation"]
  node_db[("DuckDB dataset<br/>[db.py]")]
  node_seed["Reference seeding<br/>[seed.py]"]
  node_catalog["Indicator catalog<br/>[catalog.py]"]
  node_geography["Geographic references<br/>[countries.py]"]
  node_events["Curated events<br/>[events.py]"]
end

subgraph group_refresh["Refresh pipeline"]
  node_sources["Data providers<br/>[registry.py]"]
  node_fetchers["Source fetchers<br/>[base.py]"]
  node_ingest["Ingestion loader<br/>[pipeline.py]"]
  node_validation["Record validation<br/>[validate.py]"]
  node_quality["Quality contracts<br/>[quality.py]"]
  node_flow["Refresh orchestration<br/>[ingestion.py]"]
  node_release["Parquet release<br/>[release.py]"]
end

subgraph group_analysis["Analysis and API"]
  node_graph["Structural graph<br/>[build.py]"]
  node_correlate["Correlations<br/>[correlate.py]"]
  node_propagate["Propagation analysis<br/>[propagate.py]"]
  node_api["Python data API<br/>[api.py]"]
end

subgraph group_web["Web experience"]
  node_backend["FastAPI service<br/>[main.py]"]
  node_chat["AI analyst<br/>[chat.py]"]
  node_tools["Analyst tools<br/>[tools.py]"]
  node_frontend["Next.js explorer<br/>[page.tsx]"]
end

subgraph group_dashboard["Streamlit dashboard"]
  node_streamlit["Streamlit dashboard<br/>[streamlit_app.py]"]
  node_components["Dashboard components<br/>[components.py]"]
end

node_actor(("Data explorer"))

node_sources -->|"registers"| node_fetchers
node_flow -->|"enumerates"| node_sources
node_flow -->|"runs"| node_ingest
node_ingest -->|"fetches"| node_fetchers
node_ingest -->|"validates"| node_validation
node_ingest -->|"loads records"| node_db
node_seed -->|"reads"| node_geography
node_seed -->|"reads"| node_catalog
node_seed -->|"reads"| node_events
node_seed -.->|"seeds"| node_db
node_flow -->|"builds"| node_graph
node_graph -->|"uses"| node_correlate
node_flow -->|"checks"| node_quality
node_quality -->|"checks data"| node_db
node_flow -->|"exports"| node_release
node_flow -->|"connects"| node_db
node_api -->|"queries"| node_db
node_api -->|"delegates"| node_propagate
node_actor -->|"explores"| node_streamlit
node_streamlit -->|"uses"| node_components
node_streamlit -->|"queries"| node_api
node_streamlit -->|"opens"| node_db
node_actor -->|"explores"| node_frontend
node_frontend -->|"requests data"| node_backend
node_backend -->|"wraps"| node_api
node_backend -->|"routes chat"| node_chat
node_chat -->|"executes"| node_tools
node_tools -->|"calls"| node_api

click node_sources "https://github.com/asridk/eurodata/blob/main/src/eurodata/sources/registry.py"
click node_fetchers "https://github.com/asridk/eurodata/blob/main/src/eurodata/sources/base.py"
click node_ingest "https://github.com/asridk/eurodata/blob/main/src/eurodata/ingest/pipeline.py"
click node_validation "https://github.com/asridk/eurodata/blob/main/src/eurodata/ingest/validate.py"
click node_quality "https://github.com/asridk/eurodata/blob/main/src/eurodata/ingest/quality.py"
click node_flow "https://github.com/asridk/eurodata/blob/main/flows/ingestion.py"
click node_release "https://github.com/asridk/eurodata/blob/main/src/eurodata/export/release.py"
click node_db "https://github.com/asridk/eurodata/blob/main/src/eurodata/db.py"
click node_seed "https://github.com/asridk/eurodata/blob/main/src/eurodata/model/seed.py"
click node_catalog "https://github.com/asridk/eurodata/blob/main/src/eurodata/reference/catalog.py"
click node_geography "https://github.com/asridk/eurodata/blob/main/src/eurodata/reference/countries.py"
click node_events "https://github.com/asridk/eurodata/blob/main/src/eurodata/reference/events.py"
click node_graph "https://github.com/asridk/eurodata/blob/main/src/eurodata/graph/build.py"
click node_correlate "https://github.com/asridk/eurodata/blob/main/src/eurodata/graph/correlate.py"
click node_propagate "https://github.com/asridk/eurodata/blob/main/src/eurodata/graph/propagate.py"
click node_api "https://github.com/asridk/eurodata/blob/main/src/eurodata/api.py"
click node_streamlit "https://github.com/asridk/eurodata/blob/main/app/streamlit_app.py"
click node_components "https://github.com/asridk/eurodata/blob/main/app/components.py"
click node_backend "https://github.com/asridk/eurodata/blob/main/web/backend/main.py"
click node_chat "https://github.com/asridk/eurodata/blob/main/web/backend/chat.py"
click node_tools "https://github.com/asridk/eurodata/blob/main/web/backend/tools.py"
click node_frontend "https://github.com/asridk/eurodata/blob/main/web/frontend/src/app/explore/page.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_db,node_seed,node_catalog,node_geography,node_events toneBlue
class node_sources,node_fetchers,node_ingest,node_validation,node_quality,node_flow,node_release toneAmber
class node_graph,node_correlate,node_propagate,node_api toneMint
class node_backend,node_chat,node_tools,node_frontend toneRose
class node_streamlit,node_components,node_actor toneIndigo
```

## Components

### Data foundation

- [DuckDB dataset](https://github.com/asridk/eurodata/blob/main/src/eurodata/db.py): `src/eurodata/db.py`
- [Reference seeding](https://github.com/asridk/eurodata/blob/main/src/eurodata/model/seed.py): `src/eurodata/model/seed.py`
- [Indicator catalog](https://github.com/asridk/eurodata/blob/main/src/eurodata/reference/catalog.py): `src/eurodata/reference/catalog.py`
- [Geographic references](https://github.com/asridk/eurodata/blob/main/src/eurodata/reference/countries.py): `src/eurodata/reference/countries.py`
- [Curated events](https://github.com/asridk/eurodata/blob/main/src/eurodata/reference/events.py): `src/eurodata/reference/events.py`

### Refresh pipeline

- [Data providers](https://github.com/asridk/eurodata/blob/main/src/eurodata/sources/registry.py): `src/eurodata/sources/registry.py`
- [Source fetchers](https://github.com/asridk/eurodata/blob/main/src/eurodata/sources/base.py): `src/eurodata/sources/base.py`
- [Ingestion loader](https://github.com/asridk/eurodata/blob/main/src/eurodata/ingest/pipeline.py): `src/eurodata/ingest/pipeline.py`
- [Record validation](https://github.com/asridk/eurodata/blob/main/src/eurodata/ingest/validate.py): `src/eurodata/ingest/validate.py`
- [Quality contracts](https://github.com/asridk/eurodata/blob/main/src/eurodata/ingest/quality.py): `src/eurodata/ingest/quality.py`
- [Refresh orchestration](https://github.com/asridk/eurodata/blob/main/flows/ingestion.py): `flows/ingestion.py`
- [Parquet release](https://github.com/asridk/eurodata/blob/main/src/eurodata/export/release.py): `src/eurodata/export/release.py`

### Analysis and API

- [Structural graph](https://github.com/asridk/eurodata/blob/main/src/eurodata/graph/build.py): `src/eurodata/graph/build.py`
- [Correlations](https://github.com/asridk/eurodata/blob/main/src/eurodata/graph/correlate.py): `src/eurodata/graph/correlate.py`
- [Propagation analysis](https://github.com/asridk/eurodata/blob/main/src/eurodata/graph/propagate.py): `src/eurodata/graph/propagate.py`
- [Python data API](https://github.com/asridk/eurodata/blob/main/src/eurodata/api.py): `src/eurodata/api.py`

### Web experience

- [FastAPI service](https://github.com/asridk/eurodata/blob/main/web/backend/main.py): `web/backend/main.py`
- [AI analyst](https://github.com/asridk/eurodata/blob/main/web/backend/chat.py): `web/backend/chat.py`
- [Analyst tools](https://github.com/asridk/eurodata/blob/main/web/backend/tools.py): `web/backend/tools.py`
- [Next.js explorer](https://github.com/asridk/eurodata/blob/main/web/frontend/src/app/explore/page.tsx): `web/frontend/src/app/explore/page.tsx`

### Streamlit dashboard

- [Streamlit dashboard](https://github.com/asridk/eurodata/blob/main/app/streamlit_app.py): `app/streamlit_app.py`
- [Dashboard components](https://github.com/asridk/eurodata/blob/main/app/components.py): `app/components.py`

### Other components

- **Data explorer**

## Connections

- Data providers → Source fetchers: registers (evidence: `src/eurodata/sources/registry.py`)
- Refresh orchestration → Data providers: enumerates (evidence: `flows/ingestion.py`)
- Refresh orchestration → Ingestion loader: runs (evidence: `flows/ingestion.py`)
- Ingestion loader → Source fetchers: fetches (evidence: `src/eurodata/ingest/pipeline.py`)
- Ingestion loader → Record validation: validates (evidence: `src/eurodata/ingest/pipeline.py`)
- Ingestion loader → DuckDB dataset: loads records (evidence: `src/eurodata/ingest/pipeline.py`)
- Reference seeding → Geographic references: reads (evidence: `src/eurodata/model/seed.py`)
- Reference seeding → Indicator catalog: reads (evidence: `src/eurodata/model/seed.py`)
- Reference seeding → Curated events: reads (evidence: `src/eurodata/model/seed.py`)
- Reference seeding → DuckDB dataset: seeds
- Refresh orchestration → Structural graph: builds (evidence: `flows/ingestion.py`)
- Structural graph → Correlations: uses (evidence: `src/eurodata/graph/build.py`)
- Refresh orchestration → Quality contracts: checks (evidence: `flows/ingestion.py`)
- Quality contracts → DuckDB dataset: checks data (evidence: `README.md`)
- Refresh orchestration → Parquet release: exports (evidence: `flows/ingestion.py`)
- Refresh orchestration → DuckDB dataset: connects (evidence: `flows/ingestion.py`)
- Python data API → DuckDB dataset: queries (evidence: `src/eurodata/api.py`)
- Python data API → Propagation analysis: delegates (evidence: `src/eurodata/api.py`)
- Data explorer → Streamlit dashboard: explores (evidence: `README.md`)
- Streamlit dashboard → Dashboard components: uses (evidence: `app/streamlit_app.py`)
- Streamlit dashboard → Python data API: queries (evidence: `app/streamlit_app.py`)
- Streamlit dashboard → DuckDB dataset: opens (evidence: `app/streamlit_app.py`)
- Data explorer → Next.js explorer: explores (evidence: `README.md`)
- Next.js explorer → FastAPI service: requests data (evidence: `README.md`)
- FastAPI service → Python data API: wraps (evidence: `README.md`)
- FastAPI service → AI analyst: routes chat (evidence: `web/backend/main.py`)
- AI analyst → Analyst tools: executes (evidence: `web/backend/chat.py`)
- Analyst tools → Python data API: calls (evidence: `web/backend/tools.py`)