# eclipse-edc/federatedcatalog architecture

> Architecture diagram of the GitHub repository eclipse-edc/federatedcatalog, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eclipse-edc/federatedcatalog
- Repository: https://github.com/eclipse-edc/federatedcatalog

## Overview

##### Project type and purpose
- This is a standalone backend microservice—“EDC Federated Catalog”—built on the Eclipse Data Space Connector (EDC) runtime.
- It periodically crawls a list of remote data-space participants, fetches their catalogs, transforms and caches them locally, and exposes a unified catalog API.

##### Key modules (from the file tree)
- spi/…-spi: Java Service Provider Interfaces defining crawler and federated‐catalog contracts
- core/…: core implementations (common utilities, crawler engine, federated-catalog logic, JSON transformers)
- extensions/api/federated-catalog-api: HTTP API extension (REST controllers)
- extensions/store/sql/…: optional SQL-backed implementations for cache and directory
- launchers: Docker/Java process entry points (in-memory “mocked” or DCP-secured identity)
- system-tests: integration and end-to-end tests

##### Architectural overview
- At runtime, the EDC Connector bootstrap loads all ServiceExtension implementations (from core, extensions, sql modules, etc.) into a single-process container.
- A Directory component holds the list of participant endpoints (in-memory or SQL).
- An ExecutionManager schedules crawler tasks at regular intervals.
- Each CrawlerAction fetches a participant’s catalog over HTTP (or via DCP-secured channel), hands off the raw JSON to a JsonObjectToCatalogTransformer, and then invokes the FederatedCatalogCache to store/update entries.
- The FederatedCatalogCache sits on top of an in-memory map or a SQL store, depending on which store extension is active.
- The FederatedCatalogApiController exposes `/api/catalog` over HTTP; when a client query arrives, it goes through QueryService → FederatedCatalogCache → JSON serializer → HTTP response.
- An external Identity Provider (mocked or DCP) is used for authentication/authorization on crawl requests and API calls.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "EDC Connector Runtime"
        subgraph "Bootstrap & Extension Loader"
            SPI1["ServiceExtension SPI: Crawler"]:::edc
            SPI2["ServiceExtension SPI: Federated Catalog"]:::edc
            Util["catalog-util-lib"]:::edc
        end
        subgraph "Domain Services"
            Dir["Participant Directory\n(In-memory/SQL)"]:::blue
            EM["ExecutionManager"]:::blue
            Reg["CrawlerActionRegistry"]:::blue
            subgraph "Crawler Engine"
                Crawler["CrawlerAction"]:::blue
            end
            Trans["JsonObjectToCatalogTransformer"]:::blue
            CacheExt["FederatedCatalogCacheExtension\n(In-memory/SQL)"]:::blue
        end
        subgraph "API Layer"
            APICont["FederatedCatalogApiController"]:::blue
            QS["QueryService"]:::blue
        end
    end

    IDP["Identity Provider\n(Mocked/DCP)"]:::external
    Participants["Participant Endpoints\n(HTTP)"]:::external
    Client["Client Applications"]:::external
    StorageSQL[(SQL Database)]:::db

    Dir --> EM
    EM -->|cron tick| Reg
    Reg --> Crawler
    Crawler -->|HTTP GET /catalog| Participants
    Participants -->|HTTP 200 JSON| Crawler
    Crawler -->|"JSON→Model"| Trans
    Trans -->|"store/update"| CacheExt
    CacheExt --> StorageSQL
    Client -->|HTTP GET /api/catalog| APICont
    APICont --> QS
    QS -->|"query cache"| CacheExt
    CacheExt -->|"JSON"| APICont
    APICont -->|HTTP 200 JSON| Client

    IDP ---|authN/authZ| Participants
    IDP ---|authN/authZ| Crawler
    IDP ---|authN/authZ| APICont

    click SPI1 "https://github.com/eclipse-edc/federatedcatalog/tree/main/spi/crawler-spi"
    click SPI2 "https://github.com/eclipse-edc/federatedcatalog/tree/main/spi/federated-catalog-spi"
    click Util "https://github.com/eclipse-edc/federatedcatalog/tree/main/core/common/lib/catalog-util-lib"
    click EM "https://github.com/eclipse-edc/federatedcatalog/blob/main/core/crawler-core/src/main/java/org/eclipse/edc/catalog/cache/ExecutionManager.java"
    click Reg "https://github.com/eclipse-edc/federatedcatalog/blob/main/core/crawler-core/src/main/java/org/eclipse/edc/catalog/cache/crawler/CrawlerActionRegistryImpl.java"
    click Trans "https://github.com/eclipse-edc/federatedcatalog/blob/main/core/federated-catalog-core-08/src/main/java/org/eclipse/edc/catalog/transform/JsonObjectToCatalogTransformer.java"
    click CacheExt "https://github.com/eclipse-edc/federatedcatalog/blob/main/core/federated-catalog-core-08/src/main/java/org/eclipse/edc/catalog/cache/FederatedCatalogCacheExtension.java"
    click APICont "https://github.com/eclipse-edc/federatedcatalog/blob/main/extensions/api/federated-catalog-api/src/main/java/org/eclipse/edc/catalog/api/query/FederatedCatalogApiController.java"
    click QS "https://github.com/eclipse-edc/federatedcatalog/blob/main/extensions/api/federated-catalog-api/src/main/java/org/eclipse/edc/catalog/api/query/FederatedCatalogApiExtension.java"
    classDef edc fill:#f9e79f,stroke:#d4ac0d,color:#000
    classDef blue fill:#aed6f1,stroke:#5499c7,color:#000
    classDef external fill:#a9dfbf,stroke:#27ae60,color:#000
    classDef db fill:#d7bde2,stroke:#8e44ad,color:#000
```
