# spectrum-finance/ergo-dex-backend architecture

> Architecture diagram of the GitHub repository spectrum-finance/ergo-dex-backend, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/spectrum-finance/ergo-dex-backend
- Repository: https://github.com/spectrum-finance/ergo-dex-backend

## Overview

##### Project Type and Purpose
- This is not a monolithic web app but a suite of off-chain microservices for Spectrum Finance’s DEX functionality.
- It consists of two logical service groups:
- AMM DEX services (UTXO Tracker, AMM Executor, Pool Resolver)
- Order-book DEX services (UTXO Tracker, Matcher, Orders Executor, Markets API, plus a Markets Indexer)

##### High-Level File Structure
- modules/… each top-level folder under modules is a standalone service or shared library:
- common libraries: dex-core, db, cache, http, http-cache, graphite
- AMM path: amm-executor, pool-resolver, utxo-tracker
- Orderbook path: dex-matcher, orders-executor, markets-api, markets-index, utxo-tracker (reused)
- tooling: dex-tools
- root files: docker-compose.yml, config-example.env, create_topic.sh, README.md
- project orchestrated via Docker Compose and sbt

##### Key Insights from README
- Relies on an external Ergo full node (blockchain data source)
- Uses Kafka (topics for UTXO events & inter-service messaging)
- Uses Redis (caching) and Postgres (persistent store)
- Pool Resolver also uses RocksDB for local state
- Metrics emitted to Graphite

##### Main Components
A. External Dependencies
- Ergo Node (blockchain RPC / UTXO feed)
- Kafka Cluster (topics for UTXO events, orders, pool updates)
- Redis (shared cache)
- Postgres (orders, markets, analytics, history)
- RocksDB (embedded in Pool Resolver)
- Graphite (metrics sink)

B. Microservices
1. UTXO Tracker
- consumes UTXO events from the Ergo Node (via Kafka input topic)
- parses AMM & orderbook events
- produces to Kafka topics for AMM Executor, Matcher, Orders Executor, etc.
2. AMM Executor
- consumes AMM order topics
- executes into transactions (via Ergo Node RPC)
- produces confirmations/state updates
3. Pool Resolver
- consumes pool update topics
- updates local RocksDB state
- exposes HTTP endpoints for pool lookup
4. Matcher
- consumes order topics
- matches limit orders
- writes matches to Postgres
- produces matched trades to Kafka
5. Orders Executor
- consumes matched trade topics
- executes trades via Ergo Node RPC
6. Markets API
- REST API service
- reads Postgres for orderbook and AMM stats
- may call Redis / HTTP cache for fiat rates
7. Markets Indexer
- periodically extracts history from Kafka or Postgres
- writes analytic tables in Postgres

C. Shared Libraries
- dex-core: streaming abstractions, domain models, Kafka serializers
- db: Postgres transactor, SQL utilities
- cache: Redis integration
- http, http-cache: REST routing & caching middleware
- graphite: metrics instrumentation

##### Relationships and Data Flows
- Ergo Node → UTXO Tracker (via Kafka)
- UTXO Tracker → Kafka topics → AMM Executor, Matcher, Pool Resolver
- AMM Executor → Ergo Node (RPC)
- Matcher → Postgres + Kafka → Orders Executor
- Orders Executor → Ergo Node (RPC)
- Pool Resolver → RocksDB + HTTP Server
- Markets API → Postgres + Redis → Clients (REST)
- Markets Indexer → Kafka/Postgres → analytics tables
- All services → Graphite (metrics)

- Event-driven pattern: Kafka as backbone, services decoupled by topics

##### Architectural Patterns and Principles
- Microservices: each module independently deployable (Docker Compose)
- Event-driven CQRS: write model (executors) vs. read model (Markets API, Indexer)
- Shared libraries for common concerns (DRY)
- Containerization: each service in its own container
- Configuration via environment variables (config-bundle in each service)

##### Technologies & Frameworks
- Scala + fs2-kafka for streaming
- Doobie for Postgres
- Redis client for caching
- RocksDB for local state in Pool Resolver
- http4s (assumed from http module) for REST
- Graphite metrics middleware
- Docker, Docker Compose

- Blue for services
- Green for infrastructure (Kafka, Redis, Postgres)
- Orange for external systems (Ergo Node, REST clients)
- Gray for shared libraries
- Rectangles for services

##### Detail Level
- Internal modules only if they drive architecture choices (e.g., RocksDB in Pool Resolver)
- Environment-driven configuration bundling
- Avoid over-aggregating; this is an event-driven microservice architecture, so reflect its granularity faithfully

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "External Systems"
        direction TB
        ErgoNode["Ergo Node RPC / UTXO Feed"]:::external
        RESTClients["REST Clients"]:::external
    end

    Kafka["Kafka Cluster"]:::infra

    subgraph "AMM DEX Services"
        direction TB
        UTXOTracker["UTXO Tracker"]:::service
        AMMExecutor["AMM Executor"]:::service
        PoolResolver["Pool Resolver"]:::service
    end

    subgraph "Orderbook DEX Services"
        direction TB
        Matcher["Matcher"]:::service
        OrdersExecutor["Orders Executor"]:::service
        MarketsAPI["Markets API"]:::service
        MarketsIndexer["Markets Indexer"]:::service
    end

    subgraph "Storage & Cache"
        direction TB
        Redis["Redis (Cache)"]:::infra
        Postgres["Postgres"]:::infra
        RocksDB["RocksDB (Pool Resolver)"]:::infra
    end

    subgraph "Shared Libraries"
        direction TB
        DexCore["dex-core"]:::shared
        DBLib["db"]:::shared
        CacheLib["cache"]:::shared
        HTTPLib["http"]:::shared
        HTTPCacheLib["http-cache"]:::shared
        GraphiteLib["graphite"]:::shared
        DexTools["dex-tools"]:::shared
    end

    Graphite["Graphite (Metrics Sink)"]:::infra

    %% Data Flows
    ErgoNode -->|feeds UTXO events via Kafka| Kafka
    Kafka -->|UTXO events| UTXOTracker
    UTXOTracker -->|publishes AMM topics| Kafka
    UTXOTracker -->|publishes orderbook topics| Kafka

    Kafka -->|consumes AMM orders| AMMExecutor
    AMMExecutor -->|executes via RPC| ErgoNode
    AMMExecutor -->|metrics| Graphite

    Kafka -->|pool updates| PoolResolver
    PoolResolver -->|state store| RocksDB
    PoolResolver -->|HTTP queries| RESTClients
    PoolResolver -->|metrics| Graphite

    Kafka -->|order topics| Matcher
    Matcher -->|writes matches| Postgres
    Matcher -->|publishes matches| Kafka
    Matcher -->|metrics| Graphite

    Kafka -->|matched trades| OrdersExecutor
    OrdersExecutor -->|executes via RPC| ErgoNode
    OrdersExecutor -->|metrics| Graphite

    MarketsAPI -->|reads orderbook & AMM stats| Postgres
    MarketsAPI -->|caches rates| Redis
    MarketsAPI -->|serves REST| RESTClients
    MarketsAPI -->|metrics| Graphite

    MarketsIndexer -->|reads Kafka/Postgres| MarketsIndexer
    MarketsIndexer -->|writes analytics| Postgres
    MarketsIndexer -->|metrics| Graphite

    %% Library usage (dependencies)
    UTXOTracker --> DexCore
    UTXOTracker --> DBLib
    UTXOTracker --> CacheLib
    UTXOTracker --> HTTPLib

    AMMExecutor --> DexCore
    AMMExecutor --> GraphiteLib

    PoolResolver --> DexCore
    PoolResolver --> GraphiteLib

    Matcher --> DexCore
    Matcher --> DBLib
    Matcher --> GraphiteLib

    OrdersExecutor --> DexCore
    OrdersExecutor --> GraphiteLib

    MarketsAPI --> DBLib
    MarketsAPI --> CacheLib
    MarketsAPI --> HTTPCacheLib
    MarketsAPI --> GraphiteLib

    MarketsIndexer --> DBLib
    MarketsIndexer --> GraphiteLib

    %% Click Events
    click UTXOTracker "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/utxo-tracker"
    click AMMExecutor "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/amm-executor"
    click PoolResolver "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/pool-resolver"
    click Matcher "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/dex-matcher"
    click OrdersExecutor "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/orders-executor"
    click MarketsAPI "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/markets-api"
    click MarketsIndexer "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/markets-index"
    click DexCore "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/dex-core"
    click DBLib "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/db"
    click CacheLib "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/cache"
    click HTTPLib "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/http"
    click HTTPCacheLib "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/http-cache"
    click GraphiteLib "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/graphite"
    click DexTools "https://github.com/spectrum-finance/ergo-dex-backend/tree/master/modules/dex-tools"

    %% Styles
    classDef service fill:#D0E8FF,stroke:#4A90E2,stroke-width:2px
    classDef infra fill:#DFF2D8,stroke:#3C763D,stroke-width:2px
    classDef external fill:#FFE2B3,stroke:#E67E22,stroke-width:2px
    classDef shared fill:#E0E0E0,stroke:#888888,stroke-width:1px,stroke-dasharray: 2 2
```
