How spectrum-finance/ergo-dex-backend works
Project Type and Purpose
• This is not a monolithic web app but a suite of off-chain microservices for Spectrum Finance’s DEX functionality.
• It consists of two logical service groups:
• AMM DEX services (UTXO Tracker, AMM Executor, Pool Resolver)
• Order-book DEX services (UTXO Tracker, Matcher, Orders Executor, Markets API, plus a Markets Indexer)
High-Level File Structure
• modules/… each top-level folder under modules is a standalone service or shared library:
• common libraries: dex-core, db, cache, http, http-cache, graphite
• AMM path: amm-executor, pool-resolver, utxo-tracker
• Orderbook path: dex-matcher, orders-executor, markets-api, markets-index, utxo-tracker (reused)
• tooling: dex-tools
• root files: docker-compose.yml, config-example.env, create_topic.sh, README.md
• project orchestrated via Docker Compose and sbt
Key Insights from README
• Relies on an external Ergo full node (blockchain data source)
• Uses Kafka (topics for UTXO events & inter-service messaging)
• Uses Redis (caching) and Postgres (persistent store)
• Pool Resolver also uses RocksDB for local state
• Metrics emitted to Graphite
Main Components
A. External Dependencies
• Ergo Node (blockchain RPC / UTXO feed)
• Kafka Cluster (topics for UTXO events, orders, pool updates)
• Redis (shared cache)
• Postgres (orders, markets, analytics, history)
• RocksDB (embedded in Pool Resolver)
• Graphite (metrics sink)
B. Microservices
1. UTXO Tracker
• consumes UTXO events from the Ergo Node (via Kafka input topic)
• parses AMM & orderbook events
• produces to Kafka topics for AMM Executor, Matcher, Orders Executor, etc.
2. AMM Executor
• consumes AMM order topics
• executes into transactions (via Ergo Node RPC)
• produces confirmations/state updates
3. Pool Resolver
• consumes pool update topics
• updates local RocksDB state
• exposes HTTP endpoints for pool lookup
4. Matcher
• consumes order topics
• matches limit orders
• writes matches to Postgres
• produces matched trades to Kafka
5. Orders Executor
• consumes matched trade topics
• executes trades via Ergo Node RPC
6. Markets API
• REST API service
• reads Postgres for orderbook and AMM stats
• may call Redis / HTTP cache for fiat rates
7. Markets Indexer
• periodically extracts history from Kafka or Postgres
• writes analytic tables in Postgres
C. Shared Libraries
• dex-core: streaming abstractions, domain models, Kafka serializers
• db: Postgres transactor, SQL utilities
• cache: Redis integration
• http, http-cache: REST routing & caching middleware
• graphite: metrics instrumentation
Relationships and Data Flows
• Ergo Node → UTXO Tracker (via Kafka)
• UTXO Tracker → Kafka topics → AMM Executor, Matcher, Pool Resolver
• AMM Executor → Ergo Node (RPC)
• Matcher → Postgres + Kafka → Orders Executor
• Orders Executor → Ergo Node (RPC)
• Pool Resolver → RocksDB + HTTP Server
• Markets API → Postgres + Redis → Clients (REST)
• Markets Indexer → Kafka/Postgres → analytics tables
• All services → Graphite (metrics)
• Event-driven pattern: Kafka as backbone, services decoupled by topics
Architectural Patterns and Principles
• Microservices: each module independently deployable (Docker Compose)
• Event-driven CQRS: write model (executors) vs. read model (Markets API, Indexer)
• Shared libraries for common concerns (DRY)
• Containerization: each service in its own container
• Configuration via environment variables (config-bundle in each service)
Technologies & Frameworks
• Scala + fs2-kafka for streaming
• Doobie for Postgres
• Redis client for caching
• RocksDB for local state in Pool Resolver
• http4s (assumed from http module) for REST
• Graphite metrics middleware
• Docker, Docker Compose
• Blue for services
• Green for infrastructure (Kafka, Redis, Postgres)
• Orange for external systems (Ergo Node, REST clients)
• Gray for shared libraries
• Rectangles for services
Detail Level
• Internal modules only if they drive architecture choices (e.g., RocksDB in Pool Resolver)
• Environment-driven configuration bundling
• Avoid over-aggregating; this is an event-driven microservice architecture, so reflect its granularity faithfully