How wakumo/eventop works
Project type and purpose
• This is a Node.js microservice built on NestJS.
• It scans blockchain networks for events, processes them, stores results in PostgreSQL, and publishes/subscribes via RabbitMQ.
• It also exposes HTTP endpoints to query networks, events, and processed data.
File structure (key directories)
• src/apis – REST controllers & services for domain entities: networks, events, event-messages, processed-blocks
• src/scripts – one-off CLI scripts & seeding jobs (seed networks & events)
• src/rabbitmq – producer/consumer modules for RabbitMQ
• src/config – all environment, database (typeORM), Redis, RabbitMQ configurations
• src/commons – shared utils (JSON-RPC client to blockchain nodes, retry manager), enums, interfaces, custom exceptions, cache-manager (Redis)
• src/entities & src/migrations – TypeORM entities and migrations for PostgreSQL schema
• job.module.ts + src/scripts/jobs – scheduled “scan_events”, “rescan_events” jobs that poll the blockchain
• connection.ts / pg_client.ts / create_db.ts / drop_db.ts – direct DB setup scripts
Extract insights from README
• Dockerized (Dockerfile) for linux/amd64
• Published on DockerHub (registrywkm/eventop)
• Workflow: seed events/networks → run scan_jobs → internal RabbitMQ communication → processed data
Main components & boundaries
a. External Services (outside your app):
• Blockchain JSON-RPC endpoints (e.g. BSC testnet nodes)
• RabbitMQ broker
• PostgreSQL database
• Redis cache (via cache-manager)
b. Application Container (NestJS monolith):
• HTTP API Layer (src/apis/*)
• RabbitMQ Module (src/rabbitmq/*):
– Producer (eventmq-producer.service)
– Consumer (eventmq-consumer.service)
• Job Scheduler Module (job.module.ts + scripts/jobs/*)
• TypeORM Repositories (entities + migrations)
• Shared Utilities (JSON-RPC client, retry, enums, interceptors)
Relationships & data flows
1. Scanner Jobs
– Cron or CLI-driven jobs call blockchain via JSON-RPC → retrieve blocks & events → transform to internal models → publish to RabbitMQ.
2. Message Consumers
– RabbitMQ consumers listen on queues → process event-message payload → persist to PostgreSQL via TypeORM → optionally update status and republish.
3. HTTP API
– External clients call API controllers → service layer fetches from DB (or Redis cache) → returns paginated / filtered event data.
4. Seeding Scripts
– One-time scripts read config, write seed rows into PostgreSQL for networks & events to initialize scanning.
Architectural patterns & principles
• Modular Monolith (NestJS modules)
• Event-driven, message-oriented architecture (RabbitMQ decoupling)
• Hexagonal/Onion style: controllers → services → repositories → entities
• Retry & resilience (retry-manager util, web3 rate-limit exceptions)
• Caching layer (Redis) for high-frequency reads
• NestJS (Modules, Controllers, Providers)
• TypeORM + migrations → PostgreSQL
• amqplib/RabbitMQ
• Redis (cache-manager)
• JSON-RPC client for blockchain (web3 calls)
• Docker