# wakumo/eventop architecture

> Architecture diagram of the GitHub repository wakumo/eventop, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wakumo/eventop
- Repository: https://github.com/wakumo/eventop

## Overview

##### Project type and purpose
- This is a Node.js microservice built on NestJS.
- It scans blockchain networks for events, processes them, stores results in PostgreSQL, and publishes/subscribes via RabbitMQ.
- It also exposes HTTP endpoints to query networks, events, and processed data.

##### File structure (key directories)
- src/apis – REST controllers & services for domain entities: networks, events, event-messages, processed-blocks
- src/scripts – one-off CLI scripts & seeding jobs (seed networks & events)
- src/rabbitmq – producer/consumer modules for RabbitMQ
- src/config – all environment, database (typeORM), Redis, RabbitMQ configurations
- src/commons – shared utils (JSON-RPC client to blockchain nodes, retry manager), enums, interfaces, custom exceptions, cache-manager (Redis)
- src/entities & src/migrations – TypeORM entities and migrations for PostgreSQL schema
- job.module.ts + src/scripts/jobs – scheduled “scan_events”, “rescan_events” jobs that poll the blockchain
- connection.ts / pg_client.ts / create_db.ts / drop_db.ts – direct DB setup scripts

##### Extract insights from README
- Dockerized (`Dockerfile`) for linux/amd64
- Published on DockerHub (registrywkm/eventop)
- Workflow: seed events/networks → run scan_jobs → internal RabbitMQ communication → processed data

##### Main components & boundaries
a. External Services (outside your app):
- Blockchain JSON-RPC endpoints (e.g. BSC testnet nodes)
- RabbitMQ broker
- PostgreSQL database
- Redis cache (via cache-manager)

b. Application Container (NestJS monolith):
- HTTP API Layer (src/apis/*)
- RabbitMQ Module (src/rabbitmq/*):
– Producer (eventmq-producer.service)
– Consumer (eventmq-consumer.service)
- Job Scheduler Module (job.module.ts + scripts/jobs/*)
- TypeORM Repositories (entities + migrations)
- Shared Utilities (JSON-RPC client, retry, enums, interceptors)

##### Relationships & data flows
1. **Scanner Jobs**
– Cron or CLI-driven jobs call blockchain via JSON-RPC → retrieve blocks & events → transform to internal models → publish to RabbitMQ.
2. **Message Consumers**
– RabbitMQ consumers listen on queues → process event-message payload → persist to PostgreSQL via TypeORM → optionally update status and republish.
3. **HTTP API**
– External clients call API controllers → service layer fetches from DB (or Redis cache) → returns paginated / filtered event data.
4. **Seeding Scripts**
– One-time scripts read config, write seed rows into PostgreSQL for networks & events to initialize scanning.

##### Architectural patterns & principles
- Modular Monolith (NestJS modules)
- Event-driven, message-oriented architecture (RabbitMQ decoupling)
- Hexagonal/Onion style: controllers → services → repositories → entities
- Retry & resilience (retry-manager util, web3 rate-limit exceptions)
- Caching layer (Redis) for high-frequency reads

- NestJS (Modules, Controllers, Providers)
- TypeORM + migrations → PostgreSQL
- amqplib/RabbitMQ
- Redis (cache-manager)
- JSON-RPC client for blockchain (web3 calls)
- Docker

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Services
    subgraph External_Services
        direction TB
        Blockchain_RPC["Blockchain JSON-RPC\n(8545)"]:::external
        RabbitMQ_Broker["RabbitMQ Broker\n(5672)"]:::broker
        PostgreSQL_DB["PostgreSQL DB"]:::database
        Redis_Cache["Redis Cache"]:::database
    end

    %% Application Container
    subgraph "NestJS Application" 
        direction TB

        %% Bootstrap
        subgraph "Bootstrap" 
            direction TB
            Main["main.ts"]:::app
            AppModule["app.module.ts"]:::app
            AppController["app.controller.ts"]:::app
            AppService["app.service.ts"]:::app
        end

        %% HTTP API Module
        subgraph "HTTP API Module (HTTP 3000)" 
            direction TB
            NetworksController["NetworksController"]:::api
            NetworksService["NetworksService"]:::api
            EventsController["EventsInternalController"]:::api
            EventsService["EventsService"]:::api
            EventMessageService["EventMessageService"]:::api
            ProcessedBlockService["ProcessedBlockService"]:::api
        end

        %% RabbitMQ Modules
        subgraph "RabbitMQ Modules" 
            direction TB
            EventmqAppModule["eventmq-app.module.ts"]:::app
            EventmqJobModule["eventmq-job.module.ts"]:::app
            Producer["EventMQProducerService"]:::app
            Consumer["EventMQConsumerService"]:::app
        end

        %% Job Scheduler & Scripts
        subgraph "Job Scheduler & Scripts" 
            direction TB
            JobModule["job.module.ts"]:::app
            ScanJobs["scan_events.job.ts"]:::app
            RescanJobs["rescan_events.job.ts"]:::app
            SendJobs["send_events.job.ts"]:::app
            DeleteJobs["delete_delivered_events.job.ts"]:::app
            SampleJob["sample.job.ts"]:::app
        end

        %% Persistence Layer
        subgraph "Persistence Layer" 
            direction TB
            Entities["Entities"]:::persistence
            Migrations["Migrations"]:::persistence
        end

        %% Shared Commons & Config
        subgraph "Shared Commons & Config" 
            direction TB
            JSONRPCClient["json-rpc-client.ts"]:::common
            RetryManager["retry-manager.ts"]:::common
            CacheModule["cache-manager.module.ts"]:::common
            CacheService["cache-manager.service.ts"]:::common
            Enums["Enums"]:::common
            Exceptions["Exceptions"]:::common
            Interceptor["serializer.interceptor.ts"]:::common
            Configs["Config Modules"]:::common
        end
    end

    %% Data Flows
    Blockchain_RPC -->|"JSON-RPC calls"| ScanJobs
    ScanJobs -->|"publish AMQP"| Producer
    Producer -->|AMQP publish| RabbitMQ_Broker
    RabbitMQ_Broker -->|AMQP message| Consumer
    Consumer -->|"SQL queries"| Entities
    Consumer -->|AMQP republish| RabbitMQ_Broker

    NetworksController -->|HTTP| NetworksService
    EventsController -->|HTTP| EventsService
    NetworksService -->|SQL/Cache| Entities
    EventsService -->|SQL/Cache| Entities
    EventMessageService -->|SQL/Cache| Entities
    ProcessedBlockService -->|SQL/Cache| Entities

    CacheService -->|GET/SET| Redis_Cache
    Entities -->|persist| PostgreSQL_DB

    Main --> AppModule
    AppModule --> AppController
    AppModule --> AppService

    %% Click events for component mapping
    click NetworksController "https://github.com/wakumo/eventop/blob/main/src/apis/network/network.controller.ts"
    click NetworksService "https://github.com/wakumo/eventop/blob/main/src/apis/network/network.service.ts"
    click EventsController "https://github.com/wakumo/eventop/blob/main/src/apis/events/events-internal.controller.ts"
    click EventsService "https://github.com/wakumo/eventop/blob/main/src/apis/events/events.service.ts"
    click EventMessageService "https://github.com/wakumo/eventop/blob/main/src/apis/event-message/event-message.service.ts"
    click ProcessedBlockService "https://github.com/wakumo/eventop/blob/main/src/apis/processed-block/processed-block.service.ts"
    click EventmqAppModule "https://github.com/wakumo/eventop/blob/main/src/rabbitmq/eventmq-app.module.ts"
    click EventmqJobModule "https://github.com/wakumo/eventop/blob/main/src/rabbitmq/eventmq-job.module.ts"
    click Producer "https://github.com/wakumo/eventop/blob/main/src/rabbitmq/services/eventmq-producer.service.ts"
    click Consumer "https://github.com/wakumo/eventop/blob/main/src/rabbitmq/services/eventmq-consumer.service.ts"
    click JobModule "https://github.com/wakumo/eventop/blob/main/src/job.module.ts"
    click ScanJobs "https://github.com/wakumo/eventop/blob/main/src/scripts/jobs/scan_events.job.ts"
    click RescanJobs "https://github.com/wakumo/eventop/blob/main/src/scripts/jobs/rescan_events.job.ts"
    click SendJobs "https://github.com/wakumo/eventop/blob/main/src/scripts/jobs/send_events.job.ts"
    click DeleteJobs "https://github.com/wakumo/eventop/blob/main/src/scripts/jobs/delete_delivered_events.job.ts"
    click SampleJob "https://github.com/wakumo/eventop/blob/main/src/scripts/jobs/sample.job.ts"
    click Entities "https://github.com/wakumo/eventop/tree/main/src/entities"
    click Migrations "https://github.com/wakumo/eventop/tree/main/src/migrations"
    click JSONRPCClient "https://github.com/wakumo/eventop/blob/main/src/commons/utils/json-rpc-client.ts"
    click RetryManager "https://github.com/wakumo/eventop/blob/main/src/commons/utils/retry-manager.ts"
    click CacheModule "https://github.com/wakumo/eventop/blob/main/src/commons/cache-manager/cache-manager.module.ts"
    click CacheService "https://github.com/wakumo/eventop/blob/main/src/commons/cache-manager/cache-manager.service.ts"
    click Interceptor "https://github.com/wakumo/eventop/blob/main/src/commons/interceptors/serializer.interceptor.ts"
    click Main "https://github.com/wakumo/eventop/blob/main/src/main.ts"
    click AppModule "https://github.com/wakumo/eventop/blob/main/src/app.module.ts"
    click AppController "https://github.com/wakumo/eventop/blob/main/src/app.controller.ts"
    click AppService "https://github.com/wakumo/eventop/blob/main/src/app.service.ts"

    %% Styles
    classDef external fill:#D6E9FF,stroke:#3399FF,color:#000
    classDef broker fill:#FFE0B3,stroke:#FF9900,color:#000
    classDef database fill:#E2F0D9,stroke:#33A02C,color:#000
    classDef app fill:#F2F2F2,stroke:#888,color:#000
    classDef api fill:#F2F2F2,stroke:#888,color:#000
    classDef persistence fill:#F9F9F9,stroke:#555,color:#000
    classDef common fill:#F7F7F7,stroke:#666,color:#000
```
