# agoric/agoric-subql architecture

> Architecture diagram of the GitHub repository agoric/agoric-subql, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/agoric/agoric-subql
- Repository: https://github.com/agoric/agoric-subql

## Overview

##### Project Type and Purpose
- The indexer is designed to process blockchain events from an Agoric node (or A3P instance) and index those events into a data store. It then exposes the data (including metadata, vault statistics, and various financial metrics) through a GraphQL endpoint.
- Its usage scenarios range from serving an explorer view to powering dashboards like the inter dashboard. The README spells out how to use the indexer and troubleshoot common issues.

##### Analyzing the File Structure
- Top-Level Directories:
- .github: Contains GitHub Actions and workflow files (for testing and CI/CD pipelines). This highlights automated testing workflows (PR tests, liquidation testing, vault and reserve metrics testing).
- docker & docker-compose.yml: Indicate use of Docker for creating containerized environments for running the indexer and its dependencies.
- proto: Contains Protocol Buffer definitions for various Agoric-related modules (lien, swingset, vbank, vibc, vstorage) as well as definitions from cosmos. These indicate that the indexer deals with typed network messages and event formats.
- scripts: Contains Node.js scripts with MJS extensions that appear to interact with vaults, pricing, and on-chain data operations, likely aiding testing or operational workflows.
- src: Hosts the actual indexer source code. Notably, the “mappings” subfolder includes event mappings, handler files, and utilities that process blockchain events and transform them into indexed (and queryable) data.
- Other configuration files (e.g., package.json, tsconfig.json, .prettierrc, yarnrc.yml) support tooling, code style, and project building.

##### Main Components
a. Agoric Blockchain / Data Source:
- Blockchain node or A3P instance that provides the state-change events. Indicate that this is the external “data provider” where events originate (and note that interactions are done via RPC endpoints).

b. Indexer Ingestion & Processing Module:
- The core of the project, illustrated by the “src” directory, particularly the mappings folder.
- Sub-components for:
- Event Listener / Fetcher: A module that receives state changes from the blockchain.
- Mapping Handlers: Components in src/mappings/ that parse different event types (e.g., vaults, reserves, price feed, etc.) and apply logic to generate database entries.
- Utility Functions: In src/mappings/utils.ts, which support event processing.

c. Data Storage/Database:
- Although not explicitly detailed, indexers like these typically use a database (managed via SubQuery) to store the processed events and aggregated metrics. Represent this as a dedicated storage component.

d. GraphQL API Service:
- The indexer exposes a GraphQL endpoint (accessible at http://localhost:3000 per README). This is used for querying the indexed data.

e. External Interaction / Clients:
- Clients such as the “inter dashboard” or any subscriber to the GraphQL endpoint should be represented. Mark them as external consumers querying the API.

f. Supplementary Components:
- Docker and Containerization: Represent the docker-compose setup and docker scripts (e.g., docker/load-extensions.sh, docker/pg-Dockerfile) that support the deployment and environment orchestration.
- CI/CD Pipelines: A component symbolizing GitHub Actions workflows (from .github/workflows) that run tests such as liquidation and vaults/reserve metrics tests. This ensures code quality and smooth deployment.

##### Relationships and Data Flow
- From the ingestion layer, showcase processing steps (using mapping handlers) that transform the events and then store them in the database.
- How the GraphQL API queries this database and supplies data to external clients (e.g., dashboards).
- That containerization (Docker) and CI/CD (GitHub Actions) help in deploying and testing the entire system.

##### Architectural Patterns and Design Principles
- Event-driven design where incoming blockchain events are transformed and then stored.
- Demonstrate clear separation of concerns: data ingestion, processing, storage, and querying.
- That the use of Protocol Buffers for defining message formats (within the proto folder) is an important design decision ensuring consistency in data representation.
- Mention that the system leverages a microservices style in terms of containerization even if the components are collocated in one repository (e.g., indexer code, GraphQL API, orchestration scripts).

##### Technologies and Frameworks
- SubQuery: The backbone for indexing blockchain events.
- GraphQL: Used for the API that exposes the indexed data.
- Docker: For containerizing the application.
- Node.js and TypeScript: As indicated by .ts files, package.json, and tsconfig.json.
- Protocol Buffers: Shown in the proto directory for message definitions.

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["Agoric Blockchain (A3P Instance)"]:::external

    subgraph "Indexer Engine"
        CI["Core Ingestion"]:::processing
        MH["Mapping Handlers"]:::processing
        UF["Utility Functions"]:::processing
        subgraph "Event Handlers"
            VH["Vaults Handler"]:::processing
            RH["Reserves Handler"]:::processing
            PH["Price Feed Handler"]:::processing
        end
    end

    DB["Database Storage (SubQuery)"]:::database
    API["GraphQL API Service"]:::api
    EC["External Clients (Dashboard, etc.)"]:::external

    %% Main Data Flow
    A --> CI
    CI --> MH
    MH --> VH
    MH --> RH
    MH --> PH
    UF --supports--> MH
    VH --> DB
    RH --> DB
    PH --> DB
    DB --> API
    API --> EC

    %% Deployment & Operations
    subgraph "Deployment & Operations"
        subgraph "Docker Containerization"
            DC["Docker Compose Setup"]:::ops
            DL["Load Extensions Script"]:::ops
            DP["PG Dockerfile"]:::ops
        end
        subgraph "CI/CD Pipelines"
            GH1["GitHub Workflow: Liquidation"]:::ops
            GH2["GitHub Workflow: PR"]:::ops
            GH3["GitHub Workflow: Vaults&ReserveMetrics"]:::ops
            GH4["GitHub Action: Notify Status"]:::ops
        end
        OS["Operational Scripts"]:::ops
    end

    %% Operational Support (dotted lines)
    DC --- CI
    DL --- CI
    DP --- CI
    GH1 --- CI
    GH2 --- CI
    GH3 --- CI
    GH4 --- CI
    OS --- CI
    OS --- API

    %% Click Events
    click CI "https://github.com/agoric/agoric-subql/blob/main/src/index.ts"
    click MH "https://github.com/agoric/agoric-subql/blob/main/src/mappings/mappingHandlers.ts"
    click VH "https://github.com/agoric/agoric-subql/blob/main/src/mappings/events/vaults.ts"
    click RH "https://github.com/agoric/agoric-subql/blob/main/src/mappings/events/reserves.ts"
    click PH "https://github.com/agoric/agoric-subql/blob/main/src/mappings/events/priceFeed.ts"
    click UF "https://github.com/agoric/agoric-subql/blob/main/src/mappings/utils.ts"
    click API "https://github.com/agoric/agoric-subql/blob/main/schema.graphql"
    click DC "https://github.com/agoric/agoric-subql/blob/main/docker-compose.yml"
    click DL "https://github.com/agoric/agoric-subql/blob/main/docker/load-extensions.sh"
    click DP "https://github.com/agoric/agoric-subql/tree/main/docker/pg-Dockerfile"
    click GH1 "https://github.com/agoric/agoric-subql/blob/main/.github/workflows/liquidation.yml"
    click GH2 "https://github.com/agoric/agoric-subql/blob/main/.github/workflows/pr.yaml"
    click GH3 "https://github.com/agoric/agoric-subql/blob/main/.github/workflows/vaults-and-reserve-metrics-testing.yml"
    click GH4 "https://github.com/agoric/agoric-subql/blob/main/.github/actions/notify-status/action.yml"
    click OS "https://github.com/agoric/agoric-subql/tree/main/scripts/"

    %% Styles
    classDef external fill:#ffd700,stroke:#333,stroke-width:2px;
    classDef processing fill:#add8e6,stroke:#333,stroke-width:2px;
    classDef database fill:#90ee90,stroke:#333,stroke-width:2px;
    classDef api fill:#ffb6c1,stroke:#333,stroke-width:2px;
    classDef ops fill:#ffe4b5,stroke:#333,stroke-width:2px;

    class A,EC external
    class CI,MH,VH,RH,PH,UF processing
    class DB database
    class API api
    class DC,DL,DP,GH1,GH2,GH3,GH4,OS ops
```
