# subquery/subql architecture

> Architecture diagram of the GitHub repository subquery/subql, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/subquery/subql
- Repository: https://github.com/subquery/subql

## Overview

##### Project type and purpose
- This is a multi-package monorepo for SubQuery, a decentralized, blockchain-agnostic data indexing framework.
- It provides three primary deliverables:
- A CLI tool to scaffold, build, deploy and manage SubQuery projects (@subql/cli)
- An indexer node service that ingests blockchain data and writes to a Postgres/CockroachDB database (@subql/node & @subql/node-core)
- A GraphQL query API service to expose indexed data (@subql/query)
- Shared libraries support manifest loading, type definitions, multi-chain support, utilities, testing, and common substrate specifics.

##### File structure
- Top-level “packages” folder holds each deliverable and shared libs: cli, node, query, common, common-substrate, types, utils, testing, etc.
- “deploy/k8s” holds Kubernetes manifests; Dockerfiles live alongside services and in “.github/workflows” for CI/CD.
- Each service uses NestJS modules (see node/src/meta, node/src/indexer; query/src/graphql).
- The CLI is a TypeScript binary exposing JSON-RPC clients, project scaffolding, codegen, build, deploy subcommands.

##### README insights
- Purpose: enable developers to index and query data from Polkadot/Substrate, Ethereum, Cosmos, and other chains.
- Components listed: @subql/cli, @subql/node, @subql/query, @subql/common, etc.
- Deployment guidance: Docker, Postgres database, blockchain node RPC.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Developer environment
    Developer["Developer Terminal"]:::developer
    Developer -->|"uses"| CLI

    %% CLI tool
    CLI["SubQuery CLI"]:::devtool
    click CLI "https://github.com/subquery/subql/tree/main/packages/cli"

    %% Runtime Services
    subgraph "Runtime Services"
        direction TB
        NodeService["Indexer Node Service"]:::service
        click NodeService "https://github.com/subquery/subql/tree/main/packages/node"
        NodeServiceCore["Node Core Library"]:::service
        click NodeServiceCore "https://github.com/subquery/subql/tree/main/packages/node-core"
        QueryService["GraphQL Query API Service"]:::service
        click QueryService "https://github.com/subquery/subql/tree/main/packages/query"
    end

    %% Node internals
    subgraph "Indexer Node Internals"
        direction TB
        AdminAPI["Admin API Module"]:::service
        Dispatcher["Block Dispatcher"]:::service
        DSProcessor["Data Source Processor"]:::service
        Dictionary["Dictionary Cache"]:::service
        Store["StoreService (DB Writer)"]:::service
        Worker["Worker Pool"]:::service
    end
    NodeService --> AdminAPI
    NodeService --> Dispatcher
    NodeService --> DSProcessor
    NodeService --> Dictionary
    NodeService --> Store
    NodeService --> Worker

    %% Shared Libraries
    subgraph "Shared Libraries"
        direction TB
        CommonLib["@subql/common"]:::library
        click CommonLib "https://github.com/subquery/subql/tree/main/packages/common"
        CommonSub["@subql/common-substrate"]:::library
        click CommonSub "https://github.com/subquery/subql/tree/main/packages/common-substrate"
        TypesCore["@subql/types-core"]:::library
        click TypesCore "https://github.com/subquery/subql/tree/main/packages/types-core"
        Types["@subql/types"]:::library
        click Types "https://github.com/subquery/subql/tree/main/packages/types"
        UtilsLib["@subql/utils"]:::library
        click UtilsLib "https://github.com/subquery/subql/tree/main/packages/utils"
        Testing["@subql/testing"]:::library
        click Testing "https://github.com/subquery/subql/tree/main/packages/testing"
    end

    %% External Systems
    subgraph "External Systems"
        direction TB
        Blockchain["Blockchain RPC/WS Endpoints"]:::external
        Clients["GraphQL Clients"]:::external
    end

    %% Data Store
    DB["Postgres/CockroachDB"]:::database

    %% CI/CD Pipeline
    subgraph "CI/CD & Deployment"
        direction TB
        CI["GitHub Actions"]:::cicd
        click CI "https://github.com/subquery/subql/tree/main/.github/workflows"
        Docker["Docker Images"]:::cicd
        K8s["Kubernetes Manifests"]:::cicd
        click K8s "https://github.com/subquery/subql/blob/main/deploy/k8s/deploy.yaml"
    end

    %% Data Flows and Dependencies
    CLI -->|"JSON-RPC"| NodeService
    NodeService -->|"uses"| NodeServiceCore
    CLI -->|"uses"| CommonLib
    CLI -->|"uses"| CommonSub
    CLI -->|"uses"| TypesCore
    CLI -->|"uses"| Types
    CLI -->|"uses"| UtilsLib
    CLI -->|"uses"| Testing

    NodeService -->|"depends on"| CommonLib
    NodeService -->|"depends on"| CommonSub
    NodeService -->|"depends on"| TypesCore
    NodeService -->|"depends on"| Types
    NodeService -->|"depends on"| UtilsLib
    NodeService --> Blockchain
    NodeService --> DB

    QueryService -->|"reads from"| DB
    QueryService --> Clients
    QueryService -->|"depends on"| CommonLib
    QueryService -->|"depends on"| TypesCore
    QueryService -->|"depends on"| Types
    QueryService -->|"depends on"| UtilsLib

    CI --> Docker
    Docker -->|"build images for"| NodeService
    Docker -->|"build images for"| QueryService
    Docker --> K8s
    K8s -->|"deploys"| NodeService
    K8s -->|"deploys"| QueryService

    %% Styles
    classDef developer fill:#D0E6FF,stroke:#0366d6,color:#0366d6;
    classDef devtool fill:#C8E6C9,stroke:#2E7D32,color:#2E7D32;
    classDef service fill:#E8F5E9,stroke:#388E3C,color:#388E3C;
    classDef library fill:#EDE7F6,stroke:#5E35B1,color:#5E35B1;
    classDef database fill:#FFF9C4,stroke:#FBC02D,color:#F57F17;
    classDef external fill:#ECEFF1,stroke:#607D8B,color:#455A64;
    classDef cicd fill:#FFF3E0,stroke:#F57C00,color:#E65100;
```
