# 0xintuition/intuition-rs architecture

> Architecture diagram of the GitHub repository 0xintuition/intuition-rs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/0xintuition/intuition-rs
- Repository: https://github.com/0xintuition/intuition-rs

## Overview

##### Project type and purpose
- This is a full-stack Rust workspace (“intuition-rust”) that implements a blockchain-event processing pipeline.
- It ingests on-chain data (via Substreams or Envio), decodes and enriches it with resolvers and IPFS uploads, stores it in Postgres via Hasura, and exposes it through GraphQL, REST, and a TUI/CLI.

##### Top-level components
- Data Ingestion
- substreams-sink (Substreams event consumer)
- envio-indexer (Envio-based on-chain indexer)
- Consumers
- RAW, DECODED, RESOLVER — each reads from AWS SQS queues
- histoflux (streams historical + live DB events back to SQS)
- Data Store & API
- Hasura + PostgreSQL (metadata, migrations, GraphQL)
- consumer-api (refetch API for atoms)
- Auxiliary Services
- rpc-proxy (caches eth_call results for EthMultiVault)
- image-guard (uploads/validates images to IPFS/Pinata)
- rpc providers (Alchemy, local Geth node)
- external tokens/services (Pinata, HuggingFace, Substreams API, AWS SQS)
- Client Interfaces
- cli (TUI CLI)
- local Docker Compose / Minikube / EKS orchestrations

##### Data flow and relationships
- On-chain → substreams-sink or envio-indexer → writes to Postgres
- Postgres ← Hasura migrations & config applied
- histoflux tail DB changes → sends to SQS
- RAW consumer reads raw logs from SQS, decodes → writes enriched data back to Postgres
- DECODED consumer reads from decoded-queue, enriches or parses → Postgres
- RESOLVER consumer reads from resolver-queue, resolves ENS/IPFS references → Postgres
- image-guard receives HTTP uploads → IPFS (via Pinata) → metadata written to Postgres
- rpc-proxy fronts Ethereum RPC calls → caches currentSharePrice → returns JSON RPC
- consumer-api fronts refetch requests → queries Postgres and/or RPC-proxy
- Hasura GraphQL service sits atop Postgres → exposes GraphQL endpoint
- cli app queries GraphQL or consumer-api for data display

##### Architectural patterns
- Microservices: each crate compiles to its own container/microservice.
- Event-driven: uses SQS for decoupling consumers.
- CQRS/Streaming: histoflux streams DB events to SQS; consumers update write models.
- API Layer: Hasura for GraphQL, custom REST via consumer-api and rpc-proxy.
- Feature-flagged behavior: consumer supports local vs. remote via Cargo features.

##### Technologies & frameworks
- Rust crates managed in a Cargo workspace.
- Docker / docker-compose for local development.
- AWS SQS for queueing.
- Envoy vs. Substreams ingestion.
- Hasura CLI + metadata for GraphQL layer.
- Kubernetes manifests (ArgoCD, Kustomize) for EKS or Minikube.
- PostgreSQL as primary datastore.
- External: Alchemy nodes, Pinata IPFS gateway, HuggingFace, Substreams API.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% On-chain & Ingestion Layer
    subgraph "On-chain & Ingestion" 
        direction TB
        SubstreamsAPI["Substreams API"]:::external
        EnvioAPI["Envio Indexer API"]:::external
        SSink["substreams-sink"]:::ingestion
        EIndex["envio-indexer"]:::ingestion
    end

    %% Queues
    subgraph "AWS SQS" 
        direction TB
        RAWQ["RAW Queue"]:::queue
        DECQ["DECODED Queue"]:::queue
        RESQ["RESOLVER Queue"]:::queue
    end

    %% Processing Consumers
    subgraph "Consumers" 
        direction TB
        Histoflux["histoflux"]:::ingestion
        HistoCrawler["histocrawler"]:::ingestion
        RAWConsumer["RAW Consumer"]:::processing
        DECConsumer["DECODED Consumer"]:::processing
        RESConsumer["RESOLVER Consumer"]:::processing
    end

    %% Storage & API Layer
    subgraph "Storage & API" 
        direction TB
        Postgres["PostgreSQL"]:::database
        Hasura["Hasura GraphQL"]:::graphql
        ConsumerAPI["consumer-api (REST)"]:::rest
        RPCProxy["rpc-proxy"]:::rest
        ImageGuard["image-guard"]:::processing
    end

    %% Clients
    subgraph "Clients" 
        direction TB
        CLI["CLI/TUI Client"]:::client
    end

    %% External Services
    subgraph "External Services" 
        direction TB
        EthRPC["Ethereum RPC Providers"]:::external
        Pinata["Pinata IPFS"]:::external
        HF["HuggingFace"]:::external
    end

    %% Data Flows
    SubstreamsAPI -->|on-chain events| SSink
    EnvioAPI -->|on-chain events| EIndex
    SSink -->|writes data| Postgres
    EIndex -->|writes data| Postgres

    Postgres -->|tail changes| Histoflux
    Histoflux -->|events| RAWQ
    HistoCrawler -->|backfill| RAWQ

    RAWQ -->|raw logs| RAWConsumer
    RAWConsumer -->|decoded data| Postgres
    DECQ -->|parsed logs| DECConsumer
    DECConsumer -->|enriched data| Postgres
    RESQ -->|resolve refs| RESConsumer
    RESConsumer -->|resolved data| Postgres

    Postgres --> Hasura
    Postgres --> ConsumerAPI
    Postgres --> RPCProxy

    ImageGuard -->|upload metadata| Postgres
    CLI -->|GraphQL queries| Hasura
    CLI -->|REST calls| ConsumerAPI
    CLI -->|RPC calls| RPCProxy

    EthRPC -->|RPC calls| RPCProxy
    ImageGuard -->|IPFS upload| Pinata
    ImageGuard -->|model inference| HF

    %% Queue feed for processors
    RAWConsumer -.->|push decoded| DECQ
    DECConsumer -.->|push resolve| RESQ

    %% Click Events
    click SSink "https://github.com/0xintuition/intuition-rs/tree/main/substreams-sink/"
    click EIndex "https://github.com/0xintuition/intuition-rs/tree/main/envio-indexer/"
    click RAWConsumer "https://github.com/0xintuition/intuition-rs/tree/main/consumer/src/mode/raw/"
    click DECConsumer "https://github.com/0xintuition/intuition-rs/tree/main/consumer/src/mode/decoded/"
    click RESConsumer "https://github.com/0xintuition/intuition-rs/tree/main/consumer/src/mode/resolver/"
    click Histoflux "https://github.com/0xintuition/intuition-rs/tree/main/histoflux/"
    click HistoCrawler "https://github.com/0xintuition/intuition-rs/tree/main/histocrawler/"
    click Hasura "https://github.com/0xintuition/intuition-rs/tree/main/hasura/"
    click ConsumerAPI "https://github.com/0xintuition/intuition-rs/tree/main/consumer-api/"
    click RPCProxy "https://github.com/0xintuition/intuition-rs/tree/main/rpc-proxy/"
    click ImageGuard "https://github.com/0xintuition/intuition-rs/tree/main/image-guard/"
    click CLI "https://github.com/0xintuition/intuition-rs/tree/main/cli/"
    click dockerCompose "https://github.com/0xintuition/intuition-rs/blob/main/docker-compose.yml"
    click kubernetes "https://github.com/0xintuition/intuition-rs/tree/main/devops/kubernetes/"
    click minikube "https://github.com/0xintuition/intuition-rs/tree/main/devops/minikube/"
    click eksRoll "https://github.com/0xintuition/intuition-rs/tree/main/devops/aws-rolling-releases/"
    click eks "https://github.com/0xintuition/intuition-rs/tree/main/devops/aws/"
    click argocd "https://github.com/0xintuition/intuition-rs/tree/main/devops/argocd/"
    click geth "https://github.com/0xintuition/intuition-rs/tree/main/geth/"
    click initDB "https://github.com/0xintuition/intuition-rs/blob/main/init-dbs.sh"

    %% Styles
    classDef ingestion fill:#BBDEFB,stroke:#1E88E5,stroke-width:1px
    classDef queue fill:#E1F5FE,stroke:#0288D1,stroke-width:1px
    classDef processing fill:#FFE0B2,stroke:#FB8C00,stroke-width:1px
    classDef database fill:#C8E6C9,stroke:#388E3C,stroke-width:1px
    classDef graphql fill:#A5D6A7,stroke:#2E7D32,stroke-width:1px
    classDef rest fill:#E1BEE7,stroke:#8E24AA,stroke-width:1px
    classDef client fill:#CFD8DC,stroke:#607D8B,stroke-width:1px
    classDef external fill:#F5F5F5,stroke:#9E9E9E,stroke-width:1px
```
