# helium/oracles architecture

> Architecture diagram of the GitHub repository helium/oracles, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/helium/oracles
- Repository: https://github.com/helium/oracles

## Overview

##### Overall Project Purpose and Type
- This is a modular, multi-service system that is part of the Helium “Oracles” project. It’s clearly built as a distributed set of microservices handling both Mobile and IoT data flows.

##### File Structure
- There are many modules such as boost_manager, file_store, ingest, iot_config, and several mobile and IoT related libraries. This indicates the project is organized as multiple microservices or components that perform very specific tasks (e.g., reward calculation, data ingestion, price tracking, etc.).
- Migration scripts, settings templates, and tests all exist in each module, meaning that each component can be separately deployed and scaled.
- There is also tooling for tracing, metrics, and orchestration (e.g., custom_tracing, task_manager) that are important from an observability and management perspective.

##### Main Components and Their Interactions
- For Mobile:
- Mobile Ingestor: Gathers heartbeats, speedtests, data transfer sessions, and other mobile data.
- Mobile Verifier: Validates incoming mobile data and calculates rewards.
- Mobile Packet Verifier: Handles data transfer and burns DC on Solana.
- Mobile Price: Records price data (e.g., via Pyth) and communicates with Solana.
- Mobile Config: Acts as a central access point to on-chain data and stores pubkeys.
- Mobile Reward Index: Aggregates computed rewards that are then written into an external (foundation-owned) database.

- For IoT:
- IoT Ingestor: Gathers packets from hotspots, beacons, witnesses, and maintains long-lived gRPC streams.
- IoT Verifier: Validates IoT data, calculates rewards, and processes associated entropy.
- IoT Config: Provides settings for on-chain data and stores organization and routing information.
- Helium Packet Router: Routes packets between hotspots and the Helium LNS.
- IoT Packet Verifier: Burns DC for IoT data transfers.
- IoT Entropy: Generates system entropy used by other services.
- IoT Reward Index: Aggregates IoT rewards and writes them to the foundation-owned database.
- External Components:
- External Databases: Two types are evident – one for initial configuration/data population (Foundation owned db populated by Helium) and one that stores reward totals.
- Solana is a key external blockchain system, and its role is shown for Mobile Price and Packet Verifiers.

##### Technical Stack and Architectural Patterns
- The project is built in Rust with a microservices approach, where each service has its own module and deployment pipeline.
- Out configuration management components (like Mobile Config and IoT Config) which are central for controlling service behavior.
- If needed, show testing layers or migration processes as side notes to emphasize robust deployment practices.

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% Support Services
    subgraph "Support Services"
        Observability["Observability/Tracing"]:::support
        TaskManager["Task/Job Management"]:::support
    end

    %% Mobile Subsystem
    subgraph "Mobile Subsystem"
        MI["Mobile Ingestor<br/>(heartbeats,speedtests, data transfer)"]:::mobile
        MV["Mobile Verifier<br/>(validates mobile data)"]:::mobile
        MPV["Mobile Packet Verifier<br/>(data transfer, burns DC)"]:::mobile
        MC["Mobile Config<br/>(on-chain config, pubkeys)"]:::mobile
        MP["Mobile Price<br/>(price data, Solana comms)"]:::mobile
        MRI["Mobile Reward Index<br/>(aggregates rewards)"]:::mobile
    end

    %% IoT Subsystem
    subgraph "IoT Subsystem"
        II["IoT Ingestor<br/>(packets from hotspots/beacons)"]:::iot
        IV["IoT Verifier<br/>(validates IoT data, calculates rewards)"]:::iot
        IC["IoT Config<br/>(on-chain settings, routing info)"]:::iot
        IPV["IoT Packet Verifier<br/>(burns DC for transfers)"]:::iot
        IE["IoT Entropy<br/>(generates system entropy)"]:::iot
        IRI["IoT Reward Index<br/>(aggregates IoT rewards)"]:::iot
        HPR["Helium Packet Router<br/>(routes packets to LNS)"]:::iot
    end

    %% External Systems
    subgraph "External Systems"
        SOL["External Blockchain (Solana)"]:::external
        EXPDB["Foundation Database"]:::external
    end

    %% Connections from Support Services
    Observability --- MI
    Observability --- IV
    TaskManager --- MV
    TaskManager --- IV

    %% Mobile Subsystem Connections
    MI -->|"S3"| MV
    MV -->|"S3"| MPV
    MPV -->|"gRPC"| MC
    MC -->|"config"| MP
    MPV -->|"burnDC"| SOL
    MP -->|"priceTx"| SOL
    MV -->|"rewardData"| MRI
    MRI -->|"writesRewards"| EXPDB

    %% IoT Subsystem Connections
    II -->|"gRPC"| IV
    II -->|"routes"| HPR
    IC -->|"configData"| IV
    IE -->|"entropy"| IV
    IV -->|"rewardCalc"| IRI
    IRI -->|"writesRewards"| EXPDB
    IPV -->|"burnDC"| SOL

    %% Click Events for Mobile Components
    click MI "https://github.com/helium/oracles/blob/main/ingest/src/server_mobile.rs"
    click MV "https://github.com/helium/oracles/blob/main/mobile_verifier/src/main.rs"
    click MPV "https://github.com/helium/oracles/blob/main/mobile_packet_verifier/src/main.rs"
    click MP "https://github.com/helium/oracles/blob/main/price/src/main.rs"
    click MC "https://github.com/helium/oracles/blob/main/mobile_config/src/main.rs"
    click MRI "https://github.com/helium/oracles/blob/main/reward_index/src/main.rs"

    %% Click Events for IoT Components
    click II "https://github.com/helium/oracles/blob/main/ingest/src/server_iot.rs"
    click IV "https://github.com/helium/oracles/blob/main/iot_verifier/src/main.rs"
    click IC "https://github.com/helium/oracles/blob/main/iot_config/src/main.rs"
    click IPV "https://github.com/helium/oracles/blob/main/iot_packet_verifier/src/main.rs"
    click IE "https://github.com/helium/oracles/blob/main/poc_entropy/src/main.rs"

    %% Click Event for External Blockchain
    click SOL "https://github.com/helium/oracles/blob/main/solana/src/main.rs"

    %% Click Events for Support Services
    click Observability "https://github.com/helium/oracles/blob/main/custom_tracing/src/lib.rs"
    click TaskManager "https://github.com/helium/oracles/blob/main/task_manager/src/lib.rs"

    %% Style Definitions
    classDef mobile fill:#cfe2f3,stroke:#0b5394,stroke-width:2px;
    classDef iot fill:#fce5cd,stroke:#e69138,stroke-width:2px;
    classDef external fill:#d9ead3,stroke:#6aa84f,stroke-width:2px;
    classDef support fill:#ead1dc,stroke:#b854a2,stroke-width:2px;
```
