# madara-alliance/madara-orchestrator architecture

> Architecture diagram of the GitHub repository madara-alliance/madara-orchestrator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/madara-alliance/madara-orchestrator
- Repository: https://github.com/madara-alliance/madara-orchestrator

## Overview

##### Main Architectural Components
- Orchestrator Service: The central component managing job creation, processing, verification, and error handling. Its internal logic is organized across several modules (CLI, controllers, jobs, workers, queues, and routes).
- Job Processing Queues: Multiple queues exist for different job phases (creation, processing, verification, and failure handling). These queues are either implemented internally or leverage AWS SQS.
- External Integrations and Clients:
- Data Availability (DA) Clients (Ethereum, and other DA clients in development).
- Prover Clients (ATLANTIC, SHARP).
- Settlement Clients (e.g., Ethereum, Starknet).
- Infrastructure and Storage Services:
- MongoDB for job state, logs, and error tracking.
- AWS Services for messaging (SQS), storage (S3), alerting (SNS) and scheduling (EventBridge).
- Docker, Localstack, and other testing tools for local development.
- External Dependencies:
- Madara Node and Pathfinder, which provide blockchain data and RPC endpoints required for SNOS, state updates, etc.

##### Component Interactions and Data Flow
- Cron/Worker Trigger: System begins with cron jobs or scheduled triggers that enqueue work requests on the “worker trigger queue”.
- Job Creation and Processing: Workers then inspect block data (using RPC endpoints from Pathfinder or Madara) to determine what jobs to create. New jobs are enqueued into dedicated processing queues based on type (SNOS, Proving, Data Submission, State Transition).
- Verification: After processing, jobs are moved to verification queues. The system then marks jobs as complete in MongoDB once verified.
- Error Handling: Failed jobs are diverted to a separate failure handling queue and retried based on defined retry intervals.
- External Communication: Depending on the type of job, the orchestrator interacts with external services – for example, submitting proofs to prover services or sending state updates via settlement clients.

##### Important Architectural Patterns and Technologies
- Modular Design – The project is organized into multiple crates (orchestrator, da-clients, prover-clients, settlement-clients, utils), each responsible for clear facets of functionality.
- Queue-Based Job Processing – Emphasize the three-phase job model (creation, processing, verification) and how queues isolate different processing stages.
- Integration with AWS & External Systems – Use of AWS SQS, SNS, S3, and EventBridge alongside blockchain RPC endpoints (through Pathfinder and Madara) illustrates a hybrid on-prem/cloud design.
- Error Handling and Retry Mechanisms – Indicate that the system is built to detect failures, retry jobs, and log errors in its persistent store (MongoDB).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Trigger & Ingestion Layer
    subgraph "Trigger & Ingestion"
        EventBridge["EventBridge Cron Triggers"]:::aws
        Cron["Cron/Worker Trigger"]:::internal
        TriggerQueue["Worker Trigger Queue"]:::queue
    end

    %% Core Orchestrator Service
    subgraph "Orchestrator Service"
        Orchestrator["Core Orchestrator"]:::internal
        CLI["CLI & Internal Modules"]:::internal
        JobProcessor["Job Processor"]:::internal
        QueueManager["Queue Manager"]:::queue
        WorkerModule["Worker Module"]:::internal
    end

    %% External Services & Clients
    subgraph "External Services & Clients"
        AWSQueue["AWS SQS Integration"]:::aws
        AWSSNS["AWS SNS Alerts"]:::aws
        AWSS3["AWS S3 Storage"]:::aws
        
        DAInterface["DA Client Interface"]:::external
        EthereumDA["Ethereum DA Client"]:::external
        
        ProverAtlantic["Atlantic Service"]:::external
        ProverSharp["Sharp Service"]:::external
        ProverGPS["GPS Fact Checker"]:::external
        
        SettleEthereum["Ethereum Settlement"]:::external
        SettleStarknet["Starknet Settlement"]:::external
        
        Madara["Madara Node"]:::external
        Pathfinder["Pathfinder Service"]:::external
        
        MongoDB["MongoDB Database"]:::db
    end

    %% Connections
    EventBridge --> Cron
    Cron --> TriggerQueue
    TriggerQueue --> JobProcessor
    CLI --> JobProcessor
    JobProcessor --> QueueManager
    JobProcessor --> WorkerModule
    QueueManager --> AWSQueue
    JobProcessor -->|"job update"| MongoDB
    JobProcessor -->|"fetchData"| DAInterface
    DAInterface --> EthereumDA
    JobProcessor -->|"submitProof"| ProverAtlantic
    JobProcessor -->|"submitProof"| ProverSharp
    JobProcessor -->|"checkFact"| ProverGPS
    ProverAtlantic --- ProverSharp
    ProverAtlantic --- ProverGPS
    JobProcessor -->|"stateUpdate"| SettleEthereum
    JobProcessor -->|"stateUpdate"| SettleStarknet
    SettleEthereum --- SettleStarknet
    JobProcessor -->|"blockData"| Madara
    JobProcessor -->|"RPCCall"| Pathfinder
    QueueManager -->|"notify"| AWSSNS
    JobProcessor -->|"storeFile"| AWSS3

    %% Click Events
    click Orchestrator "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/orchestrator"
    click CLI "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/orchestrator/src"
    click QueueManager "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/orchestrator/src/queue"
    click AWSQueue "https://github.com/madara-alliance/madara-orchestrator/blob/main/crates/orchestrator/src/cli/queue/aws_sqs.rs"
    click DAInterface "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/da-clients/da-client-interface"
    click EthereumDA "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/da-clients/ethereum"
    click ProverAtlantic "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/prover-clients/atlantic-service"
    click ProverSharp "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/prover-clients/sharp-service"
    click ProverGPS "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/prover-clients/gps-fact-checker"
    click SettleEthereum "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/settlement-clients/ethereum"
    click SettleStarknet "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/settlement-clients/starknet"
    click MongoDB "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/orchestrator/src/database/mongodb"
    click AWSSNS "https://github.com/madara-alliance/madara-orchestrator/tree/main/crates/orchestrator/src/alerts/aws_sns"
    click EventBridge "https://github.com/madara-alliance/madara-orchestrator/blob/main/crates/orchestrator/src/cli/cron/event_bridge.rs"
    click AWSS3 "https://github.com/madara-alliance/madara-orchestrator/blob/main/crates/orchestrator/src/cli/storage/aws_s3.rs"
    click Madara "https://github.com/madara-alliance/madara-orchestrator/tree/main/madara"
    click Pathfinder "https://github.com/madara-alliance/madara-orchestrator/tree/main/pathfinder"

    %% Styles
    classDef internal fill:#bbdefb,stroke:#0d47a1,stroke-width:2px;
    classDef external fill:#ffcc80,stroke:#e65100,stroke-width:2px;
    classDef aws fill:#c8e6c9,stroke:#1b5e20,stroke-width:2px;
    classDef db fill:#ffe082,stroke:#f57f17,stroke-width:2px;
    classDef queue fill:#d1c4e9,stroke:#512da8,stroke-width:2px;
```
