# gensyn-ai/rl-swarm architecture

> Architecture diagram of the GitHub repository gensyn-ai/rl-swarm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gensyn-ai/rl-swarm
- Repository: https://github.com/gensyn-ai/rl-swarm

## Overview

##### Project Overview and Purpose
- RL Swarm is built for peer-to-peer reinforcement learning where each node trains its own model while collaborating via a gossip-based distributed network (using Hivemind).
- The system includes on-chain identity management (via modal-login and Alchemy) to track peer progress and participation.
- There is a front-end UI (built in a Next.js/React framework) available through the Swarm UI to monitor swarm activity, as well as containerization and CI/CD support using Docker, docker-compose, and Buildkite pipelines.

##### Main Components to Identify
A. RL Training and Peer Modules:
- Represented primarily by the “hivemind_exp” directory.
- Contains key utilities and modules:
⁃ chain_utils, dht_utils, gas_strategy for blockchain interactions and decentralized communication.
⁃ Modules for varied reinforcement learning tasks (e.g., trainer, gsm8k for data/prompt generation, and runner for executing RL rounds).
- Contains configuration directories (configs/gpu and configs/mac) which adjust training parameters depending on the hardware.
- Testing subcomponents ensure the integrity of its RL and DHT functionalities.

B. Authentication and Identity Management:
- Managed by the “modal-login” directory.
- Implements the login flow in a Next.js (or similar) application.
- Handles API routes for obtaining API keys, registering peers, and setting activation status.
- Manages the creation and management of the “swarm.pem” file for on-chain identity.
- Integrates with external identity systems (e.g., Alchemy’s modal sign-in).

C. User Interface and API Backend for Swarm Monitoring:
- Composed of components in the “web” directory.
- The “web/api” folder contains the backend API server scripts (global_dht, gossip_utils, server, server_cache, server_test) that might interface with the RL training node.
- The “web/ui” directory holds the front-end code (React components, styles, configurations) that render the Swarm UI.
- These components allow users to visually monitor training, authentication status, and RL progress.

D. Containerization and Deployment Artifacts:
- The Dockerfile.webserver, docker-compose.yaml, and “private-containerfiles” show how components are containerized and orchestrated.
- The .buildkite directory contains CI/CD scripts and pipelines that handle building and deployment, ensuring continuous integration and testing.

##### Relationships and Interactions
- RL Node & RL Training:
- The hivemind_exp module acts as the “brain” of each peer node where the reinforcement learning algorithms run, communicate over a DHT/gossip network, and possibly interact with smart contracts on-chain.
- Identity Management:
- Nodes use the modal-login service to authenticate and register with the blockchain network. This layer links a “swarm.pem” file with an EOA (via Alchemy) and manages the on-chain identity.
- UI and API Layer:
- The web components (both API and UI) communicate with the RL nodes to reflect live training status, peer performance, and identity registration information.
- Deployment:
- Containerization (using Docker, docker-compose, and private container files) ties together various isolated services and allows for scaling or isolated testing environments.
- CI/CD:
- Buildkite pipeline scripts ensure automated testing and deployment of containerized components.

##### Architectural Patterns and Design Principles
- Modular Separation:
- The separation into hivemind_exp (core RL training and data management), modal-login (authentication), and web (UI/API) clearly delineates responsibilities.
- Containerization & Microservices:
- The use of Docker and dedicated container files for swarm nodes and web servers implies a microservices approach.
- Peer-to-Peer Communication and Decentralization:
- Use of DHT/gossip protocols in the RL components reinforces a decentralized network design.
- Extensibility:
- The file structure is organized to facilitate adding new training scenarios (via additional configs, contracts, or new RL modules) or changing identity providers.

## Diagram (Mermaid)

```mermaid
graph TD
    CICD["CI/CD Pipeline (.buildkite)"]:::infrastructure
    CICD --> DEP["Container/Deployment"]:::infrastructure

    subgraph "Deployment Artifacts"
        DFILE["Dockerfile.webserver"]:::infrastructure
        DCOMP["docker-compose.yaml"]:::infrastructure
        SWARM["Swarm Node Container (private-containerfiles/swarm-node)"]:::infrastructure
    end

    DEP --> RL["RL Training Node (hivemind_exp)"]:::backend
    DEP --> AUTH["Authentication Service (modal-login)"]:::authentication
    DEP --> UI["Swarm UI & API (web)"]:::frontend

    RL -->|"updates"| UI
    RL -->|"sends_identity"| AUTH
    UI -->|"requests_training"| RL
    UI -->|"displays_auth"| AUTH
    AUTH -->|"verifies_identity"| BLOCK["On-chain Identity (Alchemy)"]:::external

    click CICD "https://github.com/gensyn-ai/rl-swarm/blob/main/.buildkite"
    click RL "https://github.com/gensyn-ai/rl-swarm/tree/main/hivemind_exp"
    click AUTH "https://github.com/gensyn-ai/rl-swarm/tree/main/modal-login"
    click UI "https://github.com/gensyn-ai/rl-swarm/tree/main/web"
    click DFILE "https://github.com/gensyn-ai/rl-swarm/blob/main/Dockerfile.webserver"
    click DCOMP "https://github.com/gensyn-ai/rl-swarm/blob/main/docker-compose.yaml"
    click SWARM "https://github.com/gensyn-ai/rl-swarm/tree/main/private-containerfiles/swarm-node"

    classDef backend fill:#e0f7fa,stroke:#006064,stroke-width:2px;
    classDef authentication fill:#ffebee,stroke:#c62828,stroke-width:2px;
    classDef frontend fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px;
    classDef infrastructure fill:#fff3e0,stroke:#ef6c00,stroke-width:2px;
    classDef external fill:#ede7f6,stroke:#5e35b1,stroke-width:2px;
```
