# datura-ai/lium-io architecture

> Architecture diagram of the GitHub repository datura-ai/lium-io, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/datura-ai/lium-io
- Repository: https://github.com/datura-ai/lium-io

## Overview

##### Project Type and Purpose
- This is a decentralized, peer-to-peer GPU rental marketplace built on Bittensor Subnet 51.
- It comprises three microservices – Executor, Miner, Validator – plus a Python SDK/CLI library (Datura) and a frontend UI (lium.io).

##### File Structure
- Top‐level services under `neurons/`:
- `neurons/executor`: handles task execution on GPU pods
- `neurons/miners`: manages miner registration, pricing, and portal API interactions
- `neurons/validators`: performs hardware verification and scoring pipelines
- Shared library under `datura/`: Python package providing request/consumer abstractions for both CLI and service clients.
- Infrastructure: Dockerfiles, docker-compose configs, Alembic migrations, CI/CD workflows (`.github/workflows`), scripts for installation and deployment.
- Each service uses a relational database (via SQLAlchemy/Alembic), REST or RPC routes, and middleware components.

##### README Insights
- High-level architecture calls out Miners, Validators, Renters (frontend & CLI), and the Bittensor blockchain as external component.
- The frontend (`lium.io`) and CLI (`lium-cli` / Datura) are consumer clients of the services.

##### Main Components
a. Frontend & CLI Clients
- lium.io Web App (not in this repo)
- Datura-powered lium-cli
b. Microservices
- Miner Service (neurons/miners)
- Validator Service (neurons/validators)
- Executor Service (neurons/executor)
c. Datastore Layer
- PostgreSQL (or chosen RDBMS) per service (each has its own schema/migrations)
d. External Systems
- Bittensor Subnet blockchain (for token transfers)
- SSH endpoints on GPU machines
e. Messaging/Protocol
- REST / RPC APIs between clients and microservices
- Custom protocol modules in `protocol/` folders

##### Relationships and Data Flows
- Renters → Web UI / CLI → Miner Service: browse & request GPU pods
- Miner Service ↔ Executor Service: assign tasks to pods, track logs
- Validator Service → Miner Service: pull miner details; Validator connects via SSH to GPU hosts
- Executor Service → Validator Service: report task results or metrics
- All services → blockchain API: validators pay miners using $TAO tokens
- Microservices ↔ their local DBs for state, logs, pricing, and tasks

##### Architectural Patterns & Principles
- Microservices: each domain (miner, validator, executor) runs in its own container, own database
- CQRS‐style separation: read models (DAOs) vs write operations (services/payloads)
- Middleware pipelines in Validator for modular “checks” (GPU count, fingerprint, etc.)
- Containerized deployments using Docker & docker-compose for local/dev/prod
- CI/CD per service via GitHub Actions

##### Technologies & Frameworks
- Python 3 / FastAPI or Flask (implied by `routes/` folders)
- SQLAlchemy + Alembic for ORM & migrations
- Docker + docker-compose for container orchestration
- PDM / pyproject.toml for dependency management
- Datura Python package for client abstractions

## Diagram (Mermaid)

```mermaid
graph TB
    classDef service fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef db fill:#90EE90,stroke:#000,stroke-width:1px
    classDef external fill:#FFD700,stroke:#000,stroke-width:1px
    classDef client fill:#D3D3D3,stroke:#000,stroke-width:1px
    classDef infra fill:#E6E6FA,stroke:#000,stroke-width:1px

    subgraph "Clients"
        UI["Web UI (lium.io)"]:::client
        CLI["CLI (Datura)"]:::client
    end

    subgraph "Compute Subnet"
        subgraph "Executor Service" 
            ES["Executor Service"]:::service
        end
        subgraph "Miner Service"
            MS["Miner Service"]:::service
        end
        subgraph "Validator Service"
            VS["Validator Service"]:::service
        end
    end

    subgraph "Datastores"
        DBE["Executor DB"]:::db
        DBM["Miner DB"]:::db
        DBV["Validator DB"]:::db
    end

    subgraph "External Systems"
        BC["Bittensor Blockchain"]:::external
        GPU["GPU Hosts (SSH)"]:::external
    end

    subgraph "Infrastructure & CI/CD"
        CI["CI/CD Workflows"]:::infra
        SC["Scripts & Docker-Compose"]:::infra
    end

    UI -->|"HTTP REST"| MS
    CLI -->|"HTTP REST"| MS

    MS -->|"RPC / Task Assign"| ES
    ES -->|"Report Metrics"| VS
    VS -->|"Validation Requests (SSH)"| GPU
    VS -->|"Fetch Miner Info"| MS

    ES -->|"SQLAlchemy ORM"| DBE
    MS -->|"SQLAlchemy ORM"| DBM
    VS -->|"SQLAlchemy ORM"| DBV

    ES -->|"$TAO Payments"| BC
    MS -->|"$TAO Payments"| BC
    VS -->|"$TAO Payments"| BC

    CI -.-> ES
    CI -.-> MS
    CI -.-> VS
    SC -.-> ES
    SC -.-> MS
    SC -.-> VS

    click CLI "https://github.com/datura-ai/lium-io/tree/main/datura/datura"
    click ES "https://github.com/datura-ai/lium-io/tree/main/neurons/executor/src"
    click MS "https://github.com/datura-ai/lium-io/tree/main/neurons/miners/src"
    click VS "https://github.com/datura-ai/lium-io/tree/main/neurons/validators/src"
    click CI "https://github.com/datura-ai/lium-io/tree/main/.github/workflows"
    click SC "https://github.com/datura-ai/lium-io/tree/main/scripts"

    class UI,CLI client
    class ES,MS,VS service
    class DBE,DBM,DBV db
    class BC,GPU external
    class CI,SC infra
```
