# bitmind-ai/bitmind-subnet architecture

> Architecture diagram of the GitHub repository bitmind-ai/bitmind-subnet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bitmind-ai/bitmind-subnet
- Repository: https://github.com/bitmind-ai/bitmind-subnet

## Overview

##### Project Type and Purpose
- The project is a decentralized system built on the Bittensor network that performs deepfake detection. It involves miners (that run binary classifiers) and validators (that challenge these miners using real and synthetic media). There is also functionality for synthetic data generation and data fetching.
- The README clearly outlines three main aspects: challenge generation/scoring, data generation/downloads, and organic traffic routing.

##### File Structure
- Notice several top-level directories such as “base_miner”, “bitmind”, “neurons”, and “tests”. Each of these directories contains components that belong to different aspects of the detection system.
- The “base_miner” directory holds various configurations, detectors, loss functions, metrics, networks, and training scripts. These are likely used to implement the miner functionality.
- The “bitmind” directory covers a broader range, including base components (miner, validator, neuron) along with utility modules for image and video processing, synthetic data generation, and configuration.
- The “neurons” directory appears to provide alternative implementations or extra functionality for miners and validators (or “neurons” in Bittensor parlance).
- There is also a clear separation in configuration scripts, dataset handling, and testing utilities which help indicate an organized modular architecture following typical modular design principles.

##### Core Components
a. Miner Component:
- Represented by modules in “base_miner” (e.g., detectors, networks, loss, trainer) and “neurons/miner.py”. These are responsible for running AI models that output a float between 0 and 1 to classify content as real or synthetic.
- That miners are parameterized using various configurations (e.g., several YAML config files for different approaches like “tall.yaml”, “ucf.yaml”, “xception.yaml”).

b. Validator Component:
- Shown in “bitmind/validator” and its subdirectories with modules for configuration, caching, challenge scoring, and performance tracking.
- That validators challenge miners by feeding them randomly augmented media from both real and synthetic sources.
- How validators interact with external systems, such as the cache and challenge scripts (e.g., “run_data_generator.py”, “verify_models.py”).

c. Data Generation and Fetching:
- The synthetic data generator elements in “bitmind/synthetic_data_generation” that coordinate a VLM and LLM to generate prompts and then produce media.
- Real data fetching mechanisms, as evident from “base_miner/datasets” (for partial dataset downloads) and “bitmind/dataset_processing/…/dlib_tools”.

d. External / Infrastructure Components:
- The organic traffic/integration side: the README mentions an external API server, load balancer, and vector database, which provide application requests and caching.

e. Additional Supporting Modules:
- “gating_mechanisms” provide additional logic to possibly filter or route data (with face detection modules).
- CI/CD pipelines (located in “.github/workflows”) indicate continuous integration for testing and deployment.
- Utility libraries within “bitmind/utils”, “neurons”, and related directories offer common functionalities across components.

##### Defining Relationships and Data Flow
- Data flow for challenge creation—from the validator’s challenge generator sampling real and synthetic media and sending it to miners.
- Response flow from miners back to validators with classification results and predicted scores.
- Reporting flow: detailed challenge results (including media and metadata) traveling to external side systems like W&B.

##### Architectural Patterns and Technologies
- Modular design: Each function (mining, validating, data fetching, synthetic data generation) is encapsulated in its own module or set of modules.
- Decentralized and distributed nature: Emphasize how the subnet components communicate across the Bittensor network.
- Configuration-driven: Multiple YAML configurations and dedicated scripts (e.g., auto-update scripts, training scripts) indicate a robust pipeline mimicking microservice-like separations.
- Frameworks & Libraries: Although not explicitly listed, note the use of popular libraries for image/video processing, neural network training, and logging (W&B).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Miners"
        M_DFB["Base Miner (DFB)"]:::miner
        M_NPR["Base Miner (NPR)"]:::miner
        M_Neuron["Neuron Miner"]:::miner
    end

    subgraph "Validators"
        V_Primary["Primary Validator"]:::validator
        subgraph "Alternative Validators"
            V_Alt1["Validator (Alternative 1)"]:::validator
            V_Alt2["Validator (Alternative 2)"]:::validator
        end
    end

    subgraph "Data Pipelines"
        D_Synthetic["Synthetic Data Generator"]:::data
        subgraph "Real Data Fetching"
            D_Real1["Real Data Fetcher (Datasets)"]:::data
            D_Real2["Real Data Fetcher (Dlib Tools)"]:::data
        end
    end

    subgraph "External/Infrastructure"
        External_API["API Server & Load Balancer"]:::external
        External_Logging["External Logging (W&B)"]:::external
        CI_CD["CI/CD Pipeline"]:::external
    end

    subgraph "Supporting Modules"
        Support_Gating["Gating Mechanisms"]:::support
        subgraph "Shared Utilities"
            Support_Utils1["Shared Utilities (bitmind/base)"]:::support
            Support_Utils2["Shared Utilities (bitmind/utils)"]:::support
        end
    end

    %% Relationships
    External_API -->|"organic_traffic"| V_Primary
    External_API -->|"organic_traffic"| V_Alt1
    External_API -->|"organic_traffic"| V_Alt2

    V_Primary -->|"challenge"| M_DFB
    V_Primary -->|"challenge"| M_NPR
    V_Primary -->|"challenge"| M_Neuron
    M_DFB -->|"response"| V_Primary
    M_NPR -->|"response"| V_Primary
    M_Neuron -->|"response"| V_Primary

    D_Synthetic -->|"data_feed"| V_Primary
    D_Synthetic -->|"data_feed"| V_Alt1
    D_Synthetic -->|"data_feed"| V_Alt2
    D_Real1 -->|"data_feed"| V_Primary
    D_Real2 -->|"data_feed"| V_Primary
    D_Real1 -->|"data_feed"| V_Alt1
    D_Real2 -->|"data_feed"| V_Alt2

    V_Primary -->|"report"| External_Logging
    V_Alt1 -->|"report"| External_Logging
    V_Alt2 -->|"report"| External_Logging

    CI_CD -->|"deploy_update"| V_Primary
    CI_CD -->|"deploy_update"| M_DFB
    CI_CD -->|"deploy_update"| M_NPR
    CI_CD -->|"deploy_update"| M_Neuron

    Support_Gating ---|"supports"| M_DFB
    Support_Gating ---|"supports"| M_NPR
    Support_Gating ---|"supports"| M_Neuron
    Support_Gating ---|"supports"| V_Primary
    Support_Gating ---|"supports"| V_Alt1
    Support_Gating ---|"supports"| V_Alt2

    Support_Utils1 ---|"utilities"| M_DFB
    Support_Utils1 ---|"utilities"| V_Primary
    Support_Utils2 ---|"utilities"| M_Neuron
    Support_Utils2 ---|"utilities"| V_Alt2

    %% Click Events
    click M_DFB "https://github.com/bitmind-ai/bitmind-subnet/tree/main/base_miner/DFB/"
    click M_NPR "https://github.com/bitmind-ai/bitmind-subnet/tree/main/base_miner/NPR/"
    click M_Neuron "https://github.com/bitmind-ai/bitmind-subnet/blob/main/neurons/miner.py"

    click V_Primary "https://github.com/bitmind-ai/bitmind-subnet/tree/main/bitmind/validator/"
    click V_Alt1 "https://github.com/bitmind-ai/bitmind-subnet/blob/main/neurons/validator.py"
    click V_Alt2 "https://github.com/bitmind-ai/bitmind-subnet/blob/main/neurons/validator_proxy.py"

    click D_Synthetic "https://github.com/bitmind-ai/bitmind-subnet/tree/main/bitmind/synthetic_data_generation/"
    click D_Real1 "https://github.com/bitmind-ai/bitmind-subnet/tree/main/base_miner/datasets/"
    click D_Real2 "https://github.com/bitmind-ai/bitmind-subnet/tree/main/bitmind/dataset_processing/dlib_tools/"

    click CI_CD "https://github.com/bitmind-ai/bitmind-subnet/blob/main/.github/workflows/ci.yml"
    click External_Logging "https://github.com/bitmind-ai/bitmind-subnet/blob/main/bitmind/utils/logging.py"

    click Support_Gating "https://github.com/bitmind-ai/bitmind-subnet/tree/main/base_miner/gating_mechanisms/"
    click Support_Utils1 "https://github.com/bitmind-ai/bitmind-subnet/tree/main/bitmind/base/utils/"
    click Support_Utils2 "https://github.com/bitmind-ai/bitmind-subnet/tree/main/bitmind/utils/"

    %% Styles
    classDef miner fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef validator fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef data fill:#87CEFA,stroke:#000,stroke-width:2px;
    classDef external fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef support fill:#D3D3D3,stroke:#000,stroke-width:2px;
```
