# macrocosm-os/folding architecture

> Architecture diagram of the GitHub repository macrocosm-os/folding, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/macrocosm-os/folding
- Repository: https://github.com/macrocosm-os/folding

## Overview

##### Project Overview
- This codebase represents a decentralized, scientific computing subnet within the larger Bittensor ecosystem.
- It is built primarily for protein folding simulations, where validators create protein simulation challenges and miners execute the simulations using OpenMM.
- The objective is to support academic research by simulating protein structures and evaluating them using a physics-based energy minimization (reward) process.
- Key roles include Validators and Miners. Validation and mining logic are cleanly separated and orchestrated via a Bittensor network.

##### Analysis of the File Structure
- The source code is organized into several top-level directories:
- "folding": Contains central modules—base logic (evaluation, simulation, reward calculations, registration of miners, validators, etc.), utilities, and experiment scripts.
- "folding_api": Implements helper routines and endpoints for authentication, chain interactions, and configuration. This supports communicating with the Bittensor network.
- "neurons": Contains scripts for miners and validators. These will be the executable agents responsible for simulation or validation functionalities.
- "scripts": Provides helper shell and python scripts to run tasks such as starting miners/validators, job queries, and other utilities.
- "tests": Includes unit tests and fixtures ensuring integrity of core components.
- Additional directories like "assets", "documentation", and "contrib" help with images, research documents, and contribution guidelines respectively.
- Configurations such as ".env.example", "pyproject.toml", and "install.sh" indicate environment setup using Conda, Poetry, and shell scripts.

##### Key Components & Their Interactions
a. Core Simulation Engine and Business Logic:
- The "folding/base" modules outline core simulation routines (e.g., simulation.py), energy/reward computation (reward.py, linear_reward.py, md_rewards.py), and evaluation (evaluation.py) using OpenMM.
- Modules for miners and validators are also in "folding/miners" and "folding/validators".
- “folding/registries” allow dynamic registration of evaluation and miner implementations—an architectural pattern to enable extensibility.

b. API & External Interaction Layer:
- The "folding_api" directory encapsulates RESTful or RPC style endpoints to handle authentication, queries, protein data interactions, and external configuration.
- This layer serves as a bridge between on-chain registration processes (using btcli commands as detailed in the README) and the local computing tasks.

c. Neuron (Validator and Miner) Components:
- “neurons/validator.py” and “neurons/miner.py” are entry points for the two main roles.
- Validators are responsible for enqueuing protein simulation tasks, dispatching these to multiple miners and logging results.
- Miners execute parallel simulations, answer protein folding tasks, and may interact with a local instance of a job pool (GJP) via scripts like “query_rqlite.py”.

d. External Dependencies and Integrations:
- OpenMM: Used as the molecular dynamics (MD) simulation engine.
- Wandb: For logging and experimentation tracking (especially for validator operations).
- S3-compatible storage: Seen from the S3 configuration references for checkpoint and log storage.
- RQLite: Used for local job queue synchronization in miners, as detailed with pm2/node configuration scripts.
- Bittensor network: The blockchain type network that registers the subnet and facilitates the economic reward system.

##### Architectural Patterns and Design Principles
- Modular Design: Separation of business logic ("folding"), API interfaces ("folding_api"), and agent executables ("neurons") promotes decoupled development and testing.
- Registry Pattern: The registries’ folder and dynamic registrations help in plugging in new evaluators or miners without altering core code.
- Distributed Processing: The design leverages parallel/minimum worker set-ups for miners (using multiple threads or processes) to handle protein simulation in parallel.
- Config-driven Behavior: Extensive use of configuration files and environment variables (for OpenMM, S3, network details) ensures flexible deployment.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Neuron & Entry Points"
        V("Validator Node"):::compute
        M("Miner Node"):::compute
        NE("Neuron Entry Points"):::compute
    end

    subgraph "Folding Core & Modules"
        FC("Folding Core (Simulation Engine)"):::compute
        FMM("Folding Miner Modules"):::compute
        FVM("Folding Validator Modules"):::compute
        DR("Dynamic Registration (Registries)"):::compute
        EXP("Experiments & Analysis"):::compute
    end

    subgraph "API Integration"
        API("API Gateway"):::api
    end

    subgraph "External Systems"
        O("OpenMM"):::external
        W("Wandb"):::external
        S("S3 Storage"):::external
        R("RQLite"):::external
        B("Bittensor Network"):::external
    end

    subgraph "Deployment & Orchestration"
        DS("Deployment Scripts"):::infra
    end

    %% Neuron Layer interactions
    NE -->|"starts"| V
    NE -->|"starts"| M
    V -->|"assigns_task"| M
    M -->|"returns_result"| V

    %% Neurons interacting with API
    V -->|"chain_calls"| API
    M -->|"chain_calls"| API

    %% Neurons and Core Simulation
    V -->|"simulation_task"| FC
    M -->|"simulation_task"| FC

    %% Core delegations and module usage
    FC -->|"dynamic_registration"| DR
    M -->|"miner_execution"| FMM
    V -->|"validation_execution"| FVM
    FC -->|"analysis"| EXP

    %% Simulation engine interacts with external simulation engine and logging
    FC -->|"runs_on"| O
    FC -->|"logs_to"| W

    %% API Gateway external interactions
    API -->|"registers_to"| B
    API -->|"stores_data_in"| S
    API -->|"syncs_jobs_with"| R

    %% Deployment orchestration
    DS -->|"orchestrates"| V
    DS -->|"orchestrates"| M
    DS -->|"orchestrates"| API

    %% Click Events for Component Mapping
    click FC "https://github.com/macrocosm-os/folding/tree/main/folding/base"
    click FMM "https://github.com/macrocosm-os/folding/tree/main/folding/miners"
    click FVM "https://github.com/macrocosm-os/folding/tree/main/folding/validators"
    click DR "https://github.com/macrocosm-os/folding/tree/main/folding/registries"
    click API "https://github.com/macrocosm-os/folding/tree/main/folding_api"
    click NE "https://github.com/macrocosm-os/folding/tree/main/neurons"
    click EXP "https://github.com/macrocosm-os/folding/tree/main/folding/experiments"
    click DS "https://github.com/macrocosm-os/folding/tree/main/scripts"

    %% Styles
    classDef compute fill:#add8e6,stroke:#000,stroke-width:2px;
    classDef api fill:#90ee90,stroke:#000,stroke-width:2px;
    classDef external fill:#ffe6e6,stroke:#000,stroke-width:2px;
    classDef infra fill:#ffcc99,stroke:#000,stroke-width:2px;
```
