# nano-bittensor/inference-subnet-template architecture

> Architecture diagram of the GitHub repository nano-bittensor/inference-subnet-template, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nano-bittensor/inference-subnet-template
- Repository: https://github.com/nano-bittensor/inference-subnet-template

## Overview

##### Project Overview & Main Purpose
- The project is an inference subnet template designed for the Bittensor ecosystem. It is built to support scalable, microservice-based interactions between validators and miners. The system enables two types of request flows: synthetic (validator generated) and organic (user/app forwarded) requests. The template is aimed at high-scale inference use cases (such as LLM inference, image generation, OCR, etc.) with robust rate limiting, hybrid scoring, and direct blockchain/substrate integration.

##### File Structure Insights
- Top-Level Directories/Files:
- Files like .gitignore, pyproject.toml, .python-version, and README.md indicate standard project setup.
- The core logic is under the "inference_subnet" directory.
- Main Subdirectories:
- neurons: Contains components responsible for miner and validator functionalities.
- "miner" holds its own FastAPI app (app.py) and utility script for server address registration.
- "validator" includes the main orchestrator logic (main.py) for validating operations.
- services: Implements supporting microservices that the validator uses.
- "managing" service: Contains the app, rate limit manager, score manager, and schemas. It controls operational aspects like rate-limiting.
- "scoring" service: Handles miner evaluation with schemas.
- "sidecar_subtensor" service: Offers a direct sidecar interface (with its own README and app).
- Additional utility files such as protocol.py, settings.py, and verification.py likely form the communication and configuration layers.

##### Key Architectural Components (to include
a. Neurons Layer:
- Miner Component:
- Represented by "inference_subnet/neurons/miner" housing the main miner app and its blockchain registration script.
- This acts as an endpoint that processes computation requests.
- Validator Component:
- Represented by "inference_subnet/neurons/validator" with its main execution script for orchestrating validation.
- This component channels both synthetic and organic requests.

b. Microservices Layer (Validator Services):
- Managing Service:
- Located in "inference_subnet/services/managing" – responsible for rate limiting and operational management.
- Scoring Service:
- Located in "inference_subnet/services/scoring" – responsible for the hybrid miner scoring mechanism.
- Sidecar Subtensor Service:
- Located in "inference_subnet/services/sidecar_subtensor" – acts as a direct substrate interface.
- (If present, a Synthesizing Service might be included based on the README command for "synthesizing_service", even if its folder isn’t in the file tree. Consider a placeholder if applicable.)

c. External/Supporting Components:
- Blockchain/Network Integration:
- The miner registers its server address with the blockchain network. Illustrate an external blockchain or substrate network component.
- Data Flow and Protocol:
- “protocol.py” and “verification.py” serve as the communication backbone. They should be represented as the protocol layer or middleware ensuring message and data integrity between components.

d. Infrastructure & Deployment:
- Uvicorn is used to serve the FastAPI applications for each component.
- Environment configuration is handled using settings.py and environment variables (as shown in the README).

##### Relationships and Interactions
- From User/Application to Validator: Represent the organic request flow.
- From Validator to Miner: For both organic and synthetic request challenges.
- Within the Validator, demonstrate interactions between the neuron component (executed by main.py) and supporting microservices (managing, scoring, sidecar).
- The Miner communicates back results and registration information is sent to the blockchain network.
- Dependency directions:
- The Validator service calls managing and scoring services as sub-components.
- Miner connects to external blockchain (for registration) and processes computational challenges.

## Diagram (Mermaid)

```mermaid
graph TD

  %% User Layer
  subgraph "User Layer"
    U["User/Application"]:::user
  end

  %% Neuron Layer
  subgraph "Neuron Layer"
    VAL["Validator Orchestrator"]:::neuron
    MIN["Miner Service (FastAPI Application)"]:::neuron
  end

  %% Microservices Layer
  subgraph "Microservices Layer"
    MAN["Managing Service (Rate Limiter & Operation Manager)"]:::micro
    SCOR["Scoring Service (Hybrid Evaluator)"]:::micro
    SID["Sidecar Subtensor Service (Substrate Interface)"]:::micro
  end

  %% Core Infrastructure & Communication
  subgraph "Core Infrastructure & Communication"
    PROT["Protocol (Message & Communication)"]:::core
    VER["Verification (Integrity Checks)"]:::core
    CONF["Environment & Config Management"]:::core
  end

  %% External Integration
  subgraph "External Integration"
    BC["Blockchain/Substrate Network"]:::external
  end

  %% Connections
  U -->|"organic_request"| VAL
  
  VAL -->|"rate_limit_call"| MAN
  VAL -->|"scoring_request"| SCOR
  VAL -->|"sidecar_request"| SID
  VAL -->|"challenge_request"| MIN
  
  MIN -->|"registration"| BC
  MIN -->|"result_flow"| VAL
  
  VAL -.->|"secure_message"| PROT
  MIN -.->|"secure_message"| PROT
  
  PROT -->|"integrity_check"| VER
  CONF -.-> PROT
  CONF -.-> VER

  %% Styles
  classDef user fill:#FFF9C4,stroke:#FBC02D,stroke-width:2px;
  classDef neuron fill:#E1F5FE,stroke:#0277BD,stroke-width:2px;
  classDef micro fill:#F1F8E9,stroke:#33691E,stroke-width:2px;
  classDef external fill:#FFE0B2,stroke:#E65100,stroke-width:2px;
  classDef core fill:#F3E5F5,stroke:#6A1B9A,stroke-width:2px;

  %% Click Events
  click MIN "https://github.com/nano-bittensor/inference-subnet-template/tree/main/inference_subnet/neurons/miner"
  click VAL "https://github.com/nano-bittensor/inference-subnet-template/blob/main/inference_subnet/neurons/validator/main.py"
  click MAN "https://github.com/nano-bittensor/inference-subnet-template/tree/main/inference_subnet/services/managing"
  click SCOR "https://github.com/nano-bittensor/inference-subnet-template/tree/main/inference_subnet/services/scoring"
  click SID "https://github.com/nano-bittensor/inference-subnet-template/tree/main/inference_subnet/services/sidecar_subtensor"
  click PROT "https://github.com/nano-bittensor/inference-subnet-template/blob/main/inference_subnet/protocol.py"
  click VER "https://github.com/nano-bittensor/inference-subnet-template/blob/main/inference_subnet/verification.py"
  click CONF "https://github.com/nano-bittensor/inference-subnet-template/blob/main/inference_subnet/settings.py"
```
