How nano-bittensor/inference-subnet-template works
Project Overview & Main Purpose
• The project is an inference subnet template designed for the Bittensor ecosystem. It is built to support scalable, microservice-based interactions between validators and miners. The system enables two types of request flows: synthetic (validator generated) and organic (user/app forwarded) requests. The template is aimed at high-scale inference use cases (such as LLM inference, image generation, OCR, etc.) with robust rate limiting, hybrid scoring, and direct blockchain/substrate integration.
File Structure Insights
• Top-Level Directories/Files:
• Files like .gitignore, pyproject.toml, .python-version, and README.md indicate standard project setup.
• The core logic is under the "inference_subnet" directory.
• Main Subdirectories:
• neurons: Contains components responsible for miner and validator functionalities.
• "miner" holds its own FastAPI app (app.py) and utility script for server address registration.
• "validator" includes the main orchestrator logic (main.py) for validating operations.
• services: Implements supporting microservices that the validator uses.
• "managing" service: Contains the app, rate limit manager, score manager, and schemas. It controls operational aspects like rate-limiting.
• "scoring" service: Handles miner evaluation with schemas.
• "sidecar_subtensor" service: Offers a direct sidecar interface (with its own README and app).
• Additional utility files such as protocol.py, settings.py, and verification.py likely form the communication and configuration layers.
Key Architectural Components (to include
a. Neurons Layer:
• Miner Component:
• Represented by "inference_subnet/neurons/miner" housing the main miner app and its blockchain registration script.
• This acts as an endpoint that processes computation requests.
• Validator Component:
• Represented by "inference_subnet/neurons/validator" with its main execution script for orchestrating validation.
• This component channels both synthetic and organic requests.
b. Microservices Layer (Validator Services):
• Managing Service:
• Located in "inference_subnet/services/managing" – responsible for rate limiting and operational management.
• Scoring Service:
• Located in "inference_subnet/services/scoring" – responsible for the hybrid miner scoring mechanism.
• Sidecar Subtensor Service:
• Located in "inference_subnet/services/sidecar_subtensor" – acts as a direct substrate interface.
• (If present, a Synthesizing Service might be included based on the README command for "synthesizing_service", even if its folder isn’t in the file tree. Consider a placeholder if applicable.)
c. External/Supporting Components:
• Blockchain/Network Integration:
• The miner registers its server address with the blockchain network. Illustrate an external blockchain or substrate network component.
• Data Flow and Protocol:
• “protocol.py” and “verification.py” serve as the communication backbone. They should be represented as the protocol layer or middleware ensuring message and data integrity between components.
d. Infrastructure & Deployment:
• Uvicorn is used to serve the FastAPI applications for each component.
• Environment configuration is handled using settings.py and environment variables (as shown in the README).
Relationships and Interactions
• From User/Application to Validator: Represent the organic request flow.
• From Validator to Miner: For both organic and synthetic request challenges.
• Within the Validator, demonstrate interactions between the neuron component (executed by main.py) and supporting microservices (managing, scoring, sidecar).
• The Miner communicates back results and registration information is sent to the blockchain network.
• Dependency directions:
• The Validator service calls managing and scoring services as sub-components.
• Miner connects to external blockchain (for registration) and processes computational challenges.