How condenses/subnet-validating works
Main Components
• Validating Service:
• Represented by the “condenses_validating” directory that contains modules like main.py, log_processor.py, redis_manager.py, and others. This is the heart of the solution where the orchestration and validation logic resides.
• External Microservices:
• Synthesizing Service – exposes an endpoint (using uvicorn) that processes or synthesizes text based on a public dataset.
• Node Managing Service – manages credit and rate limits across “validator -> miner” pathways.
• Scoring Service – receives original and compressed messages for scoring based on the weighted formula described.
• Sidecar Bittensor Service – a dedicated service for metagraph synchronization and chain functions.
• Supporting Components:
• Log Viewer – a console-based tool to view and distinguish logs from various validation stages.
• Redis – hinted by the “redis_manager.py” and the installation script “scripts/install_redis.sh”, which likely acts as a cache or message broker.
• Docker – indicated by the Dockerfile, suggesting containerization and deployment orchestration for various services.
• Scripts – for setup and environment updates that hint at dynamic configuration (install_redis.sh, start.sh, update_env.py, vllm.sh).
Relationships and Interactions
• Validating Service <--> Synthesizing Service (indicating it sends user messages or requests for synthesis).
• Validating Service <--> Node Managing Service (for UID orchestration and rate limiting).
• Validating Service <--> Scoring Service (to send messages for scoring and receive evaluation results).
• Validating Service <--> Sidecar Bittensor (for blockchain/network sync operations related to weights and metagraph updates).
• Connections with Redis as an underlying datastore or message broker used by the Validating Service.
• The flow should also illustrate that Log Viewer monitors the outputs of the Validating Service for easier debugging and operational insight.
Architectural Patterns and Technologies
• Microservices/Service Orchestration:
• Each service is independently started and managed (using PM2 for process management and uvicorn hosts for REST endpoints).
• Containerization:
• Environment-based Configuration:
• Update-env scripts, configuration files (condenses_validating/config.py), and Redis management imply that runtime configuration is dynamic.
• Use of Uvicorn suggests asynchronous HTTP endpoints in Python, while PM2 indicates production process management.
Detailed Element Inclusion
• A label for each microservice along with its respective endpoint port as noted in the README (9100 for synthesizing, 9101 for node managing, 9102 for scoring, 9103 for sidecar-bittensor).
• Interaction between the validating process and the log viewer as a monitoring/debugging component.
• Essential environment variables (like HF_TOKEN, MODEL_NAME, TAOSTATS_API_KEY) near the corresponding service blocks to show external API integrations.
Additional Diagrammatic Practices
• That all components are labeled clearly with their file names or logical names where applicable.