How score-technologies/score-vision works
Project Type & Purpose
• Project Type: A decentralized computer vision framework built upon Bittensor with a dual-role architecture – “miners” and “validators.”
• Purpose: To perform real-time, large-scale video analysis for Game State Recognition in football (and potentially beyond) by leveraging innovative lightweight validation techniques that reduce cost and computational overhead.
File Structure
• There are two main directories: "miner" and "validator". Each role has its own README and setup instructions.
• The "miner" directory contains key modules:
– Core configuration and models (under miner/core).
– Endpoints that accept video streams and process data (under miner/endpoints).
– Specific sports modules under miner/sports (e.g., annotators and configs for soccer).
– Utility functions and scripts for model handling, video downloading, and processing.
• The "validator" directory mirrors this separation with:
– Modules to configure challenges and validate miner outputs (under validator/challenge).
– Database interaction and schema definitions (under validator/db).
– Endpoints (like health checks) and evaluation loops to calculate quality scores.
– Utility scripts to support asynchronous and API operations.
• Supporting files (e.g., setup.py, environment examples, requirements files) facilitate configuration and deployment.
Insights from the README
• Three core roles are described: miners (video input & processing), validators (output verification via lightweight validation techniques), and a network (managed by Subnet Owners) that oversees operations and incentives.
• Emphasis on two-phase validation: a fast, numerical count verification followed by a more comprehensive VLM-based quality assessment.
• The system is designed for real-time distributed video processing where dynamic interaction and performance-based rewards are key.
a. Main Components:
• Miners Module:
• Video Input / Streaming Interface: Receiving raw video streams.
• Frame Processing: Object detection, tracking, and game state extraction (likely orchestrated via modules in miner/core and miner/sports).
• Endpoints: Dedicated processing endpoints (e.g., soccer specific logic under miner/endpoints and sports modules).
• Supporting Scripts & Utilities: For tasks like model downloads, testing pipelines, and video handling.
• Validators Module:
• Output Validation: Lightweight validation with two phases (object count verification and Intelligent Quality Assessment using VLM).
• Evaluation & Scoring: Modules under validator/evaluation to calculate GS-HOTA and quality scores.
• Database & Challenge Handling: Components that manage persistent data, perform operations, and handle different challenge types.
• Endpoints & Utility Components: For health checks and external API interactions.
• Network/Management (External/Operational Layer):
• Although not directly shown as a dedicated module in the file tree, reference to Subnet Owner role implies a management/control plane that interfaces with both miners and validators. This can be represented as a separate block responsible for system health monitoring, reward distribution, and incentive adjustments.
b. Relationships & Interactions:
• Data flows from the Miners to Validators:
– Video streams enter the miners.
– Processed outputs (detected objects, game state data) are emitted through endpoints.
– Validators receive this output and process it further for quality and consistency checks.
• Feedback Loop:
– Results from validators (quality scores, evaluation metrics) are relayed back to the network management (Subnet Owner) which dynamically adjusts incentives and operational parameters.
• Utility and shared components:
– Common configurations, models, and utility functions are used across both roles (e.g., video processing, model management).
– Scripts support initial setup, testing, and running pipelines, fostering easy deployment and integration.
c. Architectural Patterns/Design Principles:
• Separation of Concerns: Clear distinction between miner and validator roles.
• Modularity: Encapsulated components (e.g., endpoints, evaluation, database operations) facilitate independent development, testing, and deployment.
• Distributed/Decentralized Processing: The design supports multiple miners handling concurrent streams and validators processing outputs in a scalable manner.
• Lightweight Validation: Innovative two-phase validation is central to reducing computational cost while ensuring accuracy.
d. Relevant Technologies & Frameworks:
• Python as the primary programming language.
• Use of configuration files (e.g., pyproject.toml, setup.py) and environment examples to manage deployment.
• The system builds upon Bittensor for its decentralized network.
• Additional libraries and frameworks will be evident from the requirements files and Python modules (e.g., for video processing, asynchronous operations, and database interactions).
Essential Elements
• Use clear, labeled blocks for:
• Miner Node(s): Including subcomponents like video intake, processing core, sports-specific modules.
• Validator Node(s): Including subcomponents for challenge handling, DB operations, evaluation loops, and health endpoints.
• External Network Management: Represented as a separate block interfacing with both miners and validators.
• Data flow from the miner’s video processing output to the validator’s input.
• Feedback from validators back to the network management (control loop).
• Interactions between utility services such as database and common configuration modules.
• Differentiate between processing nodes (miners/validators) and supporting systems (databases, external endpoints).
Final Implementation Note
• Be as detailed as possible—break down each major role into its smallest identifiable components based on the file structure.