How hukenovs/slovo works
Main Components
• Data Repository – Represent the Slovo dataset as a primary component. This includes the raw video files (both trimmed and original), the annotation files (like the CSV that holds metadata and timestamps), and additional annotations such as the MediaPipe landmarks JSON.
• Configuration & Constants – Include the config_example.yaml and constants.py files. These indicate that system parameters and constants are defined in a dedicated module, which the demo and possibly other scripts use.
• Demo / Inference Pipeline – The demo.py script is the entry point for running inference. It is responsible for reading the configuration, possibly setting up multi-processing (via the --mp flag), and handling the prediction request.
• Example Notebooks – The examples directory contains Jupyter notebooks that demonstrate how to perform tasks like video processing, visualizing landmarks, and possibly running the models. These serve as usage examples and testing modules.
• License and Documentation – Although not a processing component, showing a documentation module (README, licenses in the license folder) is valuable to indicate that the project is well-documented and contains multiple usage instructions and citations.
Relationships and Data Flow
• The Model Inference Engine block stands as the center of the pipeline: it loads one of the pre-trained models (with ONNX or TorchScript weights) based on configuration and then processes the pre-processed video inputs.
• The Demo and Example Notebooks blocks then interact with the Inference Engine. For example, demo.py uses command line parameters to initialize the configuration, read input video data, process it through the model, and then provide output (such as predictions showcased in the demo.gif found in the images folder).
– Data Repository → Pre-processing/Annotation Parser
– Pre-processing Module → Model Inference Engine
– Configuration/Constants → Demo Script & Model Inference Engine (they both depend on configuration settings)
– Inference Engine → Demo/Visualization Interface (demo.py and notebooks display results).
Architectural Patterns and Technologies
• Although the project is not an enterprise web app, it still exhibits a modular design. Modules are clearly separated: one for configuration, one for data and pre-processing, and one for model inference/demonstration.
• “plug-and-play” nature of pre-trained models (multiple architectures available) which benefits from uniform interfaces in demo.py.
• The use of command-line flags in demo.py (like --mp for multiprocessing) shows that the pipeline is designed with performance/scalability in mind although on a prototype scale.
• Mention that libraries such as ONNX and TorchScript are integral since they encapsulate the models and are delivered as weights. Their presence should be captured within the Model Inference Engine component.