How yakhyo/gaze-estimation works
Main Components
• Data Management and Pre-processing:
• The “data” directory (which will hold datasets like Gaze360 and MPIIFaceGaze as shown in the README’s folder structure) and the “utils/datasets.py” file indicate that data ingestion and pre-processing are core components.
• Model Definitions:
• The “models” directory contains dedicated files (resnet.py, mobilenet.py, mobileone.py) implementing different neural network backbones.
• Training Module:
• “main.py” (and additionally “mpii_train.py”) serve as the entry point for training the gaze estimation models. They make use of configuration (config.py) and utility functions (“utils/helpers.py”) to manage training workflows.
• Evaluation Module:
• “evaluate.py” handles the testing/assessment of the trained models via command-line arguments.
• Inference Module:
• “inference.py” is used for live or video-based inference. It supports viewing the results and saving output video.
• ONNX Integration:
• “onnx_export.py” provides functionality to convert PyTorch models into the ONNX format.
• “onnx_inference.py” shows how to run inference using an ONNX model.
• Configuration and Utilities:
• “config.py” provides centralized configuration details.
• “utils” folder (with helpers and datasets) includes reusable code shared across other modules.
• Assets and Weights:
• The “assets” directory holds demonstration videos and images, while the “weights” directory is used to store pre-trained model weights.
Components’ Relationships and Interactions
• Data Flow:
• Data is loaded from the “data” directory using routines in “utils/datasets.py”. Pre-processing transforms are applied and then fed into the training routines (main.py, mpii_train.py).
• Model Pipeline:
• The training module uses model definitions from the “models” folder. Configurations (potentially from “config.py”) govern parameters such as network type, learning rate, and epoch count.
• Once trained, models are saved as weight files in the “weights” folder.
• Evaluation and Inference:
• The evaluation script (“evaluate.py”) takes the saved model weights to measure performance.
• The inference script (“inference.py”) loads a model (or even uses ONNX exported models via “onnx_inference.py”) to process incoming video or camera inputs and display/store the output.
• ONNX Component:
• “onnx_export.py” converts models to a format that is optimized for runtime (lightweight, mobile-friendly inference).
• This creates an alternative pipeline for deployment where users call “onnx_inference.py” for high-performance inference.
Architectural Patterns & Design Principles
• Modularity:
• The separation between modules (data, models, training, evaluation, inference) means that each component has a clear responsibility.
• Pipeline Processing:
• The flow from data pre-processing to training, evaluating, and finally deploying on inference or ONNX-based inference pipelines is linear and well-defined.
• Extensibility:
• With clearly separated model implementations and training/inference workflows, adding new network architectures or supporting new datasets is straightforward.
Specific Elements
• “Data & Pre-processing (data/ & utils/datasets.py)”
• “Model Architectures (models/ - ResNet, MobileNet, MobileOne)”
• “Training Engine (main.py, mpii_train.py)”
• “Evaluation (evaluate.py)”
• “Inference (inference.py)”
• “ONNX Export/Infernce (onnx_export.py, onnx_inference.py)”
• “Configuration (config.py) & Utilities (utils/helpers.py)”
• Training flow and backpropagation loops
• Data input to training and data output to evaluation/inference
• Conversion from standard model weights to ONNX models, then to inference
Final Recommendations
• Don’t shy away from breaking down the system into as many distinct components as possible – the clarity is paramount.