How fkryan/gazelle works
Main Components
a. Input Processing Layer:
• Image Input (448x448 RGB)
• Input Transform Module
b. Core Architecture Components:
• Frozen DINOv2 Backbone (ViT-B/ViT-L variants)
• Lightweight Gaze Decoder
• Multi-head Attention Mechanism
c. Output Components:
• Spatial Heatmap (64x64)
• In/Out Frame Prediction (for specific models)
[Input Layer]
[Backbone Layer]
[Decoder Layer]
[Output Layer]
Specific Elements
a. Input Section:
• Transformation pipeline
b. Backbone Section:
• ViT architecture structure
• Feature extraction points
c. Decoder Section:
• Lightweight decoder architecture
• Attention mechanisms
• Learned parameters vs. frozen ones
d. Output Section:
• Heatmap generation
• Optional in/out prediction branch
Important Details to Highlight
• Parameter counts to emphasize the lightweight nature
• Feature dimensions at key points
• Attention flow
• Where xformers optimization can be applied
Technical Specifications
• Input image size (448x448)
• Output heatmap size (64x64)
• Backbone variants (ViT-B/ViT-L)
Implementation Notes
Callouts for:
• PyTorch Hub integration points
• xformers optimization locations
• Multi-person inference capability
• Batch processing support