How adobe-research/videodoodles works
Main System Components
a) Preprocessing Module
• Located in the 'preprocess' directory
• Handles video processing, depth maps, and feature tracking
• Processes raw video data before it can be used in the main application
b) Frontend Application
• Built with Svelte framework
• Contains canvas manipulation, animation controls, and UI components
• Includes WebGL shaders for rendering
c) Backend Service
• Python-based (app.py)
• Handles trajectory solving and keyframe management
• Manages data exports and processing
Raw Video Data → Preprocessing → Web Application (Frontend + Backend)
Component Interactions
• Data flow from preprocessing to storage
• WebSocket communication between frontend and backend
• User interaction flow through the UI components
• Rendering pipeline flow in the frontend
• Storage access patterns
Technical Details
For each component, add technical annotations:
Frontend:
• Svelte Framework
• WebGL/GLSL Shaders
• Canvas APIs
• Animation Controllers
Backend:
• Python Flask/FastAPI
• Trajectory Optimization
• Data Export Handlers
Preprocessing:
• Python Scripts
• Computer Vision Libraries
• Feature Extraction
Implementation Notes
• File storage structure for keyframes
• WebSocket communication protocols
• Shader pipeline for rendering
• Data format specifications