# adobe-research/videodoodles architecture

> Architecture diagram of the GitHub repository adobe-research/videodoodles, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adobe-research/videodoodles
- Repository: https://github.com/adobe-research/videodoodles

## Overview

##### Main System Components

a) Preprocessing Module
- Located in the 'preprocess' directory
- Handles video processing, depth maps, and feature tracking
- Processes raw video data before it can be used in the main application

b) Frontend Application
- Built with Svelte framework
- Contains canvas manipulation, animation controls, and UI components
- Includes WebGL shaders for rendering

c) Backend Service
- Python-based (app.py)
- Handles trajectory solving and keyframe management
- Manages data exports and processing

**Raw Video Data → Preprocessing → Web Application (Frontend + Backend)**

##### Component Interactions

- Data flow from preprocessing to storage
- WebSocket communication between frontend and backend
- User interaction flow through the UI components
- Rendering pipeline flow in the frontend
- Storage access patterns

##### Technical Details

For each component, add technical annotations:

Frontend:
- Svelte Framework
- WebGL/GLSL Shaders
- Canvas APIs
- Animation Controllers

Backend:
- Python Flask/FastAPI
- Trajectory Optimization
- Data Export Handlers

Preprocessing:
- Python Scripts
- Computer Vision Libraries
- Feature Extraction

##### Implementation Notes

- File storage structure for keyframes
- WebSocket communication protocols
- Shader pipeline for rendering
- Data format specifications

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Preprocessing
        VI[Video Input]
        DM["Depth Map Generation"]:::depth
        FT["Feature Tracking"]:::feature
        CP["Camera Processing"]:::camera
        PVD["Processed Video Data"]
        
        VI --> DM
        DM --> FT
        FT --> CP
        CP --> PVD
    end

    subgraph "Frontend Application"
        UI["UI Components"]:::ui
        CM["Canvas Management"]:::canvas
        AC["Animation Controller"]:::animation
        WR["WebGL Rendering"]:::webgl
        WC["WebSocket Client"]:::websocket
        TM["Timeline Management"]:::timeline
        
        UI --> CM
        CM --> AC
        AC --> WR
        WC --> AC
        UI --> TM
        TM --> AC
    end

    subgraph "Backend Service"
        TS["Trajectory Solver"]:::trajectory
        KM["Keyframe Manager"]:::keyframe
        EH["Export Handler"]:::export
        DS["Data Storage"]:::storage
        
        TS --> KM
        KM --> EH
        KM --> DS
    end

    PVD --> DS
    WC <--> TS
    DS --> WR

    classDef preprocessing fill:#90EE90
    classDef frontend fill:#87CEEB
    classDef backend fill:#FFA07A
    
    %% Click events for components
    click DM "https://github.com/adobe-research/VideoDoodles/blob/main/preprocess/depth_maps.py"
    click FT "https://github.com/adobe-research/VideoDoodles/blob/main/preprocess/videowalk_features.py"
    click CP "https://github.com/adobe-research/VideoDoodles/blob/main/preprocess/cameras.py"
    click CM "https://github.com/adobe-research/VideoDoodles/blob/main/app/frontend/src/Canvas.js"
    click AC "https://github.com/adobe-research/VideoDoodles/blob/main/app/frontend/src/AnimationController.js"
    click WR "https://github.com/adobe-research/VideoDoodles/tree/main/app/frontend/src/shaders/"
    click WC "https://github.com/adobe-research/VideoDoodles/blob/main/app/frontend/src/websocket.js"
    click UI "https://github.com/adobe-research/VideoDoodles/tree/main/app/frontend/src/components/"
    click TM "https://github.com/adobe-research/VideoDoodles/blob/main/app/frontend/src/Timeline.svelte"
    click TS "https://github.com/adobe-research/VideoDoodles/blob/main/app/backend/scripts/solve_trajectory.py"
    click KM "https://github.com/adobe-research/VideoDoodles/blob/main/app/backend/scripts/state_management.py"
    click EH "https://github.com/adobe-research/VideoDoodles/tree/main/app/backend/exports/"
    click DS "https://github.com/adobe-research/VideoDoodles/tree/main/app/backend/data/"

    %% Legend
    subgraph Legend
        L1["Preprocessing Component"]:::preprocessing
        L2["Frontend Component"]:::frontend
        L3["Backend Component"]:::backend
    end
```
