How yemount/pose-animator works
Project type and purpose
• This is a browser-based, front-end application (“Pose Animator”) that takes a 2D SVG illustration and animates it in real time using ML pose-estimation models (PoseNet and FaceMesh) via TensorFlow.js.
• Its primary use cases are two demos: a live camera feed demo (camera.html + camera.js) and a static-image demo (static_image.html + static_image.js).
File structure
• Top level:
• HTML entry points: camera.html, static_image.html, index.html
• JS entry scripts: camera.js, static_image.js
• Build/config: package.json, package-lock.json, firebase.json
• Third-party library: third_party/paper (Paper.js)
• Core modules: illustrationGen/, utils/
• Static assets: resources/ (gifs, illustrations, samples)
• Patterns:
• Pipeline stages: load model → read keypoints → update skeleton → redraw SVG paths
README for insights
• Describes two primary functional modules:
a) Skeleton rig parsing (from SVG skeleton group and predefined bone hierarchy)
b) Real-time bone updating (driven by PoseNet/FaceMesh keypoints)
c) Illustration surface rendering (vector path manipulation)
• Mentions TensorFlow.js and Paper.js as core frameworks.
Main components
A. Client Browser
– HTML pages (camera.html, static_image.html)
– JS bootstrap
B. Input Sources
– Webcam stream (MediaDevices API)
– Static image file drop
C. ML Inference Layer
– TensorFlow.js Loader
– PoseNet model
– FaceMesh model
D. Skeleton Module
– SVG parser (reads “skeleton” group from SVG)
– Bone hierarchy data structure
E. Illustration Module
– illustrationGen (generates/control point mapping)
– SVG path utilities (utils/svgUtils.js)
F. Rendering Engine
– Paper.js (third_party/paper) or direct SVG DOM updates
– Canvas/SVG output
G. Utilities
H. Hosting/Deployment
– Firebase Hosting (firebase.json)
Interactions
1. User opens camera.html or static_image.html
2. JS entry script initializes ML models (TensorFlow.js)
3. On each animation tick or static load:
• Frame/image → PoseNet/FaceMesh → keypoints
• Keypoints → Skeleton Module → bone transformations
• Transformations + SVG “illustration” paths → Illustration Module
• Updated paths rendered via Paper.js or direct SVG updates
5. All assets and scripts served from Firebase Hosting
Architectural patterns and design principles
• Modular separation: ML inference layer, skeletal logic, rendering logic, util libraries
• Pipeline flow: Input → Compute → Transform → Render
• Single-page-style demos with dynamic JS modules loaded at runtime
Detail
• Every module folder as its own subsystem if needed (e.g., utils/, illustrationGen/)
• Call out third_party dependencies (Paper.js)
• Two demo flows (camera vs. static) either as parallel lanes or combined with conditional annotations
• Where SVG files are parsed vs. where vector updates occur