How cupidbity/spiderlily works
Purpose
Spiderlily is an interactive audiovisual artwork driven by webcam hand gestures. It provides two independent runtimes: a browser-based 3D visualiser and the original TouchDesigner performance project.
Entry Points
• Browser runtime: spiderlily-web/index.html, served over HTTP(S) so camera access and MediaPipe tracking work.
• TouchDesigner runtime: InteractiveFlower.43.toe, opened and run inside TouchDesigner.
• Runtime-specific operation details are documented in README.md and spiderlily-web/README.md.
Web Visualisation Pipeline
spiderlily-web/index.html is a self-contained application with no build pipeline. It loads Three.js and MediaPipe from CDNs, requests webcam input, derives hand gesture state, and renders the flower scene plus a camera/tracking HUD. Browser controls support camera orbit/zoom, HUD visibility, and a debug auto-bloom mode.
Gesture-to-Plant Control Boundary
Both implementations share the same interaction model: the left hand controls plant growth and the right hand controls flower blooming; pinch/spread state is the primary continuous gesture signal. Webcam permissions, browser camera APIs, and MediaPipe hand tracking are external boundaries for the web path. spiderlily-web/index.html owns the translation from tracking output to visual parameters.
TouchDesigner Composition
InteractiveFlower.43.toe contains the original editable TouchDesigner network and is the primary composition/rendering entry point for that runtime. It depends on reusable MediaPipe-oriented TouchDesigner components under toxes/, notably toxes/hand_tracking.tox, toxes/face_tracking.tox, and toxes/pose_tracking.tox. The project is intended for live webcam-driven playback or performance rather than a compiled deployment.
Tracking Assets and External Dependency
The repository includes task-specific tracking/tox assets such as toxes/hand_tracking.tox, toxes/object_tracking.tox, and toxes/image_classification.tox. The TouchDesigner route additionally requires a separately downloaded MediaPipe.tox engine placed beside them; this is not committed due to its size. Thus the key infrastructure split is browser CDN dependencies for the web experience versus TouchDesigner plus local tox/plugin dependencies for the original piece.
Main entry points: InteractiveFlower.43.toe Interactive flower project · index.html Flower scene · hand_tracking.tox Hand tracking component · README.md Runtime documentation
Components 16
Browser runtime
- Web app entry · Self-contained browser visualiser.
spiderlily-web/index.html - Gesture translation · Maps left growth and right bloom gestures to visual parameters.
spiderlily-web/index.html - Flower scene · Renders the reactive 3D plant.
spiderlily-web/index.html - Camera and tracking HUD
spiderlily-web/index.html - Viewer controls · Orbit, zoom, HUD, and auto-bloom controls.
spiderlily-web/index.html
TouchDesigner runtime
- Interactive flower project · Primary editable performance and rendering network.
InteractiveFlower.43.toe - Live webcam input
- Hand tracking component
toxes/hand_tracking.tox - Face tracking component
toxes/face_tracking.tox - Pose tracking component
toxes/pose_tracking.tox - Other tracking assets · Includes object tracking and image classification components.
toxes/object_tracking.tox
External dependencies
- Webcam input
- Hand tracking · Produces hand landmarks and gesture state.
- Three.js CDN
- MediaPipe engine · Separately downloaded dependency placed beside repository toxes.
Other components
- Runtime documentation · Documents web serving and TouchDesigner operation.
README.md
Connections 16
- Webcam input → Web app entry: camera frames
- Web app entry → Hand tracking: submits frames
- Hand tracking → Gesture translation: hand landmarks
- Gesture translation → Flower scene: growth and bloom parameters
- Three.js CDN → Flower scene: rendering
- Flower scene → Camera and tracking HUD: tracking display
- Viewer controls → Flower scene: camera and debug settings
- Live webcam input → Interactive flower project: live video
- Interactive flower project → Hand tracking component: uses
- Interactive flower project → Face tracking component: uses
- Interactive flower project → Pose tracking component: uses
- Interactive flower project → Other tracking assets: can use
- MediaPipe engine → Hand tracking component: tracking engine
- Hand tracking component → Interactive flower project: hand gesture data
- Runtime documentation → Web app entry: web operation
- Runtime documentation → Interactive flower project: performance operation