How uzankhan/swordflow works
Swordflow turns camera or mouse input into a live motion-controlled 3D experience: the user enters a mode, hand tracking can acquire camera access and interpret local vision results as gesture signals, and the scene responds with sword-swarm or dragon gameplay. The main controller coordinates controls and routes signals into the world renderer. The README also documents formations, volley outcomes and recall, dragon recentering, pause, and tracking cancellation. Source excerpts show the worker/tracker and scene integration; unsampled implementation details are represented only where documented.
Main entry points: HandTracker.ts Camera lifecycle · SwordWorld.ts World simulation · main.ts Experience controller · inference.worker.ts Vision inference
Components 12
Experience control
- Experience controller
src/main.ts - Mouse input
src/main.ts
Local hand tracking
- Camera stream
- Camera lifecycle
src/hand/HandTracker.ts - Vision inference
src/hand/inference.worker.ts - Vision messages
src/hand/vision-types.ts - Gesture interpretation
src/hand/gesture.ts
3D world and modes
- World simulation
src/scene/SwordWorld.ts - Sword swarm
src/scene/SwordWorld.ts - Procedural dragon
src/scene/DragonFlight.ts
Other components
- Player
- MediaPipe Vision
Connections 16
- Player → Experience controller: chooses mode
README.md - Experience controller → Camera lifecycle: starts tracking
src/main.ts - Camera lifecycle → Camera stream: requests frames
src/hand/HandTracker.ts - Camera lifecycle → Vision inference: dispatches frames
src/hand/HandTracker.ts - Vision inference → MediaPipe Vision: runs inference
src/hand/inference.worker.ts - Vision inference → Camera lifecycle: returns results
src/hand/inference.worker.ts - Camera lifecycle → Gesture interpretation: interprets poses
src/hand/HandTracker.ts - Gesture interpretation → Vision messages: uses vision data
src/hand/gesture.ts - Vision inference → Vision messages: serializes results
src/hand/inference.worker.ts - Experience controller → World simulation: routes hand signals
src/main.ts - Experience controller → Mouse input: accepts fallback
README.md - Mouse input → World simulation: steers simulation
README.md - World simulation → Sword swarm: simulates swarm
src/scene/SwordWorld.ts - World simulation → Procedural dragon: updates dragon
src/scene/SwordWorld.ts - Procedural dragon → World simulation: adds scene object
src/scene/DragonFlight.ts - Player → Camera stream: grants permission
README.md