# yedp123/comfyui-yedp-action-director architecture

> Architecture diagram of the GitHub repository yedp123/comfyui-yedp-action-director, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yedp123/comfyui-yedp-action-director
- Repository: https://github.com/yedp123/comfyui-yedp-action-director

## Overview

1. **What this repo is**
   This is a ComfyUI custom node that embeds a full **client-side 3D director/viewport** into the workflow. The architecture is split between a small Python node package and a much larger browser-based scene editor/rendering engine.

2. **Python/ComfyUI boundary**
   `__init__.py` and `nodes.py` are the ComfyUI integration layer. Their job is to register the node, accept/emit workflow parameters, and carry serialized scene state inside standard ComfyUI JSON workflows. The Python side appears lightweight; the heavy 3D work is not done there.

3. **Frontend application**
   The actual editor/runtime lives under `web/`. `web/js/action_director.js` is the main application controller, with `web/yedp_manual.html` as the UI shell and `web/css/style.css` for layout. This is the interactive viewport, toolbar, gizmos, sequencer, and bake controls the user manipulates inside ComfyUI.

4. **3D engine and asset pipeline**
   The frontend is built on **Three.js** and bundles many loaders and helpers: GLTF/FBX/BVH/PLY/HDR loading, orbit and transform controls, skeleton utilities, mesh BVH acceleration, and a GPU path tracer. This means the viewport can ingest multiple DCC/export formats and render them with either raster or physically based methods.

5. **Character subsystem**
   A core subsystem is the rigged character manager. It is optimized for **Mixamo-compatible skeletons**, with retargeting/synonym handling for variant rigs. The scene supports multiple characters, per-character animation sequences, crossfading, gendered depth mesh swapping, and prop attachments that ride along with the rig.

6. **Animation and timeline data flow**
   Character motion comes from imported animation files and is organized into sequenced slots in the UI. The runtime keeps track of playback, scrubbing, and keyframed state so the same scene can be reproduced later from the saved workflow or from an external scene JSON file.

7. **Facial mocap subsystem**
   Facial capture is handled entirely in-browser using **MediaPipe Tasks Vision** assets bundled in `web/js/` (`face_landmarker.task`, `tasks_vision.js`, WASM support files). Input can be webcam or video file, and the captured facial tracks are saved as local mocap data that can then be bound to characters in the scene.

8. **Environment and background assets**
   Environments are loaded as 3D assets from the filesystem, including support for **GLB/FBX**, **HDRI lighting**, and **point cloud / Gaussian splat-style assets** via PLY-related paths and Gaussian splatting modules. The repo also includes logic for proxy shading/shadows so these environments can participate in depth and silhouette passes, not just preview rendering.

9. **Rendering and bake pipeline**
   The main output is a multi-pass bake system that produces distinct batches for **pose, depth, canny, normal, shaded, alpha, and textured** views. Rendering is performed in the browser with WebGL/WebGPU-adjacent GPU tooling, and the path-traced shaded mode uses the bundled `three-gpu-pathtracer` plus `three-mesh-bvh` for acceleration.

10. **Persistence and file-system conventions**
   The system relies on ComfyUI input folders for discovery of animations, environments, cameras, and mocap assets, with a “sync folders” behavior to hot-reload new files. Scene state can also be exported to a physical `.json` file, so the editor is not just ephemeral UI state; it is a serializable scene document plus external asset references.

11. **Operational boundary**
   Almost all compute-heavy work is local: browser GPU rendering, WebAssembly decoding, and webcam/video face tracking. There is no evidence of a remote service dependency; the architecture is a self-contained ComfyUI node plus a rich local web runtime.

12. **What to draw in an architecture diagram**
   The key boxes are: **ComfyUI node wrapper (Python)**, **browser-based 3D director app**, **asset folders on disk**, **local mocap capture**, and **multi-pass render output back into the workflow**. The main data flow is from filesystem assets and user input into the frontend scene graph, then into baked image passes and serialized scene/workflow state.

Main entry points: [`nodes.py`](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/nodes.py) (nodes.py), [`web/js/action_director.js`](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/action_director.js) (Action Director), [`web/js/Pass.js`](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/Pass.js) (Bake Pipeline), [`web/js/yedp_manual.html`](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/yedp_manual.html) (Manual UI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_python["ComfyUI Wrapper"]
  node_init_py["__init__.py<br/>comfyui entry<br/>[__init__.py]"]
  node_nodes_py["nodes.py<br/>workflow bridge<br/>[nodes.py]"]
end

subgraph group_frontend["Browser Director"]
  node_director["Action Director<br/>app controller<br/>[action_director.js]"]
  node_ui_shell["Manual UI<br/>html shell<br/>[yedp_manual.html]"]
  node_styles["Style Sheet<br/>ui layout<br/>[style.css]"]
  node_scene_graph(("Scene State<br/>scene model"))
  node_viewport["3D Viewport<br/>three.js scene"]
  node_loader_stack["Import Stack<br/>asset loaders<br/>[GLTFLoader.js]"]
  node_controls(("Manipulators<br/>interaction tools<br/>[OrbitControls.js]"))
  node_rig_system["Rig Manager<br/>character system"]
  node_anim_timeline(("Anim Timeline<br/>motion sequencing"))
  node_render_pipeline["Bake Pipeline<br/>multi-pass render<br/>[Pass.js]"]
  node_path_tracer["Path Tracer<br/>gpu shading"]
  node_mesh_bvh["Mesh BVH<br/>acceleration"]
end

subgraph group_vision["Facial Mocap"]
  node_mocap["Face Mocap<br/>vision capture<br/>[tasks_vision.js]"]
  node_vision_assets["Vision Assets<br/>wasm/task bundle"]
end

subgraph group_assets["Asset Library"]
  node_models[("Bundled Models<br/>3d assets<br/>[Yedp_Rig.glb]")]
end

node_init_py -->|"registers"| node_nodes_py
node_nodes_py -->|"serializes state"| node_director
node_ui_shell -->|"hosts"| node_director
node_styles -->|"styles"| node_ui_shell
node_director -->|"manages"| node_scene_graph
node_scene_graph -->|"drives"| node_viewport
node_loader_stack -->|"imports"| node_scene_graph
node_controls -->|"manipulates"| node_viewport
node_rig_system -->|"binds"| node_scene_graph
node_anim_timeline -->|"animates"| node_rig_system
node_anim_timeline -->|"persists"| node_scene_graph
node_render_pipeline -->|"renders"| node_viewport
node_path_tracer -->|"shades"| node_render_pipeline
node_mesh_bvh -->|"accelerates"| node_path_tracer
node_mocap -->|"feeds"| node_rig_system
node_vision_assets -->|"powers"| node_mocap
node_models -->|"supplies"| node_loader_stack
node_nodes_py -->|"returns outputs"| node_render_pipeline

click node_init_py "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/__init__.py"
click node_nodes_py "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/nodes.py"
click node_director "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/action_director.js"
click node_ui_shell "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/yedp_manual.html"
click node_styles "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/css/style.css"
click node_loader_stack "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/GLTFLoader.js"
click node_controls "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/OrbitControls.js"
click node_render_pipeline "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/Pass.js"
click node_path_tracer "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/three-gpu-pathtracer.module.js"
click node_mesh_bvh "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/three-mesh-bvh.module.js"
click node_mocap "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/tasks_vision.js"
click node_vision_assets "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/face_landmarker.task"
click node_models "https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/Yedp_Rig.glb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_init_py,node_nodes_py toneBlue
class node_director,node_ui_shell,node_styles,node_scene_graph,node_viewport,node_loader_stack,node_controls,node_rig_system,node_anim_timeline,node_render_pipeline,node_path_tracer,node_mesh_bvh toneAmber
class node_mocap,node_vision_assets toneMint
class node_models toneRose
```

## Components

### ComfyUI Wrapper

Node registration and workflow state bridge

- [__init__.py](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/__init__.py): comfyui entry · Registers the custom node package with ComfyUI · `__init__.py`
- [nodes.py](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/nodes.py): workflow bridge · Serializes node inputs and scene state for workflows · `nodes.py`

### Browser Director

Client-side 3D editor and rendering runtime

- [Action Director](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/action_director.js): app controller · Main browser-side scene editor and runtime · `web/js/action_director.js`
- [Manual UI](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/yedp_manual.html): html shell · Top-level page that hosts the director UI · `web/js/yedp_manual.html`
- [Style Sheet](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/css/style.css): ui layout · Visual styling for the editor and viewport · `web/css/style.css`
- **Scene State**: scene model · Editable scene, sequencing, and persistent director state
- **3D Viewport**: three.js scene · Interactive scene view with controls and gizmos
- [Import Stack](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/GLTFLoader.js): asset loaders · GLTF, FBX, BVH, PLY, HDR and related import helpers · `web/js/GLTFLoader.js`
- [Manipulators](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/OrbitControls.js): interaction tools · Orbit and transform controls for editing objects · `web/js/OrbitControls.js`
- **Rig Manager**: character system · Mixamo-oriented character setup, retargeting, and props
- **Anim Timeline**: motion sequencing · Playback, scrubbing, crossfades, and clip slots
- [Bake Pipeline](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/Pass.js): multi-pass render · Produces pose, depth, canny, normal, shaded, alpha, and textured outputs · `web/js/Pass.js`
- [Path Tracer](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/three-gpu-pathtracer.module.js): gpu shading · Physically based shaded render path accelerated by mesh BVH · `web/js/three-gpu-pathtracer.module.js`
- [Mesh BVH](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/three-mesh-bvh.module.js): acceleration · Spatial acceleration for ray tracing and heavy geometry queries · `web/js/three-mesh-bvh.module.js`

### Facial Mocap

In-browser face tracking and capture assets

- [Face Mocap](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/tasks_vision.js): vision capture · Browser facial tracking from webcam or video input · `web/js/tasks_vision.js`
- [Vision Assets](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/js/face_landmarker.task): wasm/task bundle · MediaPipe task model and WASM support files · `web/js/face_landmarker.task`

### Asset Library

3D models and bundled runtime resources

- [Bundled Models](https://github.com/yedp123/comfyui-yedp-action-director/blob/main/web/Yedp_Rig.glb): 3d assets · Included rig and sample scene assets · `web/Yedp_Rig.glb`

## Connections

- __init__.py → nodes.py: registers · ComfyUI exposes the node implementation through the Python wrapper
- nodes.py → Action Director: serializes state · Workflow inputs and persisted scene data are handed to the browser runtime
- Manual UI → Action Director: hosts · The HTML shell boots the application controller
- Style Sheet → Manual UI: styles · UI layout is applied to the manual shell
- Action Director → Scene State: manages · Controller owns scene composition, playback state, and persistence
- Scene State → 3D Viewport: drives · Scene state is rendered into the interactive 3D viewport
- Import Stack → Scene State: imports · External 3D assets are loaded into the scene model
- Manipulators → 3D Viewport: manipulates · User interaction tools modify the viewport scene
- Rig Manager → Scene State: binds · Characters, props, and retargeted rigs become scene entities
- Anim Timeline → Rig Manager: animates · Sequenced clips and playback control drive characters
- Anim Timeline → Scene State: persists · Timeline and playback state are saved with the scene
- Bake Pipeline → 3D Viewport: renders · Viewport can output the multi-pass bake set
- Path Tracer → Bake Pipeline: shades · Path-traced shaded mode feeds the bake system
- Mesh BVH → Path Tracer: accelerates · BVH data structures speed up ray tracing and geometry queries
- Face Mocap → Rig Manager: feeds · Captured facial tracks are bound onto characters
- Vision Assets → Face Mocap: powers · Model and WASM bundles support browser face tracking
- Bundled Models → Import Stack: supplies · Bundled GLB assets provide example rigs and scenes
- nodes.py → Bake Pipeline: returns outputs · Rendered/baked results and serialized scene state flow back into ComfyUI
