How comfy-org/comfyui-embedded-workflow-editor works
1. What this repo is
- A single Next.js + Bun application for editing ComfyUI workflow JSON embedded inside media files.
- It is browser-first: there is no database or long-lived backend service. The app loads a file, decodes embedded workflow data, lets the user edit it, then re-embeds/saves it.
2. Primary user-facing app
- app/page.tsx is the main editor entry point, with app/layout.tsx, app/globals.css, and app/fonts providing the shell and styling.
- The UI is centered around opening media, showing a preview, exposing the embedded workflow, and saving the edited file.
3. State management boundary
- app/uiState.tsx and app/persistState.tsx indicate the editor state is managed in client-side state providers.
- This is where file selection, current workflow JSON, preview state, and edit/session persistence live, rather than in a server store.
4. Media ingestion paths
- The app supports multiple ways to load content:
- local upload,
- mounting a local directory,
- direct URL input / ?url=... loading.
- app/types/file-system-access.d.ts shows explicit support for the File System Access API for local directory workflows.
5. Server-side media fetch/proxy layer
- app/api/media/route.ts plus app/api/media/detectContentType.ts form the server boundary for media loading.
- This layer likely handles fetching remote files, normalizing responses, and detecting the true MIME/content type before the browser-side codec layer processes bytes.
6. Format-specific metadata codec layer
- The core binary work lives in app/utils/.
- There is a shared dispatcher in exif.ts, with format-specific readers/writers for:
- exif-png.ts
- exif-webp.ts
- exif-flac.ts
- exif-mp3.ts
- exif-mp4.ts
- encodeTIFFBlock.ts and utils.ts are shared low-level helpers for binary chunk assembly and parsing.
7. Supported media formats
- The architecture is organized around container-specific metadata embedding for:
- PNG
- WEBP
- FLAC
- MP3
- MP4
- The editor extracts the embedded ComfyUI workflow from each format’s metadata structure, edits it as JSON, then writes it back into the same container type.
8. Output/save flow
- The runtime data flow is: bytes in → detect format → extract workflow → edit in UI → re-encode bytes → save/download.
- This means the app’s critical correctness boundary is the round-trip between raw media bytes and embedded workflow metadata.
9. Validation strategy
- Each codec has companion tests under app/utils/*.test.ts.
- Integration fixtures live in tests/ with real sample media and workflow JSON sidecars, including malformed cases for WEBP.
- This makes format parsing and re-embedding the main testable subsystem, not the UI.
10. Runtime/tooling/infrastructure
- The project uses Bun for development and CI (.github/workflows/bun-test.yml).
- Tailwind/PostCSS handle styling; Next.js provides the app router and API routes.
- middleware.ts and next.config.ts define request/runtime behavior at the framework boundary, while GitHub Actions enforce tests and formatting.
11. What is intentionally absent
- No database, message queue, auth service, or separate backend API.
- The architecture is essentially client editor + thin server fetch/type-detection layer + binary codec library.
12. How to draw it
- A good diagram would show:
- user/browser UI on the left,
- an API/media fetch boundary in the middle,
- a shared parsing/encoding library underneath,
- and container-specific codecs branching out for PNG/WEBP/FLAC/MP3/MP4.
- The main architectural concern is preserving exact media bytes while editing embedded workflow metadata.
Main entry points: exif.ts Dispatcher · exif-png.test.ts Codec Tests · page.tsx Editor · layout.tsx App Shell
Components 22
Browser App
Client-side editor shell, state, and file-loading UX.
- Editor · Main browser editor entry point for opening, previewing, editing, and saving embedded workflows.
app/page.tsx - App Shell · Top-level document shell and global app structure.
app/layout.tsx - Styles · Global styling and visual baseline for the editor.
app/globals.css - Font Assets · Local font resources used by the UI.
app/fonts - UI State · Client-side editor state provider for file selection, workflow data, preview state, and session behavior.
app/uiState.tsx - Persist State · Session and edit persistence boundary for the browser editor.
app/persistState.tsx - File Access API · Type support for directory-based file workflows via the File System Access API.
app/types/file-system-access.d.ts
Server Boundary
Thin Next.js runtime layer for media fetch and request handling.
- Media API · Server-side media loading and proxy entry point.
app/api/media/route.ts - Content Type · Content-type detection for fetched media bytes before codec processing.
app/api/media/detectContentType.ts
Codec Library
Binary metadata extraction and re-embedding for supported containers.
- Dispatcher · Shared format dispatcher that selects the appropriate embedded-workflow reader/writer.
app/utils/exif.ts - Binary Utils · Shared low-level byte and parsing helpers used by codecs.
app/utils/utils.ts - TIFF Encode · Shared block assembly helper for metadata re-encoding.
app/utils/encodeTIFFBlock.ts - PNG Codec · Embedded workflow extraction and write-back for PNG metadata.
app/utils/exif-png.ts - WEBP Codec · Embedded workflow extraction and write-back for WEBP metadata.
app/utils/exif-webp.ts - FLAC Codec · Embedded workflow extraction and write-back for FLAC metadata.
app/utils/exif-flac.ts - MP3 Codec · Embedded workflow extraction and write-back for MP3 metadata.
app/utils/exif-mp3.ts - MP4 Codec · Embedded workflow extraction and write-back for MP4 metadata.
app/utils/exif-mp4.ts
Validation
Unit and fixture-based checks for round-trip correctness.
- Codec Tests · Per-format codec tests for parse and round-trip behavior.
app/utils/exif-png.test.ts - Fixture Media · Real sample media and workflow sidecars, including malformed WEBP cases.
tests
Tooling
Framework, styling, and CI/runtime configuration.
- Bun CI · Bun-based test and formatting automation in GitHub Actions.
.github/workflows/bun-test.yml - Framework Config · Framework-level runtime and build configuration.
next.config.ts - Middleware · Framework boundary behavior for incoming requests.
middleware.ts
Connections 29
- Editor → UI State: uses · Editor UI reads and updates browser state.
- Editor → Persist State: persists · Editor session and workflow edits are persisted on the client.
- Editor → File Access API: loads dirs · Editor supports local directory workflows through browser file access.
- Editor → Media API: fetches media · Editor can load remote or proxied media through the server route.
- App Shell → Editor: hosts · App shell renders the main editor page.
- App Shell → Styles: applies · Global stylesheet affects the whole app.
- App Shell → Font Assets: includes · Local font assets are available to the shell.
- Media API → Content Type: detects · Media route uses content-type detection before returning bytes.
- Media API → Dispatcher: feeds · Fetched media is handed to the codec dispatcher after normalization.
- Dispatcher → PNG Codec: routes png · Dispatcher selects the PNG codec for PNG containers.
- Dispatcher → WEBP Codec: routes webp · Dispatcher selects the WEBP codec for WEBP containers.
- Dispatcher → FLAC Codec: routes flac · Dispatcher selects the FLAC codec for FLAC containers.
- Dispatcher → MP3 Codec: routes mp3 · Dispatcher selects the MP3 codec for MP3 containers.
- Dispatcher → MP4 Codec: routes mp4 · Dispatcher selects the MP4 codec for MP4 containers.
- PNG Codec → Binary Utils: uses · PNG codec relies on shared binary helpers.
- WEBP Codec → Binary Utils: uses · WEBP codec relies on shared binary helpers.
- FLAC Codec → TIFF Encode: uses · FLAC codec uses TIFF block assembly helpers for metadata encoding.
- MP3 Codec → Binary Utils: uses · MP3 codec relies on shared binary helpers.
- MP4 Codec → TIFF Encode: uses · MP4 codec uses shared block assembly helpers for metadata encoding.
- PNG Codec → Codec Tests: verified by · PNG codec behavior is covered by unit tests.
- WEBP Codec → Codec Tests: verified by · WEBP codec behavior is covered by unit tests.
- FLAC Codec → Codec Tests: verified by · FLAC codec behavior is covered by unit tests.
- MP3 Codec → Codec Tests: verified by · MP3 codec behavior is covered by unit tests.
- MP4 Codec → Codec Tests: verified by · MP4 codec behavior is covered by unit tests.
- Dispatcher → Fixture Media: validated by · Fixture media exercises real-world round-trip parsing and malformed cases.
- Bun CI → Codec Tests: runs · CI executes codec unit tests.
- Bun CI → Fixture Media: checks · CI relies on fixture media for integration coverage.
- Framework Config → App Shell: configures · Next.js configuration shapes app runtime and build behavior.
- Middleware → Media API: guards · Middleware sits on the request boundary before media handling.