How seanbarzilay/transcoderr works
1. Workspace shape
- This is a Rust monorepo with three runtime-facing crates: transcoderr (main server/worker), transcoderr-api-types (shared API DTOs), and transcoderr-mcp (stdio MCP client for AI tools).
- The repo also contains an embedded React UI (web/) and a marketing/documentation site (site/), both built into or alongside the product.
2. Core product boundary: one control plane, two execution modes
- transcoderr is the central application.
- It runs primarily as a single-binary server that accepts webhooks, exposes a typed API, stores state in SQLite, and drives transcoding jobs.
- The same binary can also run as a worker process, which executes delegated work from another transcoderr instance over a custom wire protocol.
3. Persistent state and configuration
- State is stored in embedded SQLite with migrations under crates/transcoderr/migrations.
- The schema covers jobs/runs, flows, sources, notifiers, plugins/plugin catalogs, API tokens, workers, checkpoints, settings, and run events.
- Config is file-based (config.toml from config.example.toml) and first boot creates it in the data volume.
4. Ingress paths: webhooks and typed API
- External media managers push into the system through webhook endpoints for Radarr, Sonarr, Lidarr, and a generic /webhook/:name.
- The frontend talks to /api/... endpoints for CRUD, validation, setup, browsing, and live status.
- Authentication is centralized around API tokens and source-specific webhook auth, so the ingress boundary is shared but policy differs by route.
5. Job lifecycle: webhook to run
- A webhook becomes a queued job in the database.
- A local worker pool claims jobs, creates a run record, and executes the configured flow step-by-step.
- Each step emits lifecycle events into run_events, which are persisted and also broadcast to the live event stream.
6. Flow engine as the main orchestration layer
- The flow/ subsystem parses and validates YAML flow definitions, builds an execution plan, and evaluates expressions/conditionals.
- Flows are declarative and trigger-based; they map source events to ordered step sequences.
- The engine carries mutable run context, including a media plan that is accumulated before execution.
7. One-pass media transformation pipeline
- The steps/ subsystem contains the step registry and all built-in step kinds.
- Important distinction: many plan.* steps only mutate the in-memory media plan; plan.execute materializes that plan into a single ffmpeg invocation.
- Other steps handle probing, verification, file output/move/copy/delete, shell/webhook actions, subtitle/audio fixes, and notification emission.
8. Media/tooling dependencies
- The transcode pipeline depends on external ffmpeg/ffprobe at runtime.
- Hardware acceleration is probed at startup (hw/) and can route work through NVENC, QSV, VAAPI, or VideoToolbox, with CPU fallback if needed.
- This means the media-processing boundary is split between internal planning logic and external codec/runtime execution.
9. Sources, browsing, and path translation
- arr/ and related API modules support browsing Radarr/Sonarr libraries and reconciling source metadata.
- Path mapping is a first-class boundary because the server may see the same media path differently than a remote worker.
- The browse/manual-transcode path uses the same flow engine as webhook-triggered jobs, but the source is chosen interactively from the UI.
10. Notifications and outbound integrations
- notifiers/ provides pluggable outbound delivery for Discord, ntfy, Telegram, generic webhook, and Jellyfin rescans.
- Notification steps are part of the flow, so they are triggered by run state rather than by a separate background subsystem.
- This keeps side effects inside the same run lifecycle and makes them visible in the event timeline.
11. Plugins and extension model
- Plugins are externally supplied executables that speak JSON-RPC over stdin/stdout.
- The server can browse plugin catalogs, download tarballs, verify them, install dependencies, and hot-reload the step registry without restart.
- Plugin state is persisted, but runtime loading is dynamic; the boundary is between trusted server orchestration and operator-controlled plugin artifacts.
12. Distributed execution
- The worker/ subsystem handles enrollment, discovery, connections, protocol, pooling, and execution on remote hosts.
- Workers can be auto-discovered on LAN or manually paired, and the coordinator dispatches heavy ffmpeg/plugin steps to them.
- This is the main horizontal scaling boundary: orchestration stays centralized, but compute can move to another host with path mappings and cancellation propagation.
13. Live observability and UI sync
- bus/ provides an internal SSE event bus; /api/stream exposes job state, queue updates, and run events to the frontend.
- The React app subscribes to this stream to render progress, timelines, queue state, and live ffmpeg status.
- Metrics are also exposed for Prometheus, so operational visibility is split between browser-facing live events and scrape-based telemetry.
14. Client surfaces
- web/ is the operator console: sources, flows, notifiers, plugins, runs, workers, settings, and manual browsing/transcode actions.
- transcoderr-mcp is a separate stdio daemon that wraps the same backend API for AI clients; its tools mirror the operator surface for read/write tasks.
- transcoderr-api-types exists to keep these client surfaces aligned with server DTOs and reduce drift.
15. Auxiliary packaging
- The embedded SPA is built into the Rust binary, keeping deployment “single image, single volume, no external DB.”
- Dockerfiles provide CPU and hardware-specific images, which materially affect the deployment/runtime boundary because ffmpeg capabilities differ by flavor.
- The documentation site and example flows/plugins are support artifacts, not part of the runtime control plane, but they reflect the intended architecture and operational model.
Main entry points: main.rs transcoderr · api/mod.rs API layer · steps/mod.rs Step registry · client.ts Web API client
Components 20
Core runtime
Primary Rust server/worker and shared API types.
- transcoderr · Main binary: API server, webhook ingress, job runner, and optional remote worker mode.
crates/transcoderr/src/main.rs - API layer · Typed API surface for CRUD, validation, setup, browsing, runs, settings, and workers.
crates/transcoderr/src/api/mod.rs - Webhook ingress · Radarr, Sonarr, Lidarr, and generic webhook endpoints with shared auth and deduping.
crates/transcoderr/src/http/mod.rs - Flow engine · Parses, validates, plans, and executes declarative flows with expression and context handling.
crates/transcoderr/src/flow/mod.rs - Step registry · Built-in step kinds, including plan mutation, ffmpeg execution, filesystem actions, notifications, and webhooks.
crates/transcoderr/src/steps/mod.rs - SQLite store · Embedded database layer for jobs, runs, flows, sources, notifiers, plugins, workers, settings, and events.
crates/transcoderr/src/db/mod.rs - Event bus · Internal live event distribution and browser-facing stream of queue, run, and status updates.
crates/transcoderr/src/bus/mod.rs - Worker runtime · Enrollment, connections, pool, protocol, local execution, and remote dispatch for distributed transcoding.
crates/transcoderr/src/worker/mod.rs - Dispatch · Chooses local or remote execution and coordinates cancellation and path-mapped work transfer.
crates/transcoderr/src/dispatch/mod.rs - Plugin system · Catalogs, installs, runtime loading, subprocess execution, and hot-reloaded plugin step registration.
crates/transcoderr/src/plugins/mod.rs - Notifiers · Discord, ntfy, Telegram, webhook, and Jellyfin delivery triggered from flow runs.
crates/transcoderr/src/notifiers/mod.rs - HW probe · Startup capability detection and semaphore control for NVENC, QSV, VAAPI, VideoToolbox, or CPU fallback.
crates/transcoderr/src/hw/mod.rs - FFmpeg bridge · Runtime integration with ffmpeg and ffprobe for probe, transcode, remux, and verification steps.
crates/transcoderr/src/ffmpeg.rs - ARR support · Browse and reconcile Radarr/Sonarr/Lidarr libraries and source metadata for manual and webhook-driven jobs.
crates/transcoderr/src/arr/mod.rs
Client surfaces
Operator UI, MCP client, and browser-based control surfaces.
- API types · Shared request/response types used by server, web UI, and MCP client.
crates/transcoderr-api-types/src/lib.rs - React UI · Embedded frontend for flows, sources, notifiers, plugins, workers, runs, settings, and manual transcode actions.
web/src/App.tsx - Web API client · Browser client and SSE subscription layer used by the React console.
web/src/api/client.ts - MCP client · Command-line MCP surface that exposes server capabilities to AI tools over stdio.
crates/transcoderr-mcp/src/main.rs
Support assets
Docs, deployment artifacts, and example flows/plugins.
- Marketing site · Public documentation and product site for deployment, flows, plugins, and MCP usage.
site/src/pages/index.astro - Docs and examples · Flow examples, plugin examples, deployment notes, and architecture writeups.
docs/flow-authoring.md
Connections 26
- transcoderr → API layer: serves · Exposes the typed control API and admin routes.
- transcoderr → Webhook ingress: accepts · Receives external webhook events and source-triggered ingress.
- transcoderr → Flow engine: executes · Uses the flow engine to turn triggers into run plans.
- transcoderr → Step registry: runs · Invokes built-in step kinds during a run.
- transcoderr → SQLite store: persists · Stores jobs, runs, settings, plugins, workers, and events.
- transcoderr → Event bus: publishes · Streams live run and queue events to connected clients.
- transcoderr → Worker runtime: coordinates · Manages local and remote worker enrollment and execution.
- transcoderr → Dispatch: routes · Chooses where work executes and how cancellation propagates.
- transcoderr → Plugin system: loads · Installs, syncs, and hot-reloads plugin-provided steps.
- transcoderr → Notifiers: emits · Sends notifications as part of the run lifecycle.
- transcoderr → HW probe: probes · Detects available hardware acceleration at startup.
- transcoderr → FFmpeg bridge: invokes · Calls ffmpeg/ffprobe for media processing and validation.
- transcoderr → ARR support: browses · Supports library browsing and source reconciliation.
- API layer → SQLite store: reads/writes · Performs CRUD and validation against persisted state.
- API layer → Event bus: streams · Feeds live UI state from the event bus.
- Flow engine → Step registry: plans · Builds ordered step sequences and execution context.
- Step registry → FFmpeg bridge: uses · Media steps materialize a single ffmpeg invocation.
- Step registry → Notifiers: calls · Notification steps dispatch outbound deliveries.
- Worker runtime → Dispatch: executes · Remote hosts receive routed jobs and step work.
- React UI → Web API client: uses · Frontend state and actions flow through the browser API client.
- Web API client → API layer: calls · Browser console talks to the backend typed API.
- Web API client → Event bus: subscribes · UI consumes the live SSE stream for progress and status.
- MCP client → API layer: wraps · MCP tools proxy backend capabilities through the same API surface.
- MCP client → API types: shares · Keeps tool contracts aligned with server DTOs.
- React UI → API types: shares · Frontend types align with backend DTOs.
- Marketing site → Docs and examples: publishes · Site content reflects docs, examples, and product positioning.