How riad804/ferrox works
1. This is a Rust workspace centered on one media engine
The repository is organized around a single reusable library crate, core, which implements media parsing, decoding, filtering, encoding, and packaging. Everything else is a frontend or integration layer on top of that engine.
2. core is the architectural heart
core/src contains the actual processing pipeline: codecs, filter graphs, media/frame abstractions, format probing, muxing/demuxing, HLS generation, GPU helpers, SIMD helpers, and WASM bindings. It is designed to be called from multiple entry points rather than from one app.
3. Primary data flow: input bytes/files → normalized media structures → output bytes/files
Media enters as file contents, uploaded blobs, or browser/mobile buffers. It is then:
- probed for format and stream metadata,
- demuxed into tracks/packets,
- decoded into frames or audio samples,
- optionally transformed by filters or filter graphs,
- re-encoded or remuxed into a target container/format.
This pipeline is the central diagram to capture.
4. Codecs are split by media type and by feature boundary
core/src/codecs is subdivided into audio, video, and image/GIF codecs. The README shows a deliberate boundary between:
- pure-Rust implementations, and
- optional C-backed codecs behind feature flags such as vp9, h264, mp3-encode, and opus-encode.
That means the architecture is “mostly pure Rust” with explicit optional native codec adapters.
5. Filters are a separate subsystem from codecs
core/src/filters implements image/video transforms like resize, blur, crop, rotate, overlay, text, brightness/contrast, resample, and volume. Filters operate on decoded media, not on compressed containers. They are used both directly and through filter_graph, which provides a named-pad DAG with FFmpeg-style expressions.
6. Container and streaming support sits above codecs
The demux/mux modules cover WebM/MKV/MP4/IVF/TS/fMP4/HLS-related packaging. HLS is not a separate app; it is a higher-level orchestration layer in core that segments encoded media into playlist + segment outputs. This is the main boundary between “single file processing” and “stream delivery.”
7. There are multiple consumption surfaces over the same core
The same engine is exposed through:
- cli for local command-line workflows,
- service for HTTP-based remote processing,
- mobile for native iOS/Android bindings,
- wasm for browser/JS usage.
These are all thin entry points, not separate implementations of the media pipeline.
8. The CLI is a direct local execution wrapper
cli/src/main.rs provides commands like probe, convert, resize, filter-apply, audio/video extraction, transcode, GIF encode/decode, and HLS segmentation. Architecturally, it is a user-facing orchestrator that takes filesystem inputs and emits files or JSON, delegating all real work to core.
9. The HTTP service is an upload-and-process boundary
service is an Axum-based server with /health, /probe, and /process. Its role is to accept multipart uploads, translate a job description into core operations, and return JSON or media bytes. It is the network boundary of the system and the place where request validation/modeling matters.
10. Mobile and WASM are bindings, not independent engines
mobile uses UniFFI to generate idiomatic Swift/Kotlin APIs around core, while core/src/wasm.rs exposes browser-friendly bindings via wasm-bindgen. Both are adapters that preserve the same underlying Rust processing behavior across platforms.
11. GPU and SIMD are acceleration paths inside the core
core/src/gpu.rs and core/src/simd_ops.rs are performance-specialized paths, not separate subsystems. GPU filters use wgpu and WGSL shaders, while SIMD paths use wide for pixel math. These sit inside the filter/resize execution path and fall back to CPU when unavailable.
12. Testing and fuzzing are intentionally placed around parser/codec boundaries
core/tests covers pipelines, muxing, HLS, WASM API behavior, and filter/audio/video flows. fuzz targets MP3, MP4, PNG, and M3U8 parsers/demuxers, which highlights the most input-sensitive boundaries in the architecture.
13. Build and packaging infrastructure reflect the multi-surface design
The repo includes:
- a multi-stage Dockerfile for shipping either CLI or service,
- GitHub Actions CI,
- scripts for Android and iOS SDK generation,
- docs for migration, mobile, and limitations.
These support distributing the same core engine to server, desktop, mobile, and browser environments.
14. A good diagram should center on core and fan out to adapters
The most accurate architecture view is a hub-and-spoke model: core at the center, with input adapters and output adapters around it. The key internal lanes are:
- media probing/demuxing,
- decoding to frames/samples,
- filtering/filter graphs,
- encoding/muxing/HLS packaging,
- optional acceleration and optional native codec backends.
Main entry points: src/lib.rs Core lib · mod.rs Codecs · main.rs HTTP service · src/lib.rs Mobile bindings
Components 20
Core engine
Reusable media processing library and its internal pipeline
- Core lib · Primary reusable library crate that orchestrates all media processing
core/src/lib.rs - Probe formats · Container and stream format detection plus media capability metadata
core/src/formats.rs - Media model · Shared media, stream, and job data structures
core/src/media.rs - Frames · Normalized frame and sample representations used across the pipeline
core/src/frame.rs - Demux graph · Packet routing and stream extraction from container inputs
core/src/demux_graph.rs - Transcode graph · End-to-end decode, filter, encode, and remux orchestration
core/src/transcode_graph.rs - Filter graph · Named-pad filter DAG with expression-driven media transforms
core/src/filter_graph.rs - Filters · Resize, blur, crop, rotate, drawtext, resample, and volume operations
core/src/filters/mod.rs - Codecs · Media codec family split into audio, video, and still-image codecs
core/src/codecs/mod.rs - Audio codecs · Audio decode/encode implementations including FLAC, MP3, Opus, Vorbis, and WAV
core/src/codecs/audio/mod.rs - Video codecs · Video and container codec helpers including VP8, VP9, H.264, AV1, and mux/demux paths
core/src/codecs/video/mod.rs - HLS · Segmenting and playlist generation for HTTP Live Streaming
core/src/hls.rs - GPU accel · GPU-backed filter and pixel processing path
core/src/gpu.rs - SIMD ops · SIMD-accelerated pixel and sample operations
core/src/simd_ops.rs
Entry points
CLI, service, mobile, and wasm surfaces over the core
- CLI · Command-line wrapper for probing, converting, filtering, transcoding, and HLS output
cli/src/main.rs - HTTP service · Axum-based upload-and-process API with probe and process endpoints
service/src/main.rs - Mobile bindings · UniFFI-based iOS and Android APIs over the core engine
mobile/src/lib.rs - WASM bindings · wasm-bindgen entry point for JavaScript and browser usage
core/src/wasm.rs
Tooling
Build, test, fuzz, and packaging support
- Integration tests · Coverage for audio, video, muxing, HLS, transcode, and WASM behavior
core/tests - Fuzz targets · Input-hardened fuzzing for MP3, MP4, PNG, and M3U8 boundaries
fuzz/src
Connections 23
- Core lib → Probe formats: probes · Identifies container and stream format before processing
- Core lib → Media model: models · Defines shared media/job structures
- Core lib → Frames: normalizes · Represents decoded audio samples and video frames
- Core lib → Demux graph: routes input · Extracts packets and streams from containers
- Core lib → Transcode graph: orchestrates · Coordinates decode, filter, encode, and remux stages
- Core lib → Filter graph: builds · Assembles filter DAG execution
- Core lib → Filters: applies · Runs concrete media transforms
- Core lib → Codecs: uses · Delegates encode/decode to codec families
- Core lib → HLS: packages · Creates HLS playlists and segments
- Core lib → GPU accel: accelerates · Uses GPU path when available
- Core lib → SIMD ops: accelerates · Uses SIMD path when available
- Codecs → Audio codecs: contains · Audio codec implementations and adapters
- Codecs → Video codecs: contains · Video codec and container helpers
- Filters → Filter graph: composes into · Filters are assembled into graph-based pipelines
- CLI → Core lib: calls · Runs local processing jobs through the core engine
- HTTP service → Core lib: dispatches to · Maps HTTP uploads and requests onto core operations
- Mobile bindings → Core lib: binds · Exposes the core through native platform APIs
- WASM bindings → Core lib: binds · Exposes the core in browser/JS environments
- Integration tests → Core lib: verifies · Exercises the main media pipeline and bindings
- Fuzz targets → Demux graph: hardens · Targets parser and demux boundaries
- Fuzz targets → Codecs: hardens · Targets decode boundaries for common formats
- HTTP service → CLI: same core · Different frontends over the same processing engine
- Mobile bindings → WASM bindings: same core · Alternative binding surfaces over shared Rust logic