# riad804/ferrox architecture

> Architecture diagram of the GitHub repository riad804/ferrox, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/riad804/ferrox
- Repository: https://github.com/riad804/ferrox

## Overview

1. **This is a Rust workspace centered on one media engine**
   The repository is organized around a single reusable library crate, `core`, which implements media parsing, decoding, filtering, encoding, and packaging. Everything else is a frontend or integration layer on top of that engine.

2. **`core` is the architectural heart**
   `core/src` contains the actual processing pipeline: codecs, filter graphs, media/frame abstractions, format probing, muxing/demuxing, HLS generation, GPU helpers, SIMD helpers, and WASM bindings. It is designed to be called from multiple entry points rather than from one app.

3. **Primary data flow: input bytes/files → normalized media structures → output bytes/files**
   Media enters as file contents, uploaded blobs, or browser/mobile buffers. It is then:
   - probed for format and stream metadata,
   - demuxed into tracks/packets,
   - decoded into frames or audio samples,
   - optionally transformed by filters or filter graphs,
   - re-encoded or remuxed into a target container/format.
   This pipeline is the central diagram to capture.

4. **Codecs are split by media type and by feature boundary**
   `core/src/codecs` is subdivided into audio, video, and image/GIF codecs. The README shows a deliberate boundary between:
   - pure-Rust implementations, and
   - optional C-backed codecs behind feature flags such as `vp9`, `h264`, `mp3-encode`, and `opus-encode`.
   That means the architecture is “mostly pure Rust” with explicit optional native codec adapters.

5. **Filters are a separate subsystem from codecs**
   `core/src/filters` implements image/video transforms like resize, blur, crop, rotate, overlay, text, brightness/contrast, resample, and volume. Filters operate on decoded media, not on compressed containers. They are used both directly and through `filter_graph`, which provides a named-pad DAG with FFmpeg-style expressions.

6. **Container and streaming support sits above codecs**
   The demux/mux modules cover WebM/MKV/MP4/IVF/TS/fMP4/HLS-related packaging. HLS is not a separate app; it is a higher-level orchestration layer in `core` that segments encoded media into playlist + segment outputs. This is the main boundary between “single file processing” and “stream delivery.”

7. **There are multiple consumption surfaces over the same core**
   The same engine is exposed through:
   - `cli` for local command-line workflows,
   - `service` for HTTP-based remote processing,
   - `mobile` for native iOS/Android bindings,
   - `wasm` for browser/JS usage.
   These are all thin entry points, not separate implementations of the media pipeline.

8. **The CLI is a direct local execution wrapper**
   `cli/src/main.rs` provides commands like probe, convert, resize, filter-apply, audio/video extraction, transcode, GIF encode/decode, and HLS segmentation. Architecturally, it is a user-facing orchestrator that takes filesystem inputs and emits files or JSON, delegating all real work to `core`.

9. **The HTTP service is an upload-and-process boundary**
   `service` is an Axum-based server with `/health`, `/probe`, and `/process`. Its role is to accept multipart uploads, translate a job description into core operations, and return JSON or media bytes. It is the network boundary of the system and the place where request validation/modeling matters.

10. **Mobile and WASM are bindings, not independent engines**
   `mobile` uses UniFFI to generate idiomatic Swift/Kotlin APIs around `core`, while `core/src/wasm.rs` exposes browser-friendly bindings via `wasm-bindgen`. Both are adapters that preserve the same underlying Rust processing behavior across platforms.

11. **GPU and SIMD are acceleration paths inside the core**
   `core/src/gpu.rs` and `core/src/simd_ops.rs` are performance-specialized paths, not separate subsystems. GPU filters use `wgpu` and WGSL shaders, while SIMD paths use `wide` for pixel math. These sit inside the filter/resize execution path and fall back to CPU when unavailable.

12. **Testing and fuzzing are intentionally placed around parser/codec boundaries**
   `core/tests` covers pipelines, muxing, HLS, WASM API behavior, and filter/audio/video flows. `fuzz` targets MP3, MP4, PNG, and M3U8 parsers/demuxers, which highlights the most input-sensitive boundaries in the architecture.

13. **Build and packaging infrastructure reflect the multi-surface design**
   The repo includes:
   - a multi-stage `Dockerfile` for shipping either CLI or service,
   - GitHub Actions CI,
   - scripts for Android and iOS SDK generation,
   - docs for migration, mobile, and limitations.
   These support distributing the same core engine to server, desktop, mobile, and browser environments.

14. **A good diagram should center on `core` and fan out to adapters**
   The most accurate architecture view is a hub-and-spoke model: `core` at the center, with input adapters and output adapters around it. The key internal lanes are:
   - media probing/demuxing,
   - decoding to frames/samples,
   - filtering/filter graphs,
   - encoding/muxing/HLS packaging,
   - optional acceleration and optional native codec backends.

Main entry points: [`core/src/lib.rs`](https://github.com/riad804/ferrox/blob/main/core/src/lib.rs) (Core lib), [`core/src/codecs/mod.rs`](https://github.com/riad804/ferrox/blob/main/core/src/codecs/mod.rs) (Codecs), [`service/src/main.rs`](https://github.com/riad804/ferrox/blob/main/service/src/main.rs) (HTTP service), [`mobile/src/lib.rs`](https://github.com/riad804/ferrox/blob/main/mobile/src/lib.rs) (Mobile bindings)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core engine"]
  node_core_lib["Core lib<br/>media engine<br/>[lib.rs]"]
  node_formats["Probe formats<br/>metadata/probing<br/>[formats.rs]"]
  node_media["Media model<br/>abstractions<br/>[media.rs]"]
  node_frame["Frames<br/>[frame.rs]"]
  node_demux_graph["Demux graph<br/>packet pipeline<br/>[demux_graph.rs]"]
  node_transcode_graph["Transcode graph<br/>media pipeline<br/>[transcode_graph.rs]"]
  node_filter_graph["Filter graph<br/>DAG filters<br/>[filter_graph.rs]"]
  node_filters["Filters<br/>transforms<br/>[mod.rs]"]
  node_codecs["Codecs<br/>encode/decode<br/>[mod.rs]"]
  node_audio_codecs["Audio codecs<br/>audio codec set<br/>[mod.rs]"]
  node_video_codecs["Video codecs<br/>video codec set<br/>[mod.rs]"]
  node_hls["HLS<br/>stream packaging<br/>[hls.rs]"]
  node_gpu["GPU accel<br/>wgpu shaders<br/>[gpu.rs]"]
  node_simd["SIMD ops<br/>vector math<br/>[simd_ops.rs]"]
end

subgraph group_adapters["Entry points"]
  node_cli["CLI<br/>local frontend<br/>[main.rs]"]
  node_service["HTTP service<br/>server frontend<br/>[main.rs]"]
  node_mobile["Mobile bindings<br/>native bindings<br/>[lib.rs]"]
  node_wasm["WASM bindings<br/>browser bindings<br/>[wasm.rs]"]
end

subgraph group_tooling["Tooling"]
  node_tests["Integration tests<br/>pipeline tests"]
  node_fuzz["Fuzz targets<br/>parser fuzzing"]
end

node_core_lib -->|"probes"| node_formats
node_core_lib -->|"models"| node_media
node_core_lib -->|"normalizes"| node_frame
node_core_lib -->|"routes input"| node_demux_graph
node_core_lib -->|"orchestrates"| node_transcode_graph
node_core_lib -->|"builds"| node_filter_graph
node_core_lib -->|"applies"| node_filters
node_core_lib -->|"uses"| node_codecs
node_core_lib -->|"packages"| node_hls
node_core_lib -->|"accelerates"| node_gpu
node_core_lib -->|"accelerates"| node_simd
node_codecs -->|"contains"| node_audio_codecs
node_codecs -->|"contains"| node_video_codecs
node_filters -->|"composes into"| node_filter_graph
node_cli -->|"calls"| node_core_lib
node_service -->|"dispatches to"| node_core_lib
node_mobile -->|"binds"| node_core_lib
node_wasm -->|"binds"| node_core_lib
node_tests -->|"verifies"| node_core_lib
node_fuzz -->|"hardens"| node_demux_graph
node_fuzz -->|"hardens"| node_codecs
node_service -.->|"same core"| node_cli
node_mobile -.->|"same core"| node_wasm

click node_core_lib "https://github.com/riad804/ferrox/blob/main/core/src/lib.rs"
click node_formats "https://github.com/riad804/ferrox/blob/main/core/src/formats.rs"
click node_media "https://github.com/riad804/ferrox/blob/main/core/src/media.rs"
click node_frame "https://github.com/riad804/ferrox/blob/main/core/src/frame.rs"
click node_demux_graph "https://github.com/riad804/ferrox/blob/main/core/src/demux_graph.rs"
click node_transcode_graph "https://github.com/riad804/ferrox/blob/main/core/src/transcode_graph.rs"
click node_filter_graph "https://github.com/riad804/ferrox/blob/main/core/src/filter_graph.rs"
click node_filters "https://github.com/riad804/ferrox/blob/main/core/src/filters/mod.rs"
click node_codecs "https://github.com/riad804/ferrox/blob/main/core/src/codecs/mod.rs"
click node_audio_codecs "https://github.com/riad804/ferrox/blob/main/core/src/codecs/audio/mod.rs"
click node_video_codecs "https://github.com/riad804/ferrox/blob/main/core/src/codecs/video/mod.rs"
click node_hls "https://github.com/riad804/ferrox/blob/main/core/src/hls.rs"
click node_gpu "https://github.com/riad804/ferrox/blob/main/core/src/gpu.rs"
click node_simd "https://github.com/riad804/ferrox/blob/main/core/src/simd_ops.rs"
click node_cli "https://github.com/riad804/ferrox/blob/main/cli/src/main.rs"
click node_service "https://github.com/riad804/ferrox/blob/main/service/src/main.rs"
click node_mobile "https://github.com/riad804/ferrox/blob/main/mobile/src/lib.rs"
click node_wasm "https://github.com/riad804/ferrox/blob/main/core/src/wasm.rs"
click node_tests "https://github.com/riad804/ferrox/tree/main/core/tests"
click node_fuzz "https://github.com/riad804/ferrox/tree/main/fuzz/src"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_core_lib,node_formats,node_media,node_frame,node_demux_graph,node_transcode_graph,node_filter_graph,node_filters,node_codecs,node_audio_codecs,node_video_codecs,node_hls,node_gpu,node_simd toneBlue
class node_cli,node_service,node_mobile,node_wasm toneAmber
class node_tests,node_fuzz toneMint
```

## Components

### Core engine

Reusable media processing library and its internal pipeline

- [Core lib](https://github.com/riad804/ferrox/blob/main/core/src/lib.rs): media engine · Primary reusable library crate that orchestrates all media processing · `core/src/lib.rs`
- [Probe formats](https://github.com/riad804/ferrox/blob/main/core/src/formats.rs): metadata/probing · Container and stream format detection plus media capability metadata · `core/src/formats.rs`
- [Media model](https://github.com/riad804/ferrox/blob/main/core/src/media.rs): abstractions · Shared media, stream, and job data structures · `core/src/media.rs`
- [Frames](https://github.com/riad804/ferrox/blob/main/core/src/frame.rs): pixel/audio frames · Normalized frame and sample representations used across the pipeline · `core/src/frame.rs`
- [Demux graph](https://github.com/riad804/ferrox/blob/main/core/src/demux_graph.rs): packet pipeline · Packet routing and stream extraction from container inputs · `core/src/demux_graph.rs`
- [Transcode graph](https://github.com/riad804/ferrox/blob/main/core/src/transcode_graph.rs): media pipeline · End-to-end decode, filter, encode, and remux orchestration · `core/src/transcode_graph.rs`
- [Filter graph](https://github.com/riad804/ferrox/blob/main/core/src/filter_graph.rs): DAG filters · Named-pad filter DAG with expression-driven media transforms · `core/src/filter_graph.rs`
- [Filters](https://github.com/riad804/ferrox/blob/main/core/src/filters/mod.rs): transforms · Resize, blur, crop, rotate, drawtext, resample, and volume operations · `core/src/filters/mod.rs`
- [Codecs](https://github.com/riad804/ferrox/blob/main/core/src/codecs/mod.rs): encode/decode · Media codec family split into audio, video, and still-image codecs · `core/src/codecs/mod.rs`
- [Audio codecs](https://github.com/riad804/ferrox/blob/main/core/src/codecs/audio/mod.rs): audio codec set · Audio decode/encode implementations including FLAC, MP3, Opus, Vorbis, and WAV · `core/src/codecs/audio/mod.rs`
- [Video codecs](https://github.com/riad804/ferrox/blob/main/core/src/codecs/video/mod.rs): video codec set · Video and container codec helpers including VP8, VP9, H.264, AV1, and mux/demux paths · `core/src/codecs/video/mod.rs`
- [HLS](https://github.com/riad804/ferrox/blob/main/core/src/hls.rs): stream packaging · Segmenting and playlist generation for HTTP Live Streaming · `core/src/hls.rs`
- [GPU accel](https://github.com/riad804/ferrox/blob/main/core/src/gpu.rs): wgpu shaders · GPU-backed filter and pixel processing path · `core/src/gpu.rs`
- [SIMD ops](https://github.com/riad804/ferrox/blob/main/core/src/simd_ops.rs): vector math · SIMD-accelerated pixel and sample operations · `core/src/simd_ops.rs`

### Entry points

CLI, service, mobile, and wasm surfaces over the core

- [CLI](https://github.com/riad804/ferrox/blob/main/cli/src/main.rs): local frontend · Command-line wrapper for probing, converting, filtering, transcoding, and HLS output · `cli/src/main.rs`
- [HTTP service](https://github.com/riad804/ferrox/blob/main/service/src/main.rs): server frontend · Axum-based upload-and-process API with probe and process endpoints · `service/src/main.rs`
- [Mobile bindings](https://github.com/riad804/ferrox/blob/main/mobile/src/lib.rs): native bindings · UniFFI-based iOS and Android APIs over the core engine · `mobile/src/lib.rs`
- [WASM bindings](https://github.com/riad804/ferrox/blob/main/core/src/wasm.rs): browser bindings · wasm-bindgen entry point for JavaScript and browser usage · `core/src/wasm.rs`

### Tooling

Build, test, fuzz, and packaging support

- [Integration tests](https://github.com/riad804/ferrox/tree/main/core/tests): pipeline tests · Coverage for audio, video, muxing, HLS, transcode, and WASM behavior · `core/tests`
- [Fuzz targets](https://github.com/riad804/ferrox/tree/main/fuzz/src): parser fuzzing · Input-hardened fuzzing for MP3, MP4, PNG, and M3U8 boundaries · `fuzz/src`

## Connections

- Core lib → Probe formats: probes · Identifies container and stream format before processing
- Core lib → Media model: models · Defines shared media/job structures
- Core lib → Frames: normalizes · Represents decoded audio samples and video frames
- Core lib → Demux graph: routes input · Extracts packets and streams from containers
- Core lib → Transcode graph: orchestrates · Coordinates decode, filter, encode, and remux stages
- Core lib → Filter graph: builds · Assembles filter DAG execution
- Core lib → Filters: applies · Runs concrete media transforms
- Core lib → Codecs: uses · Delegates encode/decode to codec families
- Core lib → HLS: packages · Creates HLS playlists and segments
- Core lib → GPU accel: accelerates · Uses GPU path when available
- Core lib → SIMD ops: accelerates · Uses SIMD path when available
- Codecs → Audio codecs: contains · Audio codec implementations and adapters
- Codecs → Video codecs: contains · Video codec and container helpers
- Filters → Filter graph: composes into · Filters are assembled into graph-based pipelines
- CLI → Core lib: calls · Runs local processing jobs through the core engine
- HTTP service → Core lib: dispatches to · Maps HTTP uploads and requests onto core operations
- Mobile bindings → Core lib: binds · Exposes the core through native platform APIs
- WASM bindings → Core lib: binds · Exposes the core in browser/JS environments
- Integration tests → Core lib: verifies · Exercises the main media pipeline and bindings
- Fuzz targets → Demux graph: hardens · Targets parser and demux boundaries
- Fuzz targets → Codecs: hardens · Targets decode boundaries for common formats
- HTTP service → CLI: same core · Different frontends over the same processing engine
- Mobile bindings → WASM bindings: same core · Alternative binding surfaces over shared Rust logic