How openhevc/ffmpeg works
Project Type & Purpose
• Open-source, modular multimedia framework and toolset (FFmpeg) for decoding, encoding, filtering and streaming audio/video/subtitles.
• Consists of a set of C libraries plus command-line tools that sit on top of them.
High-Level Architectural Layers
[External I/O & Devices]
• Files, network protocols, device capture/playback (e.g., ALSA, avfoundation, v4l2)
uses libavformat & libavdevice
↓
[Core Libraries]
• libavformat – container demux/mux, I/O, streaming protocols
• libavcodec – decode/encode codecs
• libavfilter – filter graph engine (audio/video filters)
• libswresample – audio resampling/mixing
• libpostproc – post-processing routines
• libavutil – utility functions (memory, logging, common data structures, threading)
↓
[CLI Tools & Applications]
• ffmpeg – main command-line converter/streamer
• ffplay – minimal media player (uses SDL on top of libs)
• ffprobe – introspection/analysis tool
• Additional small utilities (aviocat, qt-faststart, …)
Data Flow & Component Interactions
• Input sources → libavformat/libavdevice
– Demux container or capture device → elementary streams
• libavcodec
– Decode compressed streams → raw audio/video frames
• libavfilter
– Optionally apply filter graph to raw frames
• libswresample / libswscale
– Resample audio, scale/convert video pixel formats
• libavcodec
– Encode processed raw frames → compressed streams
• libavformat
– Mux encoded streams → output container or network protocol
• Tools invoke above pipeline via public C APIs (e.g., avformat_open_input, avcodec_send_frame, av_filter_graph_process)
Important Patterns & Principles
• Layered, clean C API per library – each lib has its own header and namespace (libavformat, libavcodec, ...)
• Plugin-style registration – codecs, filters, formats auto-registered at startup
• ABI stability – public headers vs. internal headers
• Architecture-specific optimized code in subdirectories (x86, arm, ppc)
• Threading support in libavutil (frame queues, thread messages)
Key Elements Not to Omit
• The filter graph engine (libavfilter) sitting between decode and encode.
• The helper/utility layer (libavutil) used everywhere.
• The separation of demux/mux (libavformat) vs. codecs vs. filters vs. scaling/resampling.
• The CLI frontends’ parsing layer (ffmpeg_opt.c) that configures the pipeline.
• Architecture-specific performance modules in each library.