cisco/openh264

1. What this repository is

OpenH264 is a native H.264 codec stack centered on two major runtime products: an encoder and a decoder. The codebase also ships a processing/analysis library, CLI console tools, platform demo apps, a browser/plugin module, and a fairly large test and automation harness.

2. Top-level runtime architecture

The core data flow is split in two directions:

- Encode path: raw YUV 4:2:0 planar frames enter the encoder, are optionally preprocessed, then compressed into an Annex B H.264 byte stream.

- Decode path: H.264 Annex B bitstreams enter the decoder, are parsed and reconstructed into YUV 4:2:0 planar output.

The encoder and decoder are separate subsystems with their own core logic, plus a smaller shared foundation beneath them.

3. Public API boundary

The stable external interface lives in codec/api/wels/ (codec_api.h, codec_app_def.h, codec_def.h, codec_ver.h). Everything under codec/encoder, codec/decoder, codec/common, and codec/processing is implementation detail behind that API boundary. This is the main seam an integrator uses to embed the library into an application.

4. Encoder subsystem

codec/encoder/ contains the main compression pipeline: mode decision, motion estimation, rate control, slice/NAL packaging, reference management, parameter-set handling, and multi-threaded slice encoding. It supports features like temporal scalability, simulcast, LTR, dynamic bitrate/framerate changes, and arbitrary resolutions. The plus/ layer adds export/DLL-facing wrappers on top of the core encoder.

5. Decoder subsystem

codec/decoder/ performs Annex B parsing, CABAC/CAVLC entropy decoding, reference picture management, error concealment, deblocking, and frame reconstruction. The plus/ layer mirrors the encoder side with a decoder extension/export boundary. The decoder is single-threaded across slices according to the README, but still has its own internal threading abstractions for decoding work coordination.

6. Processing subsystem

codec/processing/ is a separate library of video pre/post-processing utilities used mainly by the encoder. It includes adaptive quantization, background detection, complexity analysis, denoise, downsample, image rotation, scene-change detection, scroll detection, and VAA calculation. Architecturally, this sits between raw frame ingestion and encoding decisions, influencing compression quality/efficiency rather than producing the bitstream directly.

7. Shared foundation

codec/common/ provides cross-cutting primitives used by all codec components: threading, task queues, locks, memory alignment, CPU feature detection, trace/logging, utility code, and architecture-specific optimized primitives. This layer is also where most of the platform and CPU dispatch happens, with separate implementations for x86, ARM/NEON, AArch64, MIPS, and LoongArch, plus C/C++ fallbacks.

8. Platform-specific optimized code

Performance-critical inner loops are heavily split by architecture:

- x86 assembly in codec/common/x86, codec/encoder/core/x86, codec/decoder/core/x86, codec/processing/src/x86

- ARM/NEON and AArch64/NEON in the corresponding arm and arm64 folders

- MIPS and LoongArch optimized sources where supported

This means the architecture is not just “library + app”; it is a portable core with pluggable CPU-specialized implementations selected at build/runtime.

9. Console applications

codec/console/ contains command-line frontends for encoder and decoder use cases. These are thin application wrappers around the library, intended for local testing, benchmarks, and sample usage rather than as the main product. They show how the public API is consumed from a standalone native executable.

10. Plugin/module integration

module/ contains gmp-openh264.cpp and supporting task utilities, indicating a browser/plugin packaging boundary (GMP-style module integration). This is a separate deployment form of the same codec engine, adapted to a host/plugin runtime rather than a normal executable or static/dynamic library.

11. Test architecture

test/ is organized by subsystem:

- test/api for API-level encoder/decoder behavior

- test/encoder, test/decoder, test/processing, test/common for unit coverage of internal modules

- test/encoder_binary_comparison for bitstream/regression comparisons against SHA1 tables

There are also platform-specific test app wrappers under test/build/ for Android, iOS, Windows, and Windows Phone, showing the project verifies both library behavior and platform packaging.

12. Automation, fixtures, and regression assets

autotest/ holds performance and unit test scripts across Android and iOS, while res/ stores the H.264 bitstreams and YUV inputs used by tests and demos. This repository is therefore not just source code: it includes golden media fixtures and scripted cross-platform validation flows as first-class assets.

13. Build and packaging boundaries

The project supports multiple build systems and targets:

- Make-based builds via build/

- Meson definitions throughout codec/*/meson.build and test/*/meson.build

- Gradle/Android NDK builds

- Xcode projects for iOS

- Visual Studio / Windows Phone project files

Build scripts also generate version metadata and export definitions, so packaging is an explicit part of the architecture rather than an afterthought.

14. How to read the overall diagram

An accurate architecture diagram should show:

- Applications/tests on the outside

- Public API boundary

- Encoder/decoder/processing as separate runtime libraries

- Common utilities and CPU-specific optimized code underneath

- Platform-specific wrappers, demos, and plugin/module integrations around the core

- Media fixtures and automation scripts feeding regression and performance validation