# cisco/openh264 architecture

> Architecture diagram of the GitHub repository cisco/openh264, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cisco/openh264
- Repository: https://github.com/cisco/openh264

## Overview

1. **What this repository is**
   OpenH264 is a native H.264 codec stack centered on two major runtime products: an **encoder** and a **decoder**. The codebase also ships a **processing/analysis** library, CLI console tools, platform demo apps, a browser/plugin module, and a fairly large test and automation harness.

2. **Top-level runtime architecture**
   The core data flow is split in two directions:
   - **Encode path:** raw **YUV 4:2:0 planar frames** enter the encoder, are optionally preprocessed, then compressed into an **Annex B H.264 byte stream**.
   - **Decode path:** H.264 Annex B bitstreams enter the decoder, are parsed and reconstructed into **YUV 4:2:0 planar output**.
   The encoder and decoder are separate subsystems with their own core logic, plus a smaller shared foundation beneath them.

3. **Public API boundary**
   The stable external interface lives in `codec/api/wels/` (`codec_api.h`, `codec_app_def.h`, `codec_def.h`, `codec_ver.h`). Everything under `codec/encoder`, `codec/decoder`, `codec/common`, and `codec/processing` is implementation detail behind that API boundary. This is the main seam an integrator uses to embed the library into an application.

4. **Encoder subsystem**
   `codec/encoder/` contains the main compression pipeline: mode decision, motion estimation, rate control, slice/NAL packaging, reference management, parameter-set handling, and multi-threaded slice encoding. It supports features like temporal scalability, simulcast, LTR, dynamic bitrate/framerate changes, and arbitrary resolutions. The `plus/` layer adds export/DLL-facing wrappers on top of the core encoder.

5. **Decoder subsystem**
   `codec/decoder/` performs Annex B parsing, CABAC/CAVLC entropy decoding, reference picture management, error concealment, deblocking, and frame reconstruction. The `plus/` layer mirrors the encoder side with a decoder extension/export boundary. The decoder is single-threaded across slices according to the README, but still has its own internal threading abstractions for decoding work coordination.

6. **Processing subsystem**
   `codec/processing/` is a separate library of video pre/post-processing utilities used mainly by the encoder. It includes adaptive quantization, background detection, complexity analysis, denoise, downsample, image rotation, scene-change detection, scroll detection, and VAA calculation. Architecturally, this sits between raw frame ingestion and encoding decisions, influencing compression quality/efficiency rather than producing the bitstream directly.

7. **Shared foundation**
   `codec/common/` provides cross-cutting primitives used by all codec components: threading, task queues, locks, memory alignment, CPU feature detection, trace/logging, utility code, and architecture-specific optimized primitives. This layer is also where most of the platform and CPU dispatch happens, with separate implementations for x86, ARM/NEON, AArch64, MIPS, and LoongArch, plus C/C++ fallbacks.

8. **Platform-specific optimized code**
   Performance-critical inner loops are heavily split by architecture:
   - x86 assembly in `codec/common/x86`, `codec/encoder/core/x86`, `codec/decoder/core/x86`, `codec/processing/src/x86`
   - ARM/NEON and AArch64/NEON in the corresponding `arm` and `arm64` folders
   - MIPS and LoongArch optimized sources where supported  
   This means the architecture is not just “library + app”; it is a **portable core with pluggable CPU-specialized implementations** selected at build/runtime.

9. **Console applications**
   `codec/console/` contains command-line frontends for encoder and decoder use cases. These are thin application wrappers around the library, intended for local testing, benchmarks, and sample usage rather than as the main product. They show how the public API is consumed from a standalone native executable.

10. **Plugin/module integration**
   `module/` contains `gmp-openh264.cpp` and supporting task utilities, indicating a browser/plugin packaging boundary (GMP-style module integration). This is a separate deployment form of the same codec engine, adapted to a host/plugin runtime rather than a normal executable or static/dynamic library.

11. **Test architecture**
   `test/` is organized by subsystem:
   - `test/api` for API-level encoder/decoder behavior
   - `test/encoder`, `test/decoder`, `test/processing`, `test/common` for unit coverage of internal modules
   - `test/encoder_binary_comparison` for bitstream/regression comparisons against SHA1 tables  
   There are also platform-specific test app wrappers under `test/build/` for Android, iOS, Windows, and Windows Phone, showing the project verifies both library behavior and platform packaging.

12. **Automation, fixtures, and regression assets**
   `autotest/` holds performance and unit test scripts across Android and iOS, while `res/` stores the H.264 bitstreams and YUV inputs used by tests and demos. This repository is therefore not just source code: it includes **golden media fixtures** and **scripted cross-platform validation flows** as first-class assets.

13. **Build and packaging boundaries**
   The project supports multiple build systems and targets:
   - Make-based builds via `build/`
   - Meson definitions throughout `codec/*/meson.build` and `test/*/meson.build`
   - Gradle/Android NDK builds
   - Xcode projects for iOS
   - Visual Studio / Windows Phone project files  
   Build scripts also generate version metadata and export definitions, so packaging is an explicit part of the architecture rather than an afterthought.

14. **How to read the overall diagram**
   An accurate architecture diagram should show:
   - **Applications/tests on the outside**
   - **Public API boundary**
   - **Encoder/decoder/processing as separate runtime libraries**
   - **Common utilities and CPU-specific optimized code underneath**
   - **Platform-specific wrappers, demos, and plugin/module integrations around the core**
   - **Media fixtures and automation scripts feeding regression and performance validation**

Main entry points: [`codec/encoder/core`](https://github.com/cisco/openh264/tree/master/codec/encoder/core) (Encoder core), [`codec/api/wels`](https://github.com/cisco/openh264/tree/master/codec/api/wels) (Public API), [`codec/decoder/core`](https://github.com/cisco/openh264/tree/master/codec/decoder/core) (Decoder core), [`build`](https://github.com/cisco/openh264/tree/master/build) (Build system)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime libraries"]
  node_public_api["Public API<br/>stable C API"]
  node_encoder_core["Encoder core<br/>compression pipeline"]
  node_encoder_plus["Encoder plus<br/>export wrapper"]
  node_decoder_core["Decoder core<br/>reconstruction pipeline"]
  node_decoder_plus["Decoder plus<br/>export wrapper"]
  node_processing_lib["Processing lib<br/>video pre/post"]
  node_common_foundation["Common foundation<br/>shared runtime"]
  node_cpu_optimized{{"CPU kernels<br/>arch-specific primitives"}}
end

subgraph group_integration["Integration surfaces"]
  node_console_apps["Console apps<br/>CLI frontends"]
  node_plugin_module["Plugin module<br/>GMP module"]
  node_platform_demos["Platform demos<br/>mobile/desktop wrappers"]
end

subgraph group_validation["Validation assets"]
  node_api_tests["API tests<br/>public API coverage"]
  node_unit_tests["Unit tests<br/>internal coverage"]
  node_binary_regression["Binary regression<br/>bitstream compare"]
  node_automation["Automation<br/>perf/unit scripts"]
  node_fixtures[("Media fixtures<br/>golden assets")]
end

subgraph group_build["Build and packaging"]
  node_build_system["Build system<br/>multi-build setup"]
  node_docs["Docs"]
end

node_public_api -->|"embeds"| node_encoder_core
node_public_api -->|"embeds"| node_decoder_core
node_encoder_plus -->|"wraps"| node_encoder_core
node_decoder_plus -->|"wraps"| node_decoder_core
node_encoder_core -->|"uses"| node_processing_lib
node_encoder_core -->|"depends on"| node_common_foundation
node_decoder_core -->|"depends on"| node_common_foundation
node_processing_lib -->|"depends on"| node_common_foundation
node_common_foundation -->|"dispatches to"| node_cpu_optimized
node_encoder_core -->|"specializes"| node_cpu_optimized
node_decoder_core -->|"specializes"| node_cpu_optimized
node_processing_lib -->|"specializes"| node_cpu_optimized
node_console_apps -->|"calls"| node_public_api
node_plugin_module -->|"calls"| node_public_api
node_platform_demos -->|"calls"| node_public_api
node_api_tests -->|"verifies"| node_public_api
node_unit_tests -->|"verifies"| node_encoder_core
node_unit_tests -->|"verifies"| node_decoder_core
node_unit_tests -->|"verifies"| node_processing_lib
node_unit_tests -->|"verifies"| node_common_foundation
node_binary_regression -->|"compares against"| node_fixtures
node_automation -->|"feeds"| node_fixtures
node_build_system -->|"builds"| node_encoder_core
node_build_system -->|"builds"| node_decoder_core
node_build_system -->|"builds"| node_processing_lib
node_build_system -->|"builds"| node_console_apps
node_build_system -->|"packages"| node_platform_demos
node_docs -->|"documents"| node_public_api

click node_public_api "https://github.com/cisco/openh264/tree/master/codec/api/wels"
click node_encoder_core "https://github.com/cisco/openh264/tree/master/codec/encoder/core"
click node_encoder_plus "https://github.com/cisco/openh264/tree/master/codec/encoder/plus"
click node_decoder_core "https://github.com/cisco/openh264/tree/master/codec/decoder/core"
click node_decoder_plus "https://github.com/cisco/openh264/tree/master/codec/decoder/plus"
click node_processing_lib "https://github.com/cisco/openh264/tree/master/codec/processing"
click node_common_foundation "https://github.com/cisco/openh264/tree/master/codec/common"
click node_cpu_optimized "https://github.com/cisco/openh264/tree/master/codec/common/x86"
click node_console_apps "https://github.com/cisco/openh264/tree/master/codec/console"
click node_plugin_module "https://github.com/cisco/openh264/tree/master/module"
click node_platform_demos "https://github.com/cisco/openh264/tree/master/codec/build"
click node_api_tests "https://github.com/cisco/openh264/tree/master/test/api"
click node_unit_tests "https://github.com/cisco/openh264/tree/master/test"
click node_binary_regression "https://github.com/cisco/openh264/tree/master/test/encoder_binary_comparison"
click node_automation "https://github.com/cisco/openh264/tree/master/autotest"
click node_fixtures "https://github.com/cisco/openh264/tree/master/res"
click node_build_system "https://github.com/cisco/openh264/tree/master/build"
click node_docs "https://github.com/cisco/openh264/tree/master/docs/doxygen"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_public_api,node_encoder_core,node_encoder_plus,node_decoder_core,node_decoder_plus,node_processing_lib,node_common_foundation,node_cpu_optimized toneBlue
class node_console_apps,node_plugin_module,node_platform_demos toneAmber
class node_api_tests,node_unit_tests,node_binary_regression,node_automation,node_fixtures toneMint
class node_build_system,node_docs toneRose
```

## Components

### Runtime libraries

Core codec subsystems and shared foundation

- [Public API](https://github.com/cisco/openh264/tree/master/codec/api/wels): stable C API · External embedding boundary for encoder and decoder clients. · `codec/api/wels`
- [Encoder core](https://github.com/cisco/openh264/tree/master/codec/encoder/core): compression pipeline · Main H.264 encoding path: analysis, mode decision, motion estimation, rate control, slice/NAL packaging, and threading. · `codec/encoder/core`
- [Encoder plus](https://github.com/cisco/openh264/tree/master/codec/encoder/plus): export wrapper · DLL-facing and extension layer above the encoder core. · `codec/encoder/plus`
- [Decoder core](https://github.com/cisco/openh264/tree/master/codec/decoder/core): reconstruction pipeline · Annex B parsing, entropy decoding, reference management, concealment, deblocking, and frame reconstruction. · `codec/decoder/core`
- [Decoder plus](https://github.com/cisco/openh264/tree/master/codec/decoder/plus): export wrapper · Decoder extension and export boundary above the decoder core. · `codec/decoder/plus`
- [Processing lib](https://github.com/cisco/openh264/tree/master/codec/processing): video pre/post · Adaptive quantization, denoise, downsample, rotation, scene/scroll detection, and VAA calculations that influence encoding decisions. · `codec/processing`
- [Common foundation](https://github.com/cisco/openh264/tree/master/codec/common): shared runtime · Threading, queues, locks, memory alignment, CPU detection, tracing, and shared utilities. · `codec/common`
- [CPU kernels](https://github.com/cisco/openh264/tree/master/codec/common/x86): arch-specific primitives · Performance-critical implementations split across x86, ARM/NEON, AArch64, MIPS, and LoongArch. · `codec/common/x86`

### Integration surfaces

Console apps, plugin module, and platform demo wrappers

- [Console apps](https://github.com/cisco/openh264/tree/master/codec/console): CLI frontends · Standalone encoder and decoder command-line wrappers for testing and sample usage. · `codec/console`
- [Plugin module](https://github.com/cisco/openh264/tree/master/module): GMP module · Browser/plugin packaging boundary for the codec engine. · `module`
- [Platform demos](https://github.com/cisco/openh264/tree/master/codec/build): mobile/desktop wrappers · Android, iOS, Windows, and Windows Phone demo and wrapper projects around the codec stack. · `codec/build`

### Validation assets

Tests, automation, and media fixtures

- [API tests](https://github.com/cisco/openh264/tree/master/test/api): public API coverage · Encoder/decoder behavior tests focused on the stable external interface. · `test/api`
- [Unit tests](https://github.com/cisco/openh264/tree/master/test): internal coverage · Subsystem tests for encoder, decoder, processing, and common modules. · `test`
- [Binary regression](https://github.com/cisco/openh264/tree/master/test/encoder_binary_comparison): bitstream compare · SHA1-based bitstream comparison harness and golden-case scripts. · `test/encoder_binary_comparison`
- [Automation](https://github.com/cisco/openh264/tree/master/autotest): perf/unit scripts · Cross-platform scripted runs for performance and unit validation. · `autotest`
- [Media fixtures](https://github.com/cisco/openh264/tree/master/res): golden assets · Reference H.264 bitstreams and YUV inputs used by tests and demos. · `res`

### Build and packaging

Build systems and platform project files

- [Build system](https://github.com/cisco/openh264/tree/master/build): multi-build setup · Make, Meson, Gradle, NDK, Xcode, and Visual Studio build definitions plus version/package metadata. · `build`
- [Docs](https://github.com/cisco/openh264/tree/master/docs/doxygen): developer docs · Doxygen and usage documentation for the public encoder and decoder interfaces. · `docs/doxygen`

## Connections

- Public API → Encoder core: embeds · Consumers instantiate and drive the encoder through the stable API.
- Public API → Decoder core: embeds · Consumers instantiate and drive the decoder through the stable API.
- Encoder plus → Encoder core: wraps · Extension/export layer forwards into the encoding engine.
- Decoder plus → Decoder core: wraps · Extension/export layer forwards into the decoding engine.
- Encoder core → Processing lib: uses · Encoding consults preprocessing and analysis utilities before compression.
- Encoder core → Common foundation: depends on · Encoder shares threading, memory, logging, and utilities.
- Decoder core → Common foundation: depends on · Decoder shares threading, memory, logging, and utilities.
- Processing lib → Common foundation: depends on · Processing primitives reuse shared infrastructure.
- Common foundation → CPU kernels: dispatches to · Shared code selects architecture-specific implementations at build/runtime.
- Encoder core → CPU kernels: specializes · Encoder inner loops use CPU-tuned kernels where available.
- Decoder core → CPU kernels: specializes · Decoder inner loops use CPU-tuned kernels where available.
- Processing lib → CPU kernels: specializes · Processing filters and metrics use CPU-tuned kernels where available.
- Console apps → Public API: calls · CLI tools are thin consumers of the public codec API.
- Plugin module → Public API: calls · Plugin packaging integrates the same public codec API into host runtimes.
- Platform demos → Public API: calls · Platform sample apps exercise the library through its public interface.
- API tests → Public API: verifies · API tests validate expected external behavior.
- Unit tests → Encoder core: verifies · Encoder-focused unit coverage targets core internal modules.
- Unit tests → Decoder core: verifies · Decoder-focused unit coverage targets core internal modules.
- Unit tests → Processing lib: verifies · Processing tests exercise the analysis and filter subsystem.
- Unit tests → Common foundation: verifies · Common tests cover shared threading and utility code.
- Binary regression → Media fixtures: compares against · Golden media and SHA1 tables provide regression baselines.
- Automation → Media fixtures: feeds · Scripts run against stored media assets for performance and regression.
- Build system → Encoder core: builds · Build definitions assemble the encoder runtime.
- Build system → Decoder core: builds · Build definitions assemble the decoder runtime.
- Build system → Processing lib: builds · Build definitions assemble the processing library.
- Build system → Console apps: builds · Build definitions assemble the CLI frontends.
- Build system → Platform demos: packages · Platform project files package demos and wrappers.
- Docs → Public API: documents · Documentation centers on the stable API surface.