# deepch/vdk architecture

> Architecture diagram of the GitHub repository deepch/vdk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deepch/vdk
- Repository: https://github.com/deepch/vdk

## Overview

##### Project type and purpose
- This is a Go‐based library suite (“VDK”) for building streaming services.
- It provides demuxers/muxers for various container formats (FLV, MP4, MKV, TS, HLS, RTMP, RTSP, WebRTC, etc.), parsers for audio/video codecs (H.264, H.265, AAC, Opus, MJPEG), packet queuing, pub/sub messaging, and optional FFmpeg integration via cgo.
- Example subdirectories show typical usage: a basic test harness and a transcoder sample.

##### File structure
- Top‐level modules:
– av: core audio/video abstractions, conversion utilities, packet queue, timeline, pub/sub and high-level transcode logic
– codec: codec‐specific parsers implementing low‐level bitstream reading
– format: demuxers and muxers organized by container type (subdirs for flv, fmp4, mp4, mkv, ts, rtsp, rtmp, hls, raw, dvrip, nvr, mse, etc.)
– cgo/ffmpeg: wrappers around FFmpeg C libraries for hardware/software encode/decode
– utils: common bit‐IO utilities, buffer readers/writers
– example: sample applications demonstrating library usage
- Build and CI: go.mod/go.sum, GitHub Actions workflow for CodeQL.
- No single “server” or “frontend/backend”—it’s a modular toolkit.

##### README
- Very brief: “A set of libraries for building streaming services.”

##### Components
a. Main components
1. Input Sources: file, network streams (RTMP/RTSP), raw buffers
2. Format Demuxers: modules in format/* that split containers into elementary streams
3. Codec Parsers: modules in codec/* that parse video/audio bitstreams and produce frames or NAL units
4. Core AV Pipeline (av/*):
– Packet queue (pktque) buffers and orders packets
– Timeline synchronizes audio/video
– Conversion utilities (avutil, avconv) adjust formats or sample rates
– Pub/Sub (av/pubsub) to decouple producers/consumers
– Transcode logic (av/transcode) orchestrates encode/decode
5. FFmpeg Integration (cgo/ffmpeg): optional path for hardware/software encode/decode, invoked by av/transcode
6. Format Muxers and Streaming Protocols: format/* muxers that package AV packets back into containers or protocol streams (HLS, DASH fragments, RTMP/RTSP outputs, MP4 files, etc.)
7. Utilities (utils/*): shared bit‐level IO, helper functions supporting both codec parsers and format code
8. Example Applications: example/transcoder, example/test

b. Relationships and interactions
- Parsed frames/packets flow → Core AV Pipeline → (either) FFmpeg or pure-Go processing
- Processed packets flow → Format Muxers → Output sinks (disk, network, HLS segments, etc.)
- Pub/Sub channels decouple producers and consumers within the pipeline

c. Architectural patterns and design principles
- Modular, pluggable components; each format and codec lives in its own package
- Pipeline pattern: discrete stages (demux → parse → queue → convert → mux) connected by channels/queues
- Optional native code integration via well-defined cgo interface
- Separation of concerns: parsing, packet management, transcoding orchestration, and IO are distinct

d. Key technologies/frameworks
- Go modules for dependency management
- cgo for FFmpeg bindings
- Net/IO packages for RTMP/RTSP servers & clients
- Standard container format handling (ISO BMFF, MPEG-TS, FLV, AMF0, EBML)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Input Sources"
        File["format/raw/muxer.go"]:::input
        RTMP["format/rtmp/rtmp.go"]:::input
        RTSP["format/rtsp/client.go"]:::input
        RTSPv2["format/rtspv2/client.go"]:::input
    end

    subgraph "Format Demuxers"
        FLV["format/flv/flv.go"]:::demux
        MP4D["format/mp4/demuxer.go"]:::demux
        MKVD["format/mkv/demuxer.go"]:::demux
        TSD["format/ts/demuxer.go"]:::demux
        FMP4["format/fmp4/fragment/fragment.go"]:::demux
        NVRD["format/nvr/demuxer.go"]:::demux
        DVRIP["format/dvrip/client.go"]:::demux
        RTSPv2D["format/rtspv2/demuxer.go"]:::demux
    end

    subgraph "Codec Parsers"
        H264["codec/h264parser/parser.go"]:::codec
        H265["codec/h265parser/parser.go"]:::codec
        AAC["codec/aacparser/parser.go"]:::codec
        OPUS["codec/opusparser/opusparser.go"]:::codec
        MJPEG["codec/mjpeg/parser.go"]:::codec
    end

    subgraph "Core AV Pipeline"
        PktQue["av/pktque"]:::core
        AvUtil["av/avutil/avutil.go"]:::core
        AvConv["av/avconv/avconv.go"]:::core
        PubSub["av/pubsub/queue.go"]:::core
        Transcode["av/transcode/transcode.go"]:::core
        AvEntry["av/av.go"]:::core
    end

    subgraph "FFmpeg Integration"
        CgoFF["cgo/ffmpeg/ffmpeg.go"]:::cgo
        CgoAudio["cgo/ffmpeg/audio.go"]:::cgo
        CgoVideo["cgo/ffmpeg/video.go"]:::cgo
        CgoH["cgo/ffmpeg/ffmpeg.h"]:::cgo
    end

    subgraph "Format Muxers & Streaming Protocols"
        HLS["format/hls/muxer.go"]:::mux
        MP4M["format/mp4/muxer.go"]:::mux
        MP4F["format/mp4f/muxer.go"]:::mux
        MP4M2["format/mp4m/muxer.go"]:::mux
        TSM["format/ts/muxer.go"]:::mux
        MSE["format/mse/muxer.go"]:::mux
        NVRM["format/nvr/muxer.go"]:::mux
    end

    subgraph "Utilities"
        Bits["utils/bits"]:::util
    end

    subgraph "Example Applications"
        Test["example/test/main.go"]:::example
        TransApp["example/transcoder/main.go"]:::example
    end

    %% Data Flow
    File --> FLV
    File --> MP4D
    RTMP --> FLV
    RTSP --> TSD
    RTSPv2 --> RTSPv2D

    FLV --> H264
    MP4D --> H264
    MKVD --> H265
    TSD --> AAC
    FMP4 --> AAC
    NVRD --> H264
    DVRIP --> MJPEG
    RTSPv2D --> OPUS

    H264 --> PktQue
    H265 --> PktQue
    AAC --> PktQue
    OPUS --> PktQue
    MJPEG --> PktQue

    PktQue --> PubSub
    PubSub --> Transcode
    Transcode --> AvUtil
    Transcode --> AvConv
    Transcode --> CgoFF
    CgoFF --> CgoAudio
    CgoFF --> CgoVideo

    Transcode --> HLS
    Transcode --> MP4M
    Transcode --> MP4F
    Transcode --> MP4M2
    Transcode --> TSM
    Transcode --> MSE
    Transcode --> NVRM

    Test --> AvEntry
    TransApp --> AvEntry
    AvEntry --> File

    %% Click Events
    click File "https://github.com/deepch/vdk/blob/master/format/raw/muxer.go"
    click RTMP "https://github.com/deepch/vdk/blob/master/format/rtmp/rtmp.go"
    click RTSP "https://github.com/deepch/vdk/blob/master/format/rtsp/client.go"
    click RTSPv2 "https://github.com/deepch/vdk/blob/master/format/rtspv2/client.go"
    click FLV "https://github.com/deepch/vdk/blob/master/format/flv/flv.go"
    click MP4D "https://github.com/deepch/vdk/blob/master/format/mp4/demuxer.go"
    click MKVD "https://github.com/deepch/vdk/blob/master/format/mkv/demuxer.go"
    click TSD "https://github.com/deepch/vdk/blob/master/format/ts/demuxer.go"
    click FMP4 "https://github.com/deepch/vdk/blob/master/format/fmp4/fragment/fragment.go"
    click NVRD "https://github.com/deepch/vdk/blob/master/format/nvr/demuxer.go"
    click DVRIP "https://github.com/deepch/vdk/blob/master/format/dvrip/client.go"
    click RTSPv2D "https://github.com/deepch/vdk/blob/master/format/rtspv2/demuxer.go"
    click H264 "https://github.com/deepch/vdk/blob/master/codec/h264parser/parser.go"
    click H265 "https://github.com/deepch/vdk/blob/master/codec/h265parser/parser.go"
    click AAC "https://github.com/deepch/vdk/blob/master/codec/aacparser/parser.go"
    click OPUS "https://github.com/deepch/vdk/blob/master/codec/opusparser/opusparser.go"
    click MJPEG "https://github.com/deepch/vdk/blob/master/codec/mjpeg/parser.go"
    click PktQue "https://github.com/deepch/vdk/blob/master/av/pktque/buf.go"
    click AvUtil "https://github.com/deepch/vdk/blob/master/av/avutil/avutil.go"
    click AvConv "https://github.com/deepch/vdk/blob/master/av/avconv/avconv.go"
    click PubSub "https://github.com/deepch/vdk/blob/master/av/pubsub/queue.go"
    click Transcode "https://github.com/deepch/vdk/blob/master/av/transcode/transcode.go"
    click AvEntry "https://github.com/deepch/vdk/blob/master/av/av.go"
    click CgoFF "https://github.com/deepch/vdk/blob/master/cgo/ffmpeg/ffmpeg.go"
    click CgoAudio "https://github.com/deepch/vdk/blob/master/cgo/ffmpeg/audio.go"
    click CgoVideo "https://github.com/deepch/vdk/blob/master/cgo/ffmpeg/video.go"
    click CgoH "https://github.com/deepch/vdk/blob/master/cgo/ffmpeg/ffmpeg.h"
    click HLS "https://github.com/deepch/vdk/blob/master/format/hls/muxer.go"
    click MP4M "https://github.com/deepch/vdk/blob/master/format/mp4/muxer.go"
    click MP4F "https://github.com/deepch/vdk/blob/master/format/mp4f/muxer.go"
    click MP4M2 "https://github.com/deepch/vdk/blob/master/format/mp4m/muxer.go"
    click TSM "https://github.com/deepch/vdk/blob/master/format/ts/muxer.go"
    click MSE "https://github.com/deepch/vdk/blob/master/format/mse/muxer.go"
    click NVRM "https://github.com/deepch/vdk/blob/master/format/nvr/muxer.go"
    click Bits "https://github.com/deepch/vdk/tree/master/utils/bits"
    click Test "https://github.com/deepch/vdk/blob/master/example/test/main.go"
    click TransApp "https://github.com/deepch/vdk/blob/master/example/transcoder/main.go"

    %% Styles
    classDef input fill:#FFEDCC,stroke:#B35900;
    classDef demux fill:#A2D2FF,stroke:#1D3557;
    classDef codec fill:#FFC8DD,stroke:#6A040F;
    classDef core fill:#BDE0FE,stroke:#003566;
    classDef cgo fill:#E0C3FC,stroke:#541388;
    classDef mux fill:#CFFDE1,stroke:#1A5F20;
    classDef util fill:#D3D3D3,stroke:#404040;
    classDef example fill:#FEE440,stroke:#826B0A;
```
