How deepch/vdk works
Project type and purpose
• This is a Go‐based library suite (“VDK”) for building streaming services.
• It provides demuxers/muxers for various container formats (FLV, MP4, MKV, TS, HLS, RTMP, RTSP, WebRTC, etc.), parsers for audio/video codecs (H.264, H.265, AAC, Opus, MJPEG), packet queuing, pub/sub messaging, and optional FFmpeg integration via cgo.
• Example subdirectories show typical usage: a basic test harness and a transcoder sample.
File structure
• Top‐level modules:
– av: core audio/video abstractions, conversion utilities, packet queue, timeline, pub/sub and high-level transcode logic
– codec: codec‐specific parsers implementing low‐level bitstream reading
– format: demuxers and muxers organized by container type (subdirs for flv, fmp4, mp4, mkv, ts, rtsp, rtmp, hls, raw, dvrip, nvr, mse, etc.)
– cgo/ffmpeg: wrappers around FFmpeg C libraries for hardware/software encode/decode
– utils: common bit‐IO utilities, buffer readers/writers
– example: sample applications demonstrating library usage
• Build and CI: go.mod/go.sum, GitHub Actions workflow for CodeQL.
• No single “server” or “frontend/backend”—it’s a modular toolkit.
README
• Very brief: “A set of libraries for building streaming services.”
Components
a. Main components
1. Input Sources: file, network streams (RTMP/RTSP), raw buffers
2. Format Demuxers: modules in format/* that split containers into elementary streams
3. Codec Parsers: modules in codec/* that parse video/audio bitstreams and produce frames or NAL units
4. Core AV Pipeline (av/*):
– Packet queue (pktque) buffers and orders packets
– Timeline synchronizes audio/video
– Conversion utilities (avutil, avconv) adjust formats or sample rates
– Pub/Sub (av/pubsub) to decouple producers/consumers
– Transcode logic (av/transcode) orchestrates encode/decode
5. FFmpeg Integration (cgo/ffmpeg): optional path for hardware/software encode/decode, invoked by av/transcode
6. Format Muxers and Streaming Protocols: format/* muxers that package AV packets back into containers or protocol streams (HLS, DASH fragments, RTMP/RTSP outputs, MP4 files, etc.)
7. Utilities (utils/*): shared bit‐level IO, helper functions supporting both codec parsers and format code
8. Example Applications: example/transcoder, example/test
b. Relationships and interactions
• Parsed frames/packets flow → Core AV Pipeline → (either) FFmpeg or pure-Go processing
• Processed packets flow → Format Muxers → Output sinks (disk, network, HLS segments, etc.)
• Pub/Sub channels decouple producers and consumers within the pipeline
c. Architectural patterns and design principles
• Modular, pluggable components; each format and codec lives in its own package
• Pipeline pattern: discrete stages (demux → parse → queue → convert → mux) connected by channels/queues
• Optional native code integration via well-defined cgo interface
• Separation of concerns: parsing, packet management, transcoding orchestration, and IO are distinct
d. Key technologies/frameworks
• Go modules for dependency management
• cgo for FFmpeg bindings
• Net/IO packages for RTMP/RTSP servers & clients
• Standard container format handling (ISO BMFF, MPEG-TS, FLV, AMF0, EBML)