How belledonnecommunications/mediastreamer2 works
Project type and purpose
• This is an open-source C/C++ library (Mediastreamer2) providing a modular media streaming engine for VoIP applications (used by Linphone).
• Core purpose: capture, encode/decode, packetize/unpacketize, encrypt/decrypt, render multimedia streams.
Top-level architectural components
• Core engine (filter graph): src/base, include/mediastreamer2
• Audio filters: src/audiofilters + codec backends
• Video filters: src/videofilters + format-specific subfolders (H26x, AV1, VP8, OpenGL display, screen sharing, etc.)
• VoIP stack: src/voip (ICE, RTP, SRTP/ZRTP, conferencing, quality control)
• Platform-specific modules: src/android, include/OpenGL, java wrapper (java/src/org/linphone/mediastream), tools/ios, build/*
• External dependencies: bctoolbox (portability layer), oRTP (RTP stack), optional libs (libsrtp, libvpx, libopus, etc.)
• Test suite: tester/ (unit and integration testers, raw scenarios)
• Tools: tools/ (command-line utilities and example apps)
Relationships & data flows
• Each media stream is represented as a directed graph of filters:
• Source filter (capture or network input) → codec filter → packetizer/depktizer → transport filter (oRTP) → network → reverse path → decoder → renderer (audio sink or video display)
• The core engine (MSTicker, MSFilter, MSQueue, MSEventQueue) orchestrates filter execution and data flow scheduling.
• VoIP modules inject signalling (ICE negotiation, SDP) and security (SRTP/ZRTP) filters into the graph.
• Platform layers expose native capture/render APIs (Android OpenSL ES, iOS AVFoundation, ALSA/PulseAudio, X11/GLX).
• The Java Android wrapper delegates calls into native library via JNI.
Architectural patterns & extensibility
• Filter graph pattern: pluggable filters connected by input/output pads.
• Factory pattern for filter instantiation (msfactory).
• Abstraction layer for OS/media interfaces (bctoolbox).
• Plugin-like modularity: codecs and transports can be added by dropping new filter implementations.