# belledonnecommunications/mediastreamer2 architecture

> Architecture diagram of the GitHub repository belledonnecommunications/mediastreamer2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/belledonnecommunications/mediastreamer2
- Repository: https://github.com/belledonnecommunications/mediastreamer2

## Overview

##### Project type and purpose
- This is an open-source C/C++ library (Mediastreamer2) providing a modular media streaming engine for VoIP applications (used by Linphone).
- Core purpose: capture, encode/decode, packetize/unpacketize, encrypt/decrypt, render multimedia streams.

##### Top-level architectural components
- Core engine (filter graph): src/base, include/mediastreamer2
- Audio filters: src/audiofilters + codec backends
- Video filters: src/videofilters + format-specific subfolders (H26x, AV1, VP8, OpenGL display, screen sharing, etc.)
- VoIP stack: src/voip (ICE, RTP, SRTP/ZRTP, conferencing, quality control)
- Platform-specific modules: src/android, include/OpenGL, java wrapper (java/src/org/linphone/mediastream), tools/ios, build/*
- External dependencies: bctoolbox (portability layer), oRTP (RTP stack), optional libs (libsrtp, libvpx, libopus, etc.)
- Test suite: tester/ (unit and integration testers, raw scenarios)
- Tools: tools/ (command-line utilities and example apps)

##### Relationships & data flows
- Each media stream is represented as a directed graph of filters:
- Source filter (capture or network input) → codec filter → packetizer/depktizer → transport filter (oRTP) → network → reverse path → decoder → renderer (audio sink or video display)
- The core engine (MSTicker, MSFilter, MSQueue, MSEventQueue) orchestrates filter execution and data flow scheduling.
- VoIP modules inject signalling (ICE negotiation, SDP) and security (SRTP/ZRTP) filters into the graph.
- Platform layers expose native capture/render APIs (Android OpenSL ES, iOS AVFoundation, ALSA/PulseAudio, X11/GLX).
- The Java Android wrapper delegates calls into native library via JNI.

##### Architectural patterns & extensibility
- Filter graph pattern: pluggable filters connected by input/output pads.
- Factory pattern for filter instantiation (msfactory).
- Abstraction layer for OS/media interfaces (bctoolbox).
- Plugin-like modularity: codecs and transports can be added by dropping new filter implementations.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Subgraphs
    subgraph "Core Engine" 
        CoreEngine["Core Engine"]:::core
        CoreHeaders["Core Headers"]:::core
    end
    subgraph "Filters"
        AudioFilters["Audio Filters Module"]:::filter
        VideoFilters["Video Filters Module"]:::filter
    end
    subgraph "VoIP/Network Module"
        VoIP["VoIP/Network Module"]:::filter
    end
    subgraph "Platform Adaptors"
        AndroidAdapt["Android Adaptors"]:::platform
        iOSCapture["iOS Video Capture"]:::platform
        iOSDisplay["iOS Video Display"]:::platform
        OpenGLAdapt["OpenGL Display Adaptors"]:::platform
    end
    subgraph "Java Wrapper"
        JavaWrapper["Java/Android Wrapper"]:::wrapper
    end
    subgraph "External Libraries"
        BCToolbox["bctoolbox"]:::external
        oRTP["oRTP (RTP Transport)"]:::external
        OptionalLibs["Optional Codecs & SRTP"]:::external
        Crypto["SRTP/ZRTP Crypto Module"]:::external
        UPnP["UPnP IGD Support"]:::external
    end
    subgraph "Factory & Plugins"
        Factory["msfactory (Factory)"]:::core
    end
    subgraph "Tools & Testing"
        Tools["Tools & Examples"]:::test
        TestSuite["Test Suite"]:::test
    end

    %% Data and control flows
    CoreEngine -->|manages| AudioFilters
    CoreEngine -->|manages| VideoFilters
    CoreEngine -->|manages| VoIP
    AudioFilters -->|audio data| CoreEngine
    VideoFilters -->|video data| CoreEngine
    VoIP -->|network events| CoreEngine

    Factory -->|instantiates filters| CoreEngine
    Factory -->|instantiates| AudioFilters
    Factory -->|instantiates| VideoFilters
    Factory -->|instantiates| VoIP

    AndroidAdapt -->|capture/render| CoreEngine
    iOSCapture -->|capture| CoreEngine
    iOSDisplay -->|render| CoreEngine
    OpenGLAdapt -->|render frames| VideoFilters

    JavaWrapper -->|JNI calls| CoreEngine

    BCToolbox -->|abstraction layer| CoreEngine
    oRTP -->|RTP transport| VoIP
    Crypto -->|SRTP/ZRTP| VoIP
    UPnP -->|NAT traversal| VoIP
    OptionalLibs -->|codec plugins| AudioFilters
    OptionalLibs -->|video codecs| VideoFilters

    Tools -->|example graphs| CoreEngine
    TestSuite -->|tests graphs| CoreEngine

    %% Click events
    click CoreEngine "https://github.com/belledonnecommunications/mediastreamer2/tree/master/src/base"
    click CoreHeaders "https://github.com/belledonnecommunications/mediastreamer2/tree/master/include/mediastreamer2"
    click AudioFilters "https://github.com/belledonnecommunications/mediastreamer2/tree/master/src/audiofilters"
    click VideoFilters "https://github.com/belledonnecommunications/mediastreamer2/tree/master/src/videofilters"
    click VoIP "https://github.com/belledonnecommunications/mediastreamer2/tree/master/src/voip"
    click AndroidAdapt "https://github.com/belledonnecommunications/mediastreamer2/tree/master/src/android"
    click iOSCapture "https://github.com/belledonnecommunications/mediastreamer2/blob/master/src/videofilters/ioscapture.m"
    click iOSDisplay "https://github.com/belledonnecommunications/mediastreamer2/blob/master/src/videofilters/iosdisplay.mm"
    click OpenGLAdapt "https://github.com/belledonnecommunications/mediastreamer2/tree/master/include/OpenGL"
    click OpenGLAdapt "https://github.com/belledonnecommunications/mediastreamer2/blob/master/src/videofilters/generic_opengl_display.c"
    click JavaWrapper "https://github.com/belledonnecommunications/mediastreamer2/tree/master/java/src/org/linphone/mediastream"
    click Tools "https://github.com/belledonnecommunications/mediastreamer2/tree/master/tools/"
    click TestSuite "https://github.com/belledonnecommunications/mediastreamer2/tree/master/tester/"
    click Factory "https://github.com/belledonnecommunications/mediastreamer2/blob/master/src/base/msfactory.c"
    click Factory "https://github.com/belledonnecommunications/mediastreamer2/blob/master/include/mediastreamer2/msfactory.h"
    click oRTP "https://github.com/belledonnecommunications/mediastreamer2/blob/master/src/otherfilters/msrtp.c"
    click oRTP "https://github.com/belledonnecommunications/mediastreamer2/blob/master/include/mediastreamer2/msrtp.h"
    click Crypto "https://github.com/belledonnecommunications/mediastreamer2/blob/master/src/crypto/ms_srtp.cpp"
    click Crypto "https://github.com/belledonnecommunications/mediastreamer2/blob/master/src/crypto/dtls_srtp.cpp"
    click Crypto "https://github.com/belledonnecommunications/mediastreamer2/blob/master/include/mediastreamer2/ms_srtp.h"
    click UPnP "https://github.com/belledonnecommunications/mediastreamer2/tree/master/src/upnp"
    click UPnP "https://github.com/belledonnecommunications/mediastreamer2/blob/master/include/mediastreamer2/upnp_igd.h"

    %% Styles
    classDef core fill:#cce5ff,stroke:#004085,color:#004085
    classDef filter fill:#d4edda,stroke:#155724,color:#155724
    classDef platform fill:#fff3cd,stroke:#856404,color:#856404
    classDef wrapper fill:#e2e3e5,stroke:#6c757d,color:#6c757d
    classDef external fill:#f8f9fa,stroke:#6c757d,color:#6c757d
    classDef test fill:#fdecea,stroke:#721c24,color:#721c24
```
