How axiscommunications/media-stream-library-js works
1. The "streams" layer – a set of low-level components and pipelines that convert RTP (H.264/AAC or JPEG) or HTTP-based MP4 streams into ISO-BMFF and feed them into a browser’s Media Source Extensions (MSE). This includes various submodules for handling RTSP (parsing, authentication, depayloading, and session management), as well as MP4 muxing/parsing.
2. The "player" layer – which is provided as a React-based player component (with variations such as BasicPlayer and an advanced Player) intended to present the video state using dedicated React components (controls, feedback, image overlays, etc.). There are examples for both React (example-player-react) and Web Components (example-player-webcomponent).
3. The "overlay" module – a React-based library mainly intended for coordinate conversions for drawing SVG overlays. It provides a “Foundation” component for transforming between user and SVG coordinate systems, along with hooks and components like “Liner” and “useDraggable”.
4. Supplementary example applications and test harnesses – many example directories (for overlay, player, streams) demonstrate how end-user applications can integrate with the library. There are also scripts (e.g. using Docker, justfiles, or Vite) to facilitate local testing and CI integration.
5. Supporting infrastructure – GitHub workflows (CI, Docker, dependency updates), ESLint/TypeScript configurations, and other developer tools ensuring code quality and constant integration.
A. Main Components:
• Client (Browser): Represents the end-user environment running the player and overlay components. This includes the React components and web components consuming the streams.
• Media Stream Pipelines: Represent the low-level streaming “engines” contained under the /src/streams directory. This should include separate submodules for:
• RTP stream handling (e.g. RTSP parsing, RTP depayloading)
• MP4 capture and HTTP-MP4 pipeline for low latency
• Utility components for muxing/demuxing and conversion purposes.
• Player Module: Contained under /src/player and the corresponding example applications (example-player-react, example-player-webcomponent). This module handles the presentation of video, extraction of video state, and user interactions.
• Overlay Module: The /src/overlay components that abstract coordinate conversions, dealing with user/SVG coordinate systems. Show its relationship with visual output on top of the video.
• External Camera/RTSP Source: This is the external dependency represented by an Axis IP camera. It provides the original stream and sometimes expects two-way communications over WebSocket.
• Development/Build Tools and CI/CD: Represented by the configuration files (.github/workflows, justfile, esbuild.mjs, etc.), which manage automated testing, building, and deployment.
B. Relationships and Interactions:
• Within the pipelines, indicate sub-components (like RTSP authentication and parsing, RTP depayloading, MP4 muxing) that feed into the “Player Module.”
• The “Player Module” then provides media data to the Client (Browser) environment, which uses MSE to render the video.
• The “Overlay Module” sits as an additional layer on top of the video feed, taking cues from user-defined coordinate systems and transformation matrices to place SVG or other overlay components.
• That examples and test harnesses serve as demonstrative “front-end” clients that directly integrate with these technologies.
C. Architectural Patterns and Principles:
• Modular design: All concerns (stream processing, playback, overlay) are separated into distinct modules (directories) to allow independent development and potential reuse.
• Pipeline pattern: The streaming components align as a pipeline (or chain of responsibility) where each component processes or transforms media data.
• React-based view composition: The player and overlay layers follow a component-based architecture characteristic of React applications.
• External dependency integration: The design accounts for two-way communication with the RTSP server via WebSocket, which is crucial for playback synchronization.
D. Relevant Technologies / Frameworks:
• JavaScript/TypeScript (core language for all modules, as indicated by .ts and .tsx files).
• React (for the player and overlay).
• Web Streams API and Media Source Extensions (for dynamic stream feeding into browsers).
• Node.js and Yarn (for development and package management).
• Vite (for development server in the example React applications).
• Docker (for the gst-rtsp-launch integration and deployment testing).
• External integrations (Axis camera, WebSocket communications, HTTP-MP4 pipelines) with concise labels.
• Optionally, include a “Developer/CI/CD” block that shows the integration of GitHub Workflows, justfile, and various configuration files.
F. Tailoring to This Project Type:
• Since this is a media streaming library with both full-stack (client-server interaction) and specialized front-end rendering (React-based player and overlay) capabilities, emphasize the distinction between data acquisition (stream ingestion & pipeline processing) and stateful UI rendering (player and overlay).
• To detail how the modularity (with clearly separated subdirectories and examples) supports extensibility in building custom player or overlay solutions.
This level of granularity will capture the essential architectural elements and serve as an accurate system design blueprint for the Media Stream Library JS project.