How boul2gom/yt-dlp works
Project Type and Purpose
• This is a Rust library crate (“yt-dlp”) that provides an asynchronous wrapper around the external command-line tools yt-dlp and ffmpeg.
• Its purpose is to automatically install/manage those binaries, fetch video/audio metadata, download streams (with a download manager, priority, progress, cancellation), optionally cache files and thumbnails, and post-process/merge streams while injecting metadata.
Main Components
a. Public API Layer
• Youtube builder and façade functions (in lib.rs)
• High-level methods: with_new_binaries(), new(), download_…(), fetch_video_infos(), combine_audio_and_video(), etc.
b. Fetcher Subsystem
• fetcher::deps – LibraryInstaller, Libraries for auto-installing yt-dlp and ffmpeg
• fetcher::download_manager – orchestrates segmented/parallel downloads, priorities, retries, buffering
• fetcher::streams – handles different stream types (audio/video)
• fetcher::thumbnail – downloads thumbnails
c. Executor & Process Management
• executor module – spawns and manages external processes (yt-dlp, ffmpeg), captures stdout/JSON output
d. Caching Layer
• cache module – metadata, thumbnail, and file caching
e. Metadata Injection
• metadata module – attaches tags/technical info to files, applies optimized logic for combined vs. standalone files
f. Data Models
• model – types for VideoInfo, Format, FormatSelector, Thumbnail, Caption, Quality & CodecPreference enums
g. Utilities
• utils::file_system – path, temp dirs, I/O helpers
• utils::platform – platform detection
• utils::serde (in model/utils) – JSON (de)serialization of yt-dlp output
h. Error & Tracing
• error.rs – unified error types
• Optional tracing feature integration throughout modules
Relationships and Data Flow
1. Public API → Fetcher::deps → downloads/updates yt-dlp & ffmpeg binaries to disk
2. Public API → Fetcher → upon download request:
• Invokes Executor to run yt-dlp --dump-json → Model layer parses JSON via serde → yields VideoInfo
• Optionally, DownloadManager schedules segments → Executor spawns multiple yt-dlp or HTTP streams via reqwest
• Streams saved to disk (via utils::file_system) → optional Cache stores thumbnails/metadata
• If combining: Executor runs ffmpeg to mux audio+video → Metadata attaches tags
3. Cache sits alongside file-system I/O to avoid re-fetching
4. Errors bubble up from Executor/Fetchers into Public API’s Result
Architectural Patterns & Principles
• Modular layering: separation between Process Execution (Executor), Business Logic (Fetcher), Data Models, Caching, and Public API
• Asynchronous “actor‐like” fetcher & download_manager interacting over futures/async tasks (Tokio runtime)
• Feature‐flag gating: optional caching, tracing, rustls for TLS
• Externalized tools: treat yt-dlp/ffmpeg as “pluggable” services
Detailed Component Breakdown (
• Public API (Youtube, LibraryInstaller)
• Fetcher:
• deps (binaries installer)
• download_manager
• streams & thumbnail
• Executor (process runner)
• Cache (metadata, file, thumbnail)
• Metadata (tag injector)
• Model (VideoInfo, Format, Quality enums)
• Utils (file_system, platform, serde)
• External Services:
• yt-dlp binary
• ffmpeg binary
• HTTP/reqwest