# boul2gom/yt-dlp architecture

> Architecture diagram of the GitHub repository boul2gom/yt-dlp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/boul2gom/yt-dlp
- Repository: https://github.com/boul2gom/yt-dlp

## Overview

##### Project Type and Purpose
- This is a Rust library crate (“yt-dlp”) that provides an asynchronous wrapper around the external command-line tools `yt-dlp` and `ffmpeg`.
- Its purpose is to automatically install/manage those binaries, fetch video/audio metadata, download streams (with a download manager, priority, progress, cancellation), optionally cache files and thumbnails, and post-process/merge streams while injecting metadata.

##### Main Components
a. Public API Layer
- `Youtube` builder and façade functions (in lib.rs)
- High-level methods: `with_new_binaries()`, `new()`, `download_…()`, `fetch_video_infos()`, `combine_audio_and_video()`, etc.
b. Fetcher Subsystem
- `fetcher::deps` – `LibraryInstaller`, `Libraries` for auto-installing `yt-dlp` and `ffmpeg`
- `fetcher::download_manager` – orchestrates segmented/parallel downloads, priorities, retries, buffering
- `fetcher::streams` – handles different stream types (audio/video)
- `fetcher::thumbnail` – downloads thumbnails
c. Executor & Process Management
- `executor` module – spawns and manages external processes (`yt-dlp`, `ffmpeg`), captures stdout/JSON output
d. Caching Layer
- `cache` module – metadata, thumbnail, and file caching
e. Metadata Injection
- `metadata` module – attaches tags/technical info to files, applies optimized logic for combined vs. standalone files
f. Data Models
- `model` – types for `VideoInfo`, `Format`, `FormatSelector`, `Thumbnail`, `Caption`, `Quality` & `CodecPreference` enums
g. Utilities
- `utils::file_system` – path, temp dirs, I/O helpers
- `utils::platform` – platform detection
- `utils::serde` (in model/utils) – JSON (de)serialization of `yt-dlp` output
h. Error & Tracing
- `error.rs` – unified error types
- Optional `tracing` feature integration throughout modules

##### Relationships and Data Flow
1. Public API → Fetcher::deps → downloads/updates `yt-dlp` & `ffmpeg` binaries to disk
2. Public API → Fetcher → upon download request:
- Invokes Executor to run `yt-dlp --dump-json` → Model layer parses JSON via `serde` → yields `VideoInfo`
- Optionally, DownloadManager schedules segments → Executor spawns multiple `yt-dlp` or HTTP streams via `reqwest`
- Streams saved to disk (via utils::file_system) → optional Cache stores thumbnails/metadata
- If combining: Executor runs `ffmpeg` to mux audio+video → Metadata attaches tags
3. Cache sits alongside file-system I/O to avoid re-fetching
4. Errors bubble up from Executor/Fetchers into Public API’s Result

##### Architectural Patterns & Principles
- Modular layering: separation between Process Execution (Executor), Business Logic (Fetcher), Data Models, Caching, and Public API
- Asynchronous “actor‐like” fetcher & download_manager interacting over futures/async tasks (Tokio runtime)
- Feature‐flag gating: optional caching, tracing, rustls for TLS
- Externalized tools: treat `yt-dlp`/`ffmpeg` as “pluggable” services

##### Detailed Component Breakdown (
- Public API (`Youtube`, `LibraryInstaller`)
- Fetcher:
- deps (binaries installer)
- download_manager
- streams & thumbnail
- Executor (process runner)
- Cache (metadata, file, thumbnail)
- Metadata (tag injector)
- Model (VideoInfo, Format, Quality enums)
- Utils (file_system, platform, serde)
- External Services:
- `yt-dlp` binary
- `ffmpeg` binary
- HTTP/reqwest

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Public API Layer
    subgraph "Public API Layer"
        API["Youtube (builder & façade)"]:::api
    end

    %% Fetcher Subsystem
    subgraph "Fetcher Subsystem" 
        subgraph "Deps"
            FI["LibraryInstaller & Libraries"]:::core
        end
        subgraph "Download Manager"
            DM["DownloadManager"]:::core
        end
        subgraph "Streams"
            ST["Streams Handler"]:::core
        end
        subgraph "Thumbnail"
            TH["Thumbnail Downloader"]:::core
        end
        FM["fetcher/mod.rs"]:::core
    end

    %% Executor
    EX["Executor (process runner)"]:::core

    %% Cache Layer
    CA["Cache (metadata, file, thumbnail)"]:::util

    %% Metadata Injection
    MD["Metadata Injector"]:::data

    %% Model Layer
    subgraph "Model Layer"
        M0["model/mod.rs"]:::data
        M1["Caption"]:::data
        M2["Format"]:::data
        M3["FormatSelector"]:::data
        M4["Thumbnail Model"]:::data
    end

    %% Model Utilities
    subgraph "Model Utils"
        MU0["utils/mod.rs"]:::util
        MU1["serde JSON helpers"]:::util
    end

    %% Utilities
    subgraph "Utilities"
        U0["file_system"]:::util
        U1["platform"]:::util
        U2["utils/mod.rs"]:::util
    end

    %% Error & Tracing
    ER["Error & Tracing"]:::error

    %% Configuration & Manifest
    CG["Cargo.toml & README.md"]:::util

    %% External Services
    subgraph "External Tools"
        YTDLP["yt-dlp binary"]:::external
        FFMPEG["ffmpeg binary"]:::external
        HTTP["HTTP (reqwest)"]:::external
    end

    %% Data Flows
    API -->|"install/manage binaries"| FI
    FI -->|"writes binaries to disk"| YTDLP
    FI -->|"writes binaries to disk"| FFMPEG

    API -->|"download request"| FM
    FM -->|delegates| DM
    FM -->|delegates| ST
    FM -->|delegates| TH

    DM -->|"spawns processes"| EX
    ST -->|"streams data via HTTP"| HTTP
    ST -->|"saves streams"| U0
    TH -->|"saves thumbnails"| U0

    EX -->|"runs yt-dlp --dump-json"| YTDLP
    EX -->|"runs ffmpeg for mux"| FFMPEG
    EX -->|"captures JSON output"| M0
    M0 -->|"parses via serde"| MU1

    M0 --> M1
    M0 --> M2
    M0 --> M3
    M0 --> M4

    U0 --> CA
    CA --> FM

    EX --> MD
    MD --> U0

    FM --> ER
    EX --> ER
    API --> ER

    API --> CG

    %% Click Events
    click API "https://github.com/boul2gom/yt-dlp/blob/develop/src/lib.rs"
    click ER "https://github.com/boul2gom/yt-dlp/blob/develop/src/error.rs"
    click FI "https://github.com/boul2gom/yt-dlp/blob/develop/src/fetcher/deps/mod.rs"
    click FI "https://github.com/boul2gom/yt-dlp/blob/develop/src/fetcher/deps/ffmpeg.rs"
    click FI "https://github.com/boul2gom/yt-dlp/blob/develop/src/fetcher/deps/youtube.rs"
    click DM "https://github.com/boul2gom/yt-dlp/blob/develop/src/fetcher/download_manager.rs"
    click ST "https://github.com/boul2gom/yt-dlp/blob/develop/src/fetcher/streams.rs"
    click TH "https://github.com/boul2gom/yt-dlp/blob/develop/src/fetcher/thumbnail.rs"
    click FM "https://github.com/boul2gom/yt-dlp/blob/develop/src/fetcher/mod.rs"
    click EX "https://github.com/boul2gom/yt-dlp/blob/develop/src/executor/mod.rs"
    click CA "https://github.com/boul2gom/yt-dlp/blob/develop/src/cache/mod.rs"
    click MD "https://github.com/boul2gom/yt-dlp/blob/develop/src/metadata/mod.rs"
    click M0 "https://github.com/boul2gom/yt-dlp/blob/develop/src/model/mod.rs"
    click M1 "https://github.com/boul2gom/yt-dlp/blob/develop/src/model/caption.rs"
    click M2 "https://github.com/boul2gom/yt-dlp/blob/develop/src/model/format.rs"
    click M3 "https://github.com/boul2gom/yt-dlp/blob/develop/src/model/format_selector.rs"
    click M4 "https://github.com/boul2gom/yt-dlp/blob/develop/src/model/thumbnail.rs"
    click MU0 "https://github.com/boul2gom/yt-dlp/blob/develop/src/model/utils/mod.rs"
    click MU1 "https://github.com/boul2gom/yt-dlp/blob/develop/src/model/utils/serde.rs"
    click U0 "https://github.com/boul2gom/yt-dlp/blob/develop/src/utils/file_system.rs"
    click U1 "https://github.com/boul2gom/yt-dlp/blob/develop/src/utils/platform.rs"
    click U2 "https://github.com/boul2gom/yt-dlp/blob/develop/src/utils/mod.rs"
    click CG "https://github.com/boul2gom/yt-dlp/blob/develop/Cargo.toml"

    %% Styles
    classDef api fill:#E3F2FD,stroke:#90CAF9,color:#0D47A1
    classDef core fill:#E8EAF6,stroke:#5C6BC0,color:#1A237E
    classDef data fill:#E8F5E9,stroke:#66BB6A,color:#1B5E20
    classDef util fill:#F5F5F5,stroke:#BDBDBD,color:#424242
    classDef error fill:#FFEBEE,stroke:#EF5350,color:#B71C1C
    classDef external fill:#FFF3E0,stroke:#FFA726,color:#E65100
```
