# st1llwater/tbx-dl architecture

> Architecture diagram of the GitHub repository st1llwater/tbx-dl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/st1llwater/tbx-dl
- Repository: https://github.com/st1llwater/tbx-dl

## Overview

##### Project Type and Purpose
- This is a Node.js–based Discord bot (“TBX-DL”) whose primary function is to download, process, and serve M3U8-streamed videos on demand.
- It exposes Discord slash/prefix commands, manages an efficient concurrent download queue, uses Puppeteer for stealth browser automation, and FFmpeg for video processing.

##### File Structure
- Top-level scripts & configs:
- Dockerfile, docker-compose.yml, .env.example – containerization and environment setup
- segmentWorker.js – worker entry point for streaming segment processing
- files.txt – persistence listing (queue state)
- src/commands – command definitions and a commandHandler
- clearCommand, helpCommand, queueCommand, scrapeCommand
- src/services – core services
- browserService.js (Puppeteer setup & navigation)
- videoService.js (FFmpeg-based segment assembly & optimization)
- src/utils – helpers
- fileUtils.js (disk I/O)
- queueManager.js (in-memory + file-backed FIFO queue)
- src/workers – worker thread logic for segment processing
- segmentWorker.js
- src/config/constants.js – all configurable parameters (queue sizes, browser args, file limits)

##### Key Insights from README
- Architecture revolves around a robust, persistent FIFO queue (local file backup, auto-recovery).
- Concurrency: 2 simultaneous downloads; up to 20 worker threads for segment handling.
- Browser automation via Puppeteer Stealth plugin; anti-detection measures; dynamic user-agent & network spoofing.
- Docker‐based deployment with persistent volume mappings for output and temporary segments.

##### Essentials to Capture
- Job lifecycle from command receipt to final video delivery
- Concurrency limits and worker pool structure
- Persistence layer ensuring auto-recovery
- Browser automation & stealth plugin integration
- Containerization boundaries (Docker host, volumes)

## Diagram (Mermaid)

```mermaid
graph TD
    %% External Systems
    User["Discord User"]:::external
    DiscordAPI["Discord Gateway & API"]:::external

    %% Bot Core
    BotCore["Bot Core<br/>(src/index.js)"]:::core

    %% Commands Subgraph
    subgraph "Commands Layer"
        direction TB
        CmdHandler["CommandHandler<br/>(src/commands/commandHandler.js)"]:::core
        ClearCmd["clearCommand<br/>(src/commands/clearCommand.js)"]:::core
        HelpCmd["helpCommand<br/>(src/commands/helpCommand.js)"]:::core
        QueueCmd["queueCommand<br/>(src/commands/queueCommand.js)"]:::core
        ScrapeCmd["scrapeCommand<br/>(src/commands/scrapeCommand.js)"]:::core
    end

    %% Utils & Queue Manager
    subgraph "Utils & Persistence"
        direction TB
        QueueMgr["Queue Manager<br/>(src/utils/queueManager.js)"]:::core
        FilesTXT[(files.txt)]:::storage
        FileUtils["File Utilities<br/>(src/utils/fileUtils.js)"]:::core
    end

    %% Services Layer
    subgraph "Services Layer"
        direction TB
        BrowserSvc["BrowserService<br/>(src/services/browserService.js)"]:::core
        VideoSvc["VideoService<br/>(src/services/videoService.js)"]:::core
    end

    %% Worker Pool
    subgraph "Worker Pool"
        direction TB
        WorkerEntry["segmentWorker.js"]:::core
        WorkerLogic["segmentWorker<br/>(src/workers/segmentWorker.js)"]:::core
    end

    %% Storage
    subgraph "File Storage"
        direction TB
        Segments[(segments/)]:::storage
        Output[(output/)]:::storage
    end

    %% Containerization
    subgraph "Containerization" 
        direction TB
        Dockerfile["Dockerfile"]:::infra
        DockerCompose["docker-compose.yml"]:::infra
        EnvExample[".env.example"]:::infra
    end

    %% Configuration
    Constants["Config Constants<br/>(src/config/constants.js)"]:::core

    %% Flow
    User -->|"invokes command"| DiscordAPI
    DiscordAPI -->|"passes event"| BotCore
    BotCore -->|"routes to"| CmdHandler

    CmdHandler -->|"/clear"| ClearCmd
    CmdHandler -->|"/help"| HelpCmd
    CmdHandler -->|"/queue"| QueueCmd
    CmdHandler -->|"/scrape"| ScrapeCmd

    ScrapeCmd -->|"enqueue job"| QueueMgr
    QueueCmd -->|"query"| QueueMgr
    ClearCmd -->|"clear jobs"| QueueMgr

    QueueMgr -->|"persist state"| FilesTXT
    QueueMgr -->|"dispatch jobs"| WorkerEntry
    WorkerEntry --> WorkerLogic

    WorkerLogic -->|"use"| BrowserSvc
    WorkerLogic -->|"use"| VideoSvc
    WorkerLogic -->|"write segments"| Segments
    WorkerLogic -->|"write final mp4"| Output

    WorkerLogic -->|"notify completion"| QueueMgr
    QueueMgr -->|"trigger message"| BotCore
    BotCore -->|"send message"| DiscordAPI
    DiscordAPI --> User

    BotCore --> Constants

    BotCore --> FileUtils

    Containerization --> BotCore
    Containerization --> Segments
    Containerization --> Output

    %% Click Events
    click DiscordAPI "https://github.com/st1llwater/tbx-dl/blob/main/src/index.js"
    click BotCore "https://github.com/st1llwater/tbx-dl/blob/main/src/index.js"
    click CmdHandler "https://github.com/st1llwater/tbx-dl/blob/main/src/commands/commandHandler.js"
    click ClearCmd "https://github.com/st1llwater/tbx-dl/blob/main/src/commands/clearCommand.js"
    click HelpCmd "https://github.com/st1llwater/tbx-dl/blob/main/src/commands/helpCommand.js"
    click QueueCmd "https://github.com/st1llwater/tbx-dl/blob/main/src/commands/queueCommand.js"
    click ScrapeCmd "https://github.com/st1llwater/tbx-dl/blob/main/src/commands/scrapeCommand.js"
    click QueueMgr "https://github.com/st1llwater/tbx-dl/blob/main/src/utils/queueManager.js"
    click FilesTXT "https://github.com/st1llwater/tbx-dl/blob/main/files.txt"
    click FileUtils "https://github.com/st1llwater/tbx-dl/blob/main/src/utils/fileUtils.js"
    click BrowserSvc "https://github.com/st1llwater/tbx-dl/blob/main/src/services/browserService.js"
    click VideoSvc "https://github.com/st1llwater/tbx-dl/blob/main/src/services/videoService.js"
    click WorkerEntry "https://github.com/st1llwater/tbx-dl/blob/main/segmentWorker.js"
    click WorkerLogic "https://github.com/st1llwater/tbx-dl/blob/main/src/workers/segmentWorker.js"
    click Constants "https://github.com/st1llwater/tbx-dl/blob/main/src/config/constants.js"
    click Dockerfile "https://github.com/st1llwater/tbx-dl/tree/main/Dockerfile"
    click DockerCompose "https://github.com/st1llwater/tbx-dl/blob/main/docker-compose.yml"
    click EnvExample "https://github.com/st1llwater/tbx-dl/blob/main/.env.example"

    %% Styles
    classDef external fill:#cce5ff,stroke:#004085,color:#004085
    classDef core fill:#d4edda,stroke:#155724,color:#155724
    classDef infra fill:#e2e3e5,stroke:#6c757d,color:#6c757d
    classDef storage fill:#fff3cd,stroke:#856404,color:#856404
```
