# lencodigitexer/yadisk-dl architecture

> Architecture diagram of the GitHub repository lencodigitexer/yadisk-dl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lencodigitexer/yadisk-dl
- Repository: https://github.com/lencodigitexer/yadisk-dl

## Overview

##### Project Type and Purpose
- This is a command-line utility (CLI) for mass-downloading photos from Yandex.Disk. It is composed of three sequential modules (authentication, link collection, download), each packaged as a standalone executable.

##### High-Level Components
a. User / CLI
- The human operator who invokes each executable in turn.
b. Auth Module (00auth.py → 01_ya_auth.exe)
- Uses Selenium and a real browser to log into Yandex.Disk.
- Outputs: session.pkl (serialized session credentials).
c. Link-Collection Module (01getlinks.py → 02_ya_getlinks.exe)
- Reads session.pkl.
- Crawls the user’s Yandex.Disk via HTTP requests (Requests library).
- Persists photo URLs into SQLite database links.db.
d. Download Module (02download.py → 03_ya_download.exe)
- Reads links.db.
- Spawns up to 10 worker threads to fetch each URL and write files into downloads/.
e. Data Stores
- session.pkl – authentication state.
- links.db – SQLite database of photo URLs.
- downloads/ – local filesystem directory for images.
f. External Service
- Yandex.Disk API / web endpoints (target of HTTP calls).
g. Build & CI Pipeline
- pyproject.toml / requirements.txt – dependency manifests.
- GitHub Actions (build.yml) – runs tests and pyinstaller packaging.
- PyInstaller – produces one-file executables for each module.

##### Interactions and Data Flow
- Step 1: User → Auth Module → Browser (Selenium) → Yandex.Disk → session.pkl
- Step 2: User → Link-Collection Module → session.pkl → Yandex.Disk API → links.db
- Step 3: User → Download Module → links.db → Yandex.Disk API → downloads/
- CI: Push to repo → GitHub Actions → PyInstaller → Release binaries

##### Architectural Patterns & Principles
- Pipeline / Chain of Responsibility: three distinct stages, each taking the output of the previous.
- Separation of concerns: authentication, metadata gathering, and file I/O are isolated.
- Offline data storage: use of session file and local SQLite cache to decouple external calls.
- Concurrency in download stage via thread pool to maximize throughput.

##### Detailed Items to Capture
- The exact file names of the executables (01_ya_auth.exe, etc.).
- The libraries involved (Selenium, Requests, SQLite3, threading/TQDM).
- The CI workflow file (.github/workflows/build.yml) and its purpose.
- The packaging step (PyInstaller) and where outputs go (releases).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Control Flow
    User("User / CLI"):::user
    User -.->|invokes| AuthExe

    %% Authentication Stage
    subgraph "Authentication Stage"
        AuthExe["01_ya_auth.exe\n(Uses Selenium)"]:::auth
        Browser["Browser\n(Selenium)"]:::auth
        YD1("Yandex.Disk\n(web endpoints)"):::external
        Session["session.pkl"]:::store
    end

    %% Link Collection Stage
    subgraph "Link-Collection Stage"
        LinkExe["02_ya_getlinks.exe\n(Uses Requests)"]:::link
        YD2("Yandex.Disk API"):::external
        LinksDB["links.db\n(SQLite)"]:::store
    end

    %% Download Stage
    subgraph "Download Stage"
        DownloadExe["03_ya_download.exe\n(threading/TQDM)"]:::download
        YD3("Yandex.Disk API"):::external
        Downloads["downloads/"]:::store
    end

    %% CI & Build Pipeline
    subgraph "CI & Build Pipeline"
        CI["GitHub Actions\n(.github/workflows/build.yml)"]:::ci
        PyInst["PyInstaller"]:::ci
        Manifests["pyproject.toml\nrequirements.txt"]:::ci
        Docs["docs/"]:::ci
        README["README.md\nLICENSE"]:::ci
        Releases["releases/"]:::ci
    end

    %% Data Flow
    AuthExe -->|writes session.pkl| Session
    AuthExe -->|launches| Browser
    Browser -->|authenticates| YD1
    LinkExe -->|reads session.pkl| Session
    LinkExe -->|queries| YD2
    LinkExe -->|writes links.db| LinksDB
    DownloadExe -->|reads links.db| LinksDB
    DownloadExe -->|downloads| YD3
    DownloadExe -->|writes files| Downloads

    %% User Invocations
    User -.->|invokes| LinkExe
    User -.->|invokes| DownloadExe

    %% CI Flow
    CI -->|installs| Manifests
    CI -->|runs| PyInst
    PyInst -->|produces| AuthExe
    PyInst -->|produces| LinkExe
    PyInst -->|produces| DownloadExe
    CI -->|pushes| Releases

    %% Click Events
    click AuthExe "https://github.com/lencodigitexer/yadisk-dl/blob/main/src/00auth.py"
    click LinkExe "https://github.com/lencodigitexer/yadisk-dl/blob/main/src/01getlinks.py"
    click DownloadExe "https://github.com/lencodigitexer/yadisk-dl/blob/main/src/02download.py"
    click CI "https://github.com/lencodigitexer/yadisk-dl/blob/main/.github/workflows/build.yml"
    click Manifests "https://github.com/lencodigitexer/yadisk-dl/blob/main/pyproject.toml"
    click Manifests "https://github.com/lencodigitexer/yadisk-dl/blob/main/requirements.txt"
    click Docs "https://github.com/lencodigitexer/yadisk-dl/tree/main/docs/"
    click README "https://github.com/lencodigitexer/yadisk-dl/blob/main/README.md"
    click README "https://github.com/lencodigitexer/yadisk-dl/tree/main/LICENSE"

    %% Styles
    classDef user fill:#f9f,stroke:#333,stroke-width:1px
    classDef auth fill:#bbdefb,stroke:#0d47a1,stroke-width:1px
    classDef link fill:#ffe0b2,stroke:#e65100,stroke-width:1px
    classDef download fill:#c8e6c9,stroke:#1b5e20,stroke-width:1px
    classDef store fill:#eeeeee,stroke:#424242,stroke-width:1px,stroke-dasharray: 5 5
    classDef external fill:#eceff1,stroke:#607d8b,stroke-width:1px,stroke-dasharray: 2 2
    classDef ci fill:#d1c4e9,stroke:#4a148c,stroke-width:1px
```
