# mikf/gallery-dl architecture

> Architecture diagram of the GitHub repository mikf/gallery-dl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mikf/gallery-dl
- Repository: https://github.com/mikf/gallery-dl

## Overview

gallery-dl is a command-line gallery and collection downloader. A user supplies site URLs and options; the CLI hands work to the job runner, which uses site-specific extractors to obtain media URLs and metadata, then selects downloaders to save files. Configuration, naming, logging, archives, and postprocessors shape that flow. The map preserves the documented breadth of supported sites and optional media handling. Some subsystem links rely on the source index’s import references; unsampled implementations are not treated as inspected.

Main entry points: [`gallery_dl/job.py`](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/job.py) (Job runner), [`gallery_dl/extractor/common.py`](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/common.py) (Extractor base), [`gallery_dl/extractor`](https://github.com/mikf/gallery-dl/tree/master/gallery_dl/extractor) (Site extractors), [`gallery_dl/downloader/http.py`](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/downloader/http.py) (HTTP and media transfer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["CLI and orchestration"]
  node_cli["Options and CLI<br/>[option.py]"]
  node_job["Job runner<br/>[job.py]"]
end

subgraph group_extract["Extraction and authentication"]
  node_registry["Extractor registry<br/>[__init__.py]"]
  node_extractors["Site extractors"]
  node_extract_common["Extractor base<br/>[common.py]"]
  node_oauth["OAuth flows<br/>[oauth.py]"]
  node_cookies["Cookie loading<br/>[cookies.py]"]
end

subgraph group_transfer["Download and output"]
  node_downloader_registry["Downloader registry<br/>[__init__.py]"]
  node_downloader["HTTP and media transfer<br/>[http.py]"]
  node_downloader_base["Downloader base<br/>[common.py]"]
  node_output["Logging and output<br/>[output.py]"]
end

subgraph group_state["Configuration and naming"]
  node_config["Configuration<br/>[config.py]"]
  node_paths["File paths<br/>[path.py]"]
  node_format["Formatting<br/>[formatter.py]"]
end

subgraph group_post["Archives and postprocessing"]
  node_archive[("Download archive<br/>[archive.py]")]
  node_post_registry["Postprocessor registry<br/>[__init__.py]"]
  node_post_common["Postprocessor base<br/>[common.py]"]
  node_post_features["File postprocessing"]
end

subgraph group_core["Shared runtime"]
  node_shared["Shared utilities<br/>[util.py]"]
  node_dates["Date and time<br/>[dt.py]"]
  node_text["Text utilities<br/>[text.py]"]
  node_errors["Errors and control<br/>[exception.py]"]
end

node_user(("CLI user"))
node_danbooru(("Danbooru service"))
node_filesystem[("Local files")]

node_user -->|"supplies URLs"| node_cli
node_cli -->|"starts jobs"| node_job
node_job -->|"selects extractor"| node_registry
node_registry -.->|"resolves site"| node_extractors
node_extractors -->|"requests content"| node_danbooru
node_extractors -->|"uses base"| node_extract_common
node_extract_common -->|"reads settings"| node_config
node_extract_common -->|"loads cookies"| node_cookies
node_extractors -.->|"offers authorization"| node_oauth
node_job -->|"selects downloader"| node_downloader_registry
node_downloader_registry -.->|"dispatches transfer"| node_downloader
node_downloader -->|"uses base"| node_downloader_base
node_downloader_base -->|"reads settings"| node_config
node_job -->|"prepares paths"| node_paths
node_job -->|"formats metadata"| node_format
node_format -->|"evaluates expressions"| node_shared
node_downloader -.->|"writes media"| node_filesystem
node_job -->|"reports progress"| node_output
node_job -->|"tracks downloads"| node_archive
node_job -->|"configures processors"| node_post_registry
node_post_registry -.->|"selects processor"| node_post_features
node_post_features -->|"uses base"| node_post_common
node_post_common -->|"opens archive"| node_archive
node_extract_common -->|"normalizes dates"| node_dates
node_extract_common -->|"raises extraction errors"| node_errors
node_downloader -->|"handles transfer errors"| node_errors
node_shared -->|"uses text helpers"| node_text

click node_cli "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/option.py"
click node_job "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/job.py"
click node_registry "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/__init__.py"
click node_extractors "https://github.com/mikf/gallery-dl/tree/master/gallery_dl/extractor"
click node_extract_common "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/common.py"
click node_oauth "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/oauth.py"
click node_cookies "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/cookies.py"
click node_downloader_registry "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/downloader/__init__.py"
click node_downloader "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/downloader/http.py"
click node_downloader_base "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/downloader/common.py"
click node_output "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/output.py"
click node_config "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/config.py"
click node_paths "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/path.py"
click node_format "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/formatter.py"
click node_archive "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/archive.py"
click node_post_registry "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/postprocessor/__init__.py"
click node_post_common "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/postprocessor/common.py"
click node_post_features "https://github.com/mikf/gallery-dl/tree/master/gallery_dl/postprocessor"
click node_shared "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/util.py"
click node_dates "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/dt.py"
click node_text "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/text.py"
click node_errors "https://github.com/mikf/gallery-dl/blob/master/gallery_dl/exception.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_job,node_user toneBlue
class node_registry,node_extractors,node_extract_common,node_oauth,node_cookies,node_filesystem toneAmber
class node_downloader_registry,node_downloader,node_downloader_base,node_output toneMint
class node_config,node_paths,node_format toneRose
class node_archive,node_post_registry,node_post_common,node_post_features,node_danbooru toneIndigo
class node_shared,node_dates,node_text,node_errors toneTeal
```

## Components

### CLI and orchestration

- [Options and CLI](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/option.py): `gallery_dl/option.py`
- [Job runner](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/job.py): `gallery_dl/job.py`

### Extraction and authentication

- [Extractor registry](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/__init__.py): `gallery_dl/extractor/__init__.py`
- [Site extractors](https://github.com/mikf/gallery-dl/tree/master/gallery_dl/extractor): `gallery_dl/extractor`
- [Extractor base](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/common.py): `gallery_dl/extractor/common.py`
- [OAuth flows](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/oauth.py): `gallery_dl/extractor/oauth.py`
- [Cookie loading](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/cookies.py): `gallery_dl/cookies.py`

### Download and output

- [Downloader registry](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/downloader/__init__.py): `gallery_dl/downloader/__init__.py`
- [HTTP and media transfer](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/downloader/http.py): `gallery_dl/downloader/http.py`
- [Downloader base](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/downloader/common.py): `gallery_dl/downloader/common.py`
- [Logging and output](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/output.py): `gallery_dl/output.py`

### Configuration and naming

- [Configuration](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/config.py): `gallery_dl/config.py`
- [File paths](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/path.py): `gallery_dl/path.py`
- [Formatting](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/formatter.py): `gallery_dl/formatter.py`

### Archives and postprocessing

- [Download archive](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/archive.py): `gallery_dl/archive.py`
- [Postprocessor registry](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/postprocessor/__init__.py): `gallery_dl/postprocessor/__init__.py`
- [Postprocessor base](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/postprocessor/common.py): `gallery_dl/postprocessor/common.py`
- [File postprocessing](https://github.com/mikf/gallery-dl/tree/master/gallery_dl/postprocessor): `gallery_dl/postprocessor`

### Shared runtime

- [Shared utilities](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/util.py): `gallery_dl/util.py`
- [Date and time](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/dt.py): `gallery_dl/dt.py`
- [Text utilities](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/text.py): `gallery_dl/text.py`
- [Errors and control](https://github.com/mikf/gallery-dl/blob/master/gallery_dl/exception.py): `gallery_dl/exception.py`

### Other components

- **CLI user**
- **Danbooru service**
- **Local files**

## Connections

- CLI user → Options and CLI: supplies URLs
- Options and CLI → Job runner: starts jobs (evidence: `gallery_dl/option.py`)
- Job runner → Extractor registry: selects extractor (evidence: `gallery_dl/job.py`)
- Extractor registry → Site extractors: resolves site
- Site extractors → Danbooru service: requests content
- Site extractors → Extractor base: uses base (evidence: `gallery_dl/extractor/deviantart.py`)
- Extractor base → Configuration: reads settings (evidence: `gallery_dl/extractor/common.py`)
- Extractor base → Cookie loading: loads cookies (evidence: `gallery_dl/extractor/common.py`)
- Site extractors → OAuth flows: offers authorization (evidence: `gallery_dl/extractor/oauth.py`)
- Job runner → Downloader registry: selects downloader (evidence: `gallery_dl/job.py`)
- Downloader registry → HTTP and media transfer: dispatches transfer
- HTTP and media transfer → Downloader base: uses base (evidence: `gallery_dl/downloader/http.py`)
- Downloader base → Configuration: reads settings (evidence: `gallery_dl/downloader/common.py`)
- Job runner → File paths: prepares paths (evidence: `gallery_dl/job.py`)
- Job runner → Formatting: formats metadata (evidence: `gallery_dl/job.py`)
- Formatting → Shared utilities: evaluates expressions (evidence: `gallery_dl/formatter.py`)
- HTTP and media transfer → Local files: writes media
- Job runner → Logging and output: reports progress (evidence: `gallery_dl/job.py`)
- Job runner → Download archive: tracks downloads (evidence: `gallery_dl/job.py`)
- Job runner → Postprocessor registry: configures processors (evidence: `gallery_dl/job.py`)
- Postprocessor registry → File postprocessing: selects processor
- File postprocessing → Postprocessor base: uses base (evidence: `gallery_dl/postprocessor/exec.py`)
- Postprocessor base → Download archive: opens archive (evidence: `gallery_dl/postprocessor/common.py`)
- Extractor base → Date and time: normalizes dates (evidence: `gallery_dl/extractor/common.py`)
- Extractor base → Errors and control: raises extraction errors (evidence: `gallery_dl/extractor/common.py`)
- HTTP and media transfer → Errors and control: handles transfer errors (evidence: `gallery_dl/downloader/http.py`)
- Shared utilities → Text utilities: uses text helpers (evidence: `gallery_dl/util.py`)
