# sdbds/qinglong-captions architecture

> Architecture diagram of the GitHub repository sdbds/qinglong-captions, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sdbds/qinglong-captions
- Repository: https://github.com/sdbds/qinglong-captions

## Overview

##### Project type and purpose
- This is a command-line Python toolkit (“qinglong-captioner”) for batch video caption generation and dataset management.
- It imports videos into a LanceDB data store, segments scenes, calls external APIs (Gemini, Pixtral, Mistral OCR, Qwen-VL, GLM) for caption or tag generation, aligns and exports timestamps as SRT, and can re-export media and captions.

##### File structure
- Top-level PowerShell scripts (install, run, lanceImport.ps1, lanceExport.ps1) act as CLI entry points.
- `config/` holds configuration loader (`config.py`) and `config.toml`.
- `datasets/` is the input directory placeholder for media files.
- `module/` contains core Python modules:
- `lanceImport.py` & `lanceexport.py` for data ingestion/export
- `scenedetect.py` for video segmentation
- `captioner.py` & `api_handler.py` for caption pipeline
- `lanceImport.py`/`lanceexport.py` wrappers
- `utils/` provides shared utilities (console I/O, streaming, WD tagger integration).
- Requirements files specify dependencies.

##### README for additional insights
- The README confirms the above modules, describes features (scene detection, SRT export, multi-model API support), and shows the workflow: media import → segmentation → caption/tag generation → SRT export → optional re-export.

a. Main components
- CLI Layer: PowerShell scripts (install, import, run, export)
- Configuration: `config/config.py` + `config/config.toml`
- Data Store: LanceDB (video + metadata + captions)
- Ingestion Module: `module/lanceImport.py`
- Segmentation Module: `module/scenedetect.py`
- Caption Pipeline:
– Caption Orchestrator (`module/captioner.py`)
– API Handler (`module/api_handler.py`)
– Tagging (`module/utils/wdtagger.py`)
- Export Module: `module/lanceexport.py`
- Utilities: console and stream helpers
- External Services: Google Gemini API, Pixtral API, Mistral OCR, Qwen-VL, GLM
b. Relationships and interactions
1. CLI → Configuration loads → `datasets/` path
2. CLI import script → Ingestion Module reads files → writes to LanceDB
3. CLI run script → reads from LanceDB → Segmentation Module emits clips/data
4. Caption Pipeline reads clips → calls External APIs → returns captions/tags
5. Timestamp alignment (in captioner) → updates records in LanceDB
6. CLI export script → Export Module reads LanceDB → writes SRT alongside original videos
c. Architectural patterns and design principles
- Layered architecture: CLI / Config / Core modules / External APIs
- Single Responsibility: each module focuses on one concern (ingest, segment, caption, export)
- Plugin-like external API support: abstraction in `api_handler`
- Data-centric design: LanceDB as central store
d. Key technologies/frameworks
- Python 3.x
- LanceDB (for dataset storage)
- PySceneDetect (scene detection)
- Google Gemini SDK, Pixtral, Mistral, Qwen-VL, GLM SDKs
- ONNX/TensorRT for WDtagger acceleration
- TOML for config

##### Detail
- Retry logic and streaming (in caption pipeline) as annotations.
- Configuration parameters (scene_detector, thresholds) feeding into segmentation.
- Multi-model flexibility in `api_handler`.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layers and Components
    subgraph "CLI Layer"
        install1["install pwsh.sh"]:::cli
        install2["install-uv-qinglong.ps1"]:::cli
        import_cli["lanceImport.ps1"]:::cli
        run_cli["run.ps1"]:::cli
        video_split["video_spliter.ps1"]:::cli
        tagger_cli["tagger.ps1"]:::cli
        export_cli["lanceExport.ps1"]:::cli
    end

    subgraph "Configuration"
        config_loader["config/config.py"]:::config
        config_file["config/config.toml"]:::config
    end

    subgraph "Utils"
        console_util["utils/console_util.py"]:::utils
        stream_util["utils/stream_util.py"]:::utils
    end

    subgraph "Core Modules"
        subgraph "Ingestion"
            ingest["module/lanceImport.py"]:::core
        end
        subgraph "Segmentation"
            segment["module/scenedetect.py"]:::core
        end
        subgraph "Caption Pipeline"
            captioner["module/captioner.py"]:::core
            api_handler["module/api_handler.py"]:::core
            wdtagger["utils/wdtagger.py"]:::utils
        end
        export_mod["module/lanceexport.py"]:::core
    end

    ds["datasets/"]:::data
    db["LanceDB"]:::db

    subgraph "External Services"
        gemini["Google Gemini API"]:::external
        pixtral["Pixtral API"]:::external
        mistral["Mistral OCR"]:::external
        qwen["Qwen-VL"]:::external
        glm["GLM SDK"]:::external
    end

    %% Data Flow
    ds -->|feeds| import_cli
    import_cli --> ingest
    ingest --> db
    run_cli --> segment
    segment --> captioner
    captioner --> api_handler
    api_handler --> gemini
    api_handler --> pixtral
    api_handler --> mistral
    api_handler --> qwen
    api_handler --> glm
    captioner -->|uses| wdtagger
    captioner --> db
    export_cli --> export_mod
    export_mod --> db
    export_mod -->|writes SRT| ds

    %% Configuration flows
    install1 --> config_loader
    install2 --> config_loader
    import_cli --> config_loader
    run_cli --> config_loader
    tagger_cli --> config_loader
    video_split --> config_loader
    export_cli --> config_loader
    config_loader --> config_file

    %% Utilities usage
    ingest --> console_util
    segment --> console_util
    captioner --> console_util
    export_mod --> console_util
    ingest --> stream_util
    captioner --> stream_util

    %% Click Events
    click install1 "https://github.com/sdbds/qinglong-captions/blob/main/0、install pwsh.sh"
    click install2 "https://github.com/sdbds/qinglong-captions/blob/main/1、install-uv-qinglong.ps1"
    click import_cli "https://github.com/sdbds/qinglong-captions/blob/main/lanceImport.ps1"
    click run_cli "https://github.com/sdbds/qinglong-captions/blob/main/run.ps1"
    click video_split "https://github.com/sdbds/qinglong-captions/blob/main/video_spliter.ps1"
    click tagger_cli "https://github.com/sdbds/qinglong-captions/blob/main/tagger.ps1"
    click export_cli "https://github.com/sdbds/qinglong-captions/blob/main/lanceExport.ps1"
    click config_loader "https://github.com/sdbds/qinglong-captions/blob/main/config/config.py"
    click config_file "https://github.com/sdbds/qinglong-captions/blob/main/config/config.toml"
    click ingest "https://github.com/sdbds/qinglong-captions/blob/main/module/lanceImport.py"
    click segment "https://github.com/sdbds/qinglong-captions/blob/main/module/scenedetect.py"
    click captioner "https://github.com/sdbds/qinglong-captions/blob/main/module/captioner.py"
    click api_handler "https://github.com/sdbds/qinglong-captions/blob/main/module/api_handler.py"
    click wdtagger "https://github.com/sdbds/qinglong-captions/blob/main/utils/wdtagger.py"
    click export_mod "https://github.com/sdbds/qinglong-captions/blob/main/module/lanceexport.py"
    click console_util "https://github.com/sdbds/qinglong-captions/blob/main/utils/console_util.py"
    click stream_util "https://github.com/sdbds/qinglong-captions/blob/main/utils/stream_util.py"

    %% Styles
    classDef cli fill:#cccccc,stroke:#666666,color:#000000
    classDef config fill:#dddddd,stroke:#888888,color:#000000
    classDef utils fill:#eeeeee,stroke:#777777,color:#000000
    classDef core fill:#AED6F1,stroke:#21618C,color:#000000
    classDef db fill:#A9DFBF,stroke:#196F3D,color:#000000
    classDef external fill:#F5B041,stroke:#B9770E,color:#000000
    classDef data fill:#D5D8DC,stroke:#808B96,color:#000000
```
