# leolinpotato/drive architecture

> Architecture diagram of the GitHub repository leolinpotato/drive, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/leolinpotato/drive
- Repository: https://github.com/leolinpotato/drive

## Overview

##### Project type and purpose
- DRIVE is a command‐line and script-driven video‐editing pipeline (not a webapp—although a minimal Web UI exists) designed for instance-level control. It glues together three major subcomponents:
- A preprocessing/tracking stage (mask generation via SAMURAI or similar).
- A diffusion–denoising engine (built on the RAVE codebase plus EasyInv for inversion).
- A harmonization post‐processing stage (the Harmonizer codebase).

##### File structure
- Top‐level directories:
- Harmonizer/  – Contains the “Harmonizer” module: model code, training code, pre‐trained weights, video & image demos.
- RAVE/        – Contains the RAVE diffusion engine, a large “annotator” collection of vision submodules, YAML configs, pipelines/, scripts/, main.py, webui.py.
- configs/, data/, pipelines/, scripts/, utils/ – Orchestrate the DRIVE pipeline at a high level.
- Key files:
- main.py & webui.py  – Entry points for CLI and minimal UI.
- pipelines/*.py      – Glue code that sequences preprocessing, denoising, composition, harmonization.
- video_harmonization.sh – Shell wrapper that invokes Harmonizer/demo/video_harmonization/run.py.
- configs/*.yaml      – Parameterize mask/denoise/harmonize settings.
- Harmonizer/pretrained & RAVE/annotator/* – Store pretrained models and vision backbones.

##### README for additional insights
- The methodology section breaks the pipeline into five clear stages: Preprocessing, Denoising, Composition, Harmonization, Detail Refinement (EasyInv).
- External dependencies: SAMURAI for masks; EasyInv for DDIM inversion; PyTorch & associated vision libraries.

a. Main components
1. User Interface Layer
- CLI (main.py, scripts/run_experiment*.py)
- Notebook (demo_notebook.ipynb)
- Web UI (webui.py)
2. Configuration Store
- YAML configs under configs/
3. Preprocessing Module
- Mask generator (external: SAMURAI)
- Preprocessor utils (RAVE/utils/preprocesser_utils.py)
4. Diffusion Engine (RAVE)
- Core denoising (RAVE/annotator + pipelines orchestrator)
- EasyInv inversion logic integrated into pipelines
5. Composition Module
- Mask‐guided compositional denoising formula
6. Harmonization Module (Harmonizer)
- Foreground‐background blending (Harmonizer/src/model, demo/video_harmonization)
7. Output Store
- Edited video frames/video files in data/ or demo output folders

b. Relationships and interactions
- UI triggers pipeline entry point → loads configs → calls Preprocessing → invokes Diffusion Engine → produces latent or frame outputs → feeds Composition logic → hands off to Harmonization → writes final video.
- Preprocessing writes masked frames to disk or memory, consumed by diffusion.
- Diffusion Engine may call external EasyInv routines.
- Harmonizer/demo scripts read foreground, background, mask videos to refine and harmonize.

c. Architectural patterns and principles
- Pipeline (staged‐line) pattern: each stage is a self‐contained module with a clear input/output contract (e.g., frames + masks → denoised latents → composited frames → harmonized video).
- Plugin/annotator pattern in RAVE: lots of interchangeable vision modules (backbones, filters) under annotator/
- Configuration‐driven orchestration (YAML + command‐line flags) decouple code from parameters.

d. Key technologies
- Python 3, PyTorch
- RAVE’s “annotator” subpackage for vision backbones
- Harmonizer’s model/filter modules for post‐processing
- EasyInv code invoked via pipelines
- SAMURAI (external mask generator)

## Diagram (Mermaid)

```mermaid
graph TD
    %% User Interface Layer
    subgraph "User Interface"
        CLI["CLI\n(main.py, run_experiment*.py)"]:::ui
        WebUI["Web UI\n(webui.py)"]:::ui
        Notebook["Notebook\ndemo_notebook.ipynb"]:::ui
    end

    %% Configuration Store
    Config["YAML Configs\n(configs/*.yaml)"]:::config

    %% Core Pipeline Modules
    Preproc["Preprocessing Module\nSAMURAI mask gen & utils"]:::core
    Diffusion["Diffusion Engine\nRAVE/annotator + orchestrator"]:::core
    Composition["Composition Module\nmask-guided pipelines"]:::core
    Harmonization["Harmonization Post-Processing\nHarmonizer/src/model"]:::core

    %% Output Store
    Output["Output Store\nedited videos"]:::output

    %% External Tools
    SAM["SAMURAI\n(external mask generator)"]:::external
    EasyInv["EasyInv\n(DDIM inversion)"]:::external

    %% Data Flow
    CLI -->|"invoke pipeline"| Config
    WebUI -->|"invoke pipeline"| Config
    Notebook -->|"invoke pipeline"| Config
    Config -->|"load params"| Preproc
    Preproc -->|"frames + masks"| Diffusion
    Diffusion -->|"latent states"| Composition
    Composition -->|"composited frames"| Harmonization
    Harmonization -->|"final video"| Output

    %% External Integrations
    Preproc -->|uses| SAM
    Diffusion -->|calls| EasyInv

    %% Click Events
    click CLI "https://github.com/leolinpotato/drive/blob/main/RAVE/main.py"
    click WebUI "https://github.com/leolinpotato/drive/blob/main/RAVE/webui.py"
    click Notebook "https://github.com/leolinpotato/drive/blob/main/RAVE/demo_notebook.ipynb"
    click Config "https://github.com/leolinpotato/drive/tree/main/RAVE/configs/"
    click Preproc "https://github.com/leolinpotato/drive/blob/main/RAVE/utils/preprocesser_utils.py"
    click Diffusion "https://github.com/leolinpotato/drive/tree/main/RAVE/annotator/"
    click Composition "https://github.com/leolinpotato/drive/tree/main/RAVE/pipelines/"
    click EasyInv "https://github.com/leolinpotato/drive/blob/main/RAVE/pipelines/sd_controlnet_rave_mask.py"
    click Harmonization "https://github.com/leolinpotato/drive/tree/main/Harmonizer/src/model/"
    click Output "https://github.com/leolinpotato/drive/tree/main/RAVE/data/mp4_videos/"

    %% Styles
    classDef ui fill:#f9f,stroke:#333,stroke-width:1px
    classDef config fill:#ff9,stroke:#333,stroke-width:1px
    classDef core fill:#9f9,stroke:#333,stroke-width:1px
    classDef external fill:#f99,stroke:#333,stroke-width:1px
    classDef output fill:#99f,stroke:#333,stroke-width:1px
```
