# mlsakiit/forgetube architecture

> Architecture diagram of the GitHub repository mlsakiit/forgetube, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mlsakiit/forgetube
- Repository: https://github.com/mlsakiit/forgetube

## Overview

##### Project Type and Purpose
- Project Type: This is a multi-modal automated video generation pipeline that integrates AI-based text generation, image synthesis, and text-to-speech (TTS) to automatically generate narrated videos.
- Purpose: The system takes a text prompt, uses a large language model (Gemini API) for script generation, then leverages diffusion models (Stable Diffusion XL) to create images, and converts text to audio (using Kokoro) before assembling the final video using video processing libraries (MoviePy/FFmpeg).

##### File Structure
- Key Directories and Files:
- assembly/: Contains scripts and templates (video_template.json) for video assembly, likely using MoviePy and FFmpeg.
- diffusion/: Contains scripts (generate_image.py, generate_image_local.py, Modelfile, etc.) for image generation via diffusion models.
- tts/: Contains scripts (generate_audio.py) for the text-to-speech functionality.
- main.py and main_local.py: The central entry points that orchestrate the entire pipeline.
- resources/: Holds assets like intro videos, fonts, and potentially placeholders for images and audio.
- Additional files such as pyproject.toml, .python-version, and uv.lock help manage project dependencies and Python versioning.

##### Components, Interactions, and Architectural Patterns
- Component Breakdown:
- Script Generation Module: Uses Gemini API (and Serp API) to generate a JSON-based script with text, audio, and visual prompts. (Interaction point: main.py/main_local.py where API keys are provided.)
- Image Generation Module (Diffusion): Processes the script’s prompts to generate images. It supports running the model either on Modal (cloud GPU) or locally. The diffusion folder contains different scripts that handle various image generation scenarios.
- Audio Generation Module (TTS): Converts text into speech using Kokoro. This is implemented in tts/scripts/generate_audio.py.
- Video Assembly Module: Collects the generated images, audio, and additional resources (intro, fonts, subtitles) to create the final video. This is handled by the assembly/scripts/assembly_video.py.
- Resource and Asset Management: The resources/ directory manages static files (e.g., intros, fonts) and the pipeline uses these assets during video finalization.
- Architectural Principles:
- Modular Architecture: Each functionality (script generation, image creation, audio synthesis, video assembly) is well encapsulated in its own folder.
- Sequential Pipeline: The output of one stage feeds into the next (script → images/audio → video assembly).
- Integration with External Services: Gemini and Serp APIs, FFmpeg for video processing, and Modal for cloud-based GPU acceleration.
- Dependencies and Technologies:
- Python 3.11, PyTorch, Diffusers (Hugging Face), MoviePy/FFmpeg, and Kokoro for TTS along with UV and Modal for dependency and resource management.

##### Final Summary
- The top layer represents external interactions and inputs (user prompts, API calls).
- The middle layer is divided into three main processing components (Script Generation, Image Generation, TTS).
- The bottom layer is the Video Assembly which integrates outputs from the processing layer along with resource assets.
- Modularity and sequential processing, which is central to this automated video generation pipeline.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Top Layer
    A["User Input (Text Prompt)"]:::input
    B["Script Generation (LLM)"]:::module
    C["Gemini API"]:::external
    D["Serp API"]:::external

    A -->|"provides_prompt"| B
    B -->|"calls"| C
    B -->|"calls"| D

    %% Middle Layer
    E["Image Generation (Diffusion)"]:::module
    F["Text-to-Speech (TTS)"]:::module

    B -->|"outputs_script"| E
    B -->|"outputs_script"| F
    G["Modal (Cloud GPU)"]:::external
    E -->|"uses_GPUs"| G
    H["Kokoro_TTS_Service"]:::external
    F -->|"utilizes"| H

    %% Bottom Layer
    I["Video Assembly Engine"]:::module
    J["Resource Management"]:::module
    E -->|"sends_images"| I
    F -->|"sends_audio"| I
    J -->|"provides_assets"| I
    K["FFmpeg/MoviePy"]:::external
    I -->|"processes_video"| K

    %% Click Events for Script Generation Module
    click B "https://github.com/mlsakiit/forgetube/blob/main/main.py"
    click B "https://github.com/mlsakiit/forgetube/blob/main/main_local.py"
    click B "https://github.com/mlsakiit/forgetube/blob/main/diffusion/scripts/generate_script.py"

    %% Click Events for Image Generation Module
    click E "https://github.com/mlsakiit/forgetube/tree/main/diffusion/"
    click E "https://github.com/mlsakiit/forgetube/blob/main/diffusion/scripts/generate_image.py"
    click E "https://github.com/mlsakiit/forgetube/blob/main/diffusion/scripts/generate_image_local.py"
    click E "https://github.com/mlsakiit/forgetube/tree/main/diffusion/scripts/Modelfile"

    %% Click Event for Audio Generation Module
    click F "https://github.com/mlsakiit/forgetube/blob/main/tts/scripts/generate_audio.py"

    %% Click Events for Video Assembly Module
    click I "https://github.com/mlsakiit/forgetube/blob/main/assembly/scripts/assembly_video.py"
    click I "https://github.com/mlsakiit/forgetube/blob/main/assembly/templates/video_template.json"

    %% Click Events for Resource Management
    click J "https://github.com/mlsakiit/forgetube/tree/main/resources/"
    click J "https://github.com/mlsakiit/forgetube/tree/main/resources/Intro"
    click J "https://github.com/mlsakiit/forgetube/tree/main/resources/font"

    %% Styles
    classDef external fill:#f9e79f,stroke:#d35400,stroke-width:2px;
    classDef module fill:#aed6f1,stroke:#2471a3,stroke-width:2px;
    classDef input fill:#d5f5e3,stroke:#28b463,stroke-width:2px;
```
