# browser-use/video-use architecture

> Architecture diagram of the GitHub repository browser-use/video-use, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/browser-use/video-use
- Repository: https://github.com/browser-use/video-use

## Overview

1. **What this repo is**
   - `video-use` is not a conventional app; it is an **agent-oriented video editing skill**. The primary “runtime” is a Claude Code–style agent with shell access that reads the repo docs, runs helper scripts, and iterates on edits.

2. **Core control plane**
   - The human gives a folder of raw footage and a goal.
   - The agent follows `SKILL.md` and `install.md`, inventories the sources, proposes an edit strategy, then drives the rest of the pipeline.
   - The repo is designed so outputs stay in `<videos_dir>/edit/`, while the skill repo itself remains clean.

3. **Input/output contract**
   - **Inputs:** raw video/audio takes in a working folder, plus configuration via `.env` and the agent session.
   - **Outputs:** edited media artifacts, especially `final.mp4`, plus intermediate transcripts, packed notes, render previews, and session memory like `project.md`.

4. **Transcription subsystem**
   - `helpers/transcribe.py` and `helpers/transcribe_batch.py` form the ingestion layer for source media.
   - The README indicates these rely on **ElevenLabs Scribe** to produce **word-level timestamps**, **speaker diarization**, and audio events like laughter or applause.
   - This is the foundation for all cut decisions: the LLM reasons over text, not raw frames.

5. **Transcript packing layer**
   - `helpers/pack_transcripts.py` compresses the per-take transcript output into a compact markdown view such as `takes_packed.md`.
   - This packed transcript is the agent’s main reading surface and is intentionally small enough to support precise editing decisions without frame-by-frame inspection.

6. **On-demand visual inspection**
   - `helpers/timeline_view.py` generates a composite visual for selected time ranges: filmstrip, waveform, speaker/word labels, and cut candidates.
   - This is used only at decision points or sanity checks, so visual data acts as a targeted supplement to transcript reasoning rather than the primary representation.

7. **Render and finishing pipeline**
   - `helpers/render.py` is the execution boundary that turns the chosen edit decisions into actual media output.
   - The README highlights the main finishing operations: **filler-word removal**, **dead-space cuts**, **30 ms audio fades**, **subtitle burn-in**, **color grading**, and **overlay insertion**.
   - `helpers/grade.py` suggests grading is treated as a separate concern, likely reusable across segments or styles.

8. **Self-evaluation loop**
   - After rendering, the system re-inspects the output around each cut boundary using the same timeline-view approach.
   - This creates a **render → inspect → fix → re-render** loop, capped to a small number of retries, so problems like pops, jumps, or hidden captions are caught before the user sees the final result.

9. **Animation overlay subsystem**
   - The `skills/manim-video` subtree is a specialized sub-skill for generating animation overlays.
   - Its `references/` directory acts as a knowledge base for Manim-specific topics like scene planning, rendering, camera/3D, graphs, equations, and production quality.
   - This is a parallel capability, not the core edit pipeline: overlays can be generated by separate sub-agents and then composed into the main edit.

10. **Tooling and external dependencies**
    - The repo is Python-based (`pyproject.toml`) and expects **ffmpeg** for media processing.
    - `yt-dlp` is optional for downloading online sources.
    - `.env.example` indicates API-key based integration, with **ElevenLabs** explicitly required for transcription.
    - The repo is built to be used under different agent hosts, but the architecture assumes an LLM that can read docs, invoke shell commands, and manage iterative media workflows.

11. **Important architectural boundary**
    - The key design boundary is between **LLM reasoning** and **media execution**.
    - The LLM does not “watch” the whole video; it reasons from packed transcripts and selective timeline composites, then delegates concrete media operations to helper scripts and ffmpeg-based tooling.
    - That separation is what makes the system scalable across content types: talking heads, tutorials, travel, interviews, montages, and more.

Main entry points: [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) (render), [`skills/manim-video`](https://github.com/browser-use/video-use/tree/main/skills/manim-video) (manim-video), [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py) (transcribe), [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) (SKILL.md)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_root["Repo Surface"]
  node_node_skill["SKILL.md<br/>agent skill spec<br/>[SKILL.md]"]
  node_node_readme["README.md<br/>workflow guide<br/>[README.md]"]
  node_node_install["install.md<br/>setup guide<br/>[install.md]"]
end

subgraph group_group_pipeline["Edit Pipeline"]
  node_node_transcribe["transcribe<br/>ingest tool<br/>[transcribe.py]"]
  node_node_transcribe_batch["batch transcribe<br/>ingest orchestration"]
  node_node_pack["pack transcripts<br/>compression tool"]
  node_node_timeline["timeline view<br/>inspection tool<br/>[timeline_view.py]"]
  node_node_render["render<br/>media executor<br/>[render.py]"]
  node_node_grade["grade<br/>finishing tool<br/>[grade.py]"]
  node_node_output["final.mp4<br/>render output"]
  node_node_project_md["project.md<br/>session memory"]
  node_node_edit_dir[("edit folder<br/>working output")]
end

subgraph group_group_overlay["Overlay Skill"]
  node_node_manim_skill["manim-video<br/>sub-skill"]
  node_node_manim_skill_md["manim SKILL<br/>skill spec<br/>[SKILL.md]"]
  node_node_manim_readme["manim README<br/>skill guide<br/>[README.md]"]
  node_node_refs["reference notes<br/>knowledge base"]
  node_node_setup_sh["setup.sh<br/>bootstrap script<br/>[setup.sh]"]
end

subgraph group_group_support["Support"]
  node_node_env[".env.example<br/>env template<br/>[.env.example]"]
  node_node_pyproject["pyproject.toml<br/>python package<br/>[pyproject.toml]"]
  node_node_ffmpeg["ffmpeg<br/>media runtime"]
  node_node_elevenlabs(("ElevenLabs Scribe<br/>transcription api"))
  node_node_ytdlp(("yt-dlp<br/>optional downloader"))
end

node_node_skill -->|"orients"| node_node_readme
node_node_skill -->|"depends on"| node_node_install
node_node_skill -->|"drives"| node_node_transcribe
node_node_readme -->|"describes"| node_node_transcribe
node_node_install -->|"configures"| node_node_env
node_node_pyproject -->|"packages around"| node_node_ffmpeg
node_node_env -->|"authenticates"| node_node_elevenlabs
node_node_transcribe_batch -->|"coordinates"| node_node_transcribe
node_node_transcribe -->|"calls"| node_node_elevenlabs
node_node_transcribe -->|"feeds"| node_node_pack
node_node_pack -.->|"supplements"| node_node_timeline
node_node_timeline -->|"validates"| node_node_render
node_node_render -->|"invokes"| node_node_grade
node_node_render -->|"produces"| node_node_output
node_node_render -->|"writes to"| node_node_edit_dir
node_node_edit_dir -->|"stores"| node_node_project_md
node_node_timeline -->|"reads from"| node_node_edit_dir
node_node_manim_skill -->|"specified by"| node_node_manim_skill_md
node_node_manim_skill -->|"explained by"| node_node_manim_readme
node_node_manim_skill -->|"relies on"| node_node_refs
node_node_manim_skill -->|"bootstraps"| node_node_setup_sh
node_node_manim_skill -.->|"composes with"| node_node_render
node_node_ytdlp -.->|"optionally supplies"| node_node_transcribe_batch
node_node_render -->|"executes via"| node_node_ffmpeg

click node_node_skill "https://github.com/browser-use/video-use/blob/main/SKILL.md"
click node_node_readme "https://github.com/browser-use/video-use/blob/main/README.md"
click node_node_install "https://github.com/browser-use/video-use/blob/main/install.md"
click node_node_env "https://github.com/browser-use/video-use/blob/main/.env.example"
click node_node_pyproject "https://github.com/browser-use/video-use/blob/main/pyproject.toml"
click node_node_transcribe "https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py"
click node_node_transcribe_batch "https://github.com/browser-use/video-use/blob/main/helpers/transcribe_batch.py"
click node_node_pack "https://github.com/browser-use/video-use/blob/main/helpers/pack_transcripts.py"
click node_node_timeline "https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py"
click node_node_render "https://github.com/browser-use/video-use/blob/main/helpers/render.py"
click node_node_grade "https://github.com/browser-use/video-use/blob/main/helpers/grade.py"
click node_node_manim_skill "https://github.com/browser-use/video-use/tree/main/skills/manim-video"
click node_node_manim_skill_md "https://github.com/browser-use/video-use/blob/main/skills/manim-video/SKILL.md"
click node_node_manim_readme "https://github.com/browser-use/video-use/blob/main/skills/manim-video/README.md"
click node_node_refs "https://github.com/browser-use/video-use/tree/main/skills/manim-video/references"
click node_node_setup_sh "https://github.com/browser-use/video-use/blob/main/skills/manim-video/scripts/setup.sh"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_skill,node_node_readme,node_node_install toneBlue
class node_node_transcribe,node_node_transcribe_batch,node_node_pack,node_node_timeline,node_node_render,node_node_grade,node_node_output,node_node_project_md,node_node_edit_dir toneAmber
class node_node_manim_skill,node_node_manim_skill_md,node_node_manim_readme,node_node_refs,node_node_setup_sh toneMint
class node_node_env,node_node_pyproject,node_node_ffmpeg,node_node_elevenlabs,node_node_ytdlp toneRose
```

## Components

### Repo Surface

User-facing docs and repo-wide entry points that define the skill workflow.

- [SKILL.md](https://github.com/browser-use/video-use/blob/main/SKILL.md): agent skill spec · Primary operating instructions for the agent workflow. · `SKILL.md`
- [README.md](https://github.com/browser-use/video-use/blob/main/README.md): workflow guide · Explains the repository purpose and editing flow. · `README.md`
- [install.md](https://github.com/browser-use/video-use/blob/main/install.md): setup guide · Environment and setup instructions for using the skill. · `install.md`

### Edit Pipeline

Main agent-driven transcription, planning, rendering, and review loop.

- [transcribe](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py): ingest tool · Single-source transcription entry point with word-level timing. · `helpers/transcribe.py`
- [batch transcribe](https://github.com/browser-use/video-use/blob/main/helpers/transcribe_batch.py): ingest orchestration · Runs transcription across multiple source takes. · `helpers/transcribe_batch.py`
- [pack transcripts](https://github.com/browser-use/video-use/blob/main/helpers/pack_transcripts.py): compression tool · Condenses per-take transcripts into a compact agent-readable summary. · `helpers/pack_transcripts.py`
- [timeline view](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py): inspection tool · Builds filmstrip and waveform composites for targeted review windows. · `helpers/timeline_view.py`
- [render](https://github.com/browser-use/video-use/blob/main/helpers/render.py): media executor · Applies edit decisions and produces final media outputs. · `helpers/render.py`
- [grade](https://github.com/browser-use/video-use/blob/main/helpers/grade.py): finishing tool · Handles color grading as a separate finishing concern. · `helpers/grade.py`
- **final.mp4**: render output · Primary finished deliverable produced in the edit folder.
- **project.md**: session memory · Persistent project notes and working memory maintained by the agent.
- **edit folder**: working output · Destination for outputs and intermediates under the videos directory.

### Overlay Skill

Specialized Manim-based sub-skill for animation overlays and references.

- [manim-video](https://github.com/browser-use/video-use/tree/main/skills/manim-video): sub-skill · Specialized skill for generating animation overlays. · `skills/manim-video`
- [manim SKILL](https://github.com/browser-use/video-use/blob/main/skills/manim-video/SKILL.md): skill spec · Operating instructions for the Manim overlay sub-skill. · `skills/manim-video/SKILL.md`
- [manim README](https://github.com/browser-use/video-use/blob/main/skills/manim-video/README.md): skill guide · Overview of the overlay sub-skill and its usage. · `skills/manim-video/README.md`
- [reference notes](https://github.com/browser-use/video-use/tree/main/skills/manim-video/references): knowledge base · Topic-specific Manim guidance for planning, rendering, and production quality. · `skills/manim-video/references`
- [setup.sh](https://github.com/browser-use/video-use/blob/main/skills/manim-video/scripts/setup.sh): bootstrap script · Shell setup for the Manim sub-skill environment. · `skills/manim-video/scripts/setup.sh`

### Support

Project metadata, packaging, and install/dependency boundary.

- [.env.example](https://github.com/browser-use/video-use/blob/main/.env.example): env template · Template for API keys and runtime configuration. · `.env.example`
- [pyproject.toml](https://github.com/browser-use/video-use/blob/main/pyproject.toml): python package · Python project metadata and dependency definition. · `pyproject.toml`
- **ffmpeg**: media runtime · Core external tool used for media processing and rendering.
- **ElevenLabs Scribe**: transcription api · External transcription service providing timestamps, diarization, and audio events.
- **yt-dlp**: optional downloader · Optional source acquisition tool for online media.

## Connections

- SKILL.md → README.md: orients · The skill instructions are explained by the repository README.
- SKILL.md → install.md: depends on · Setup guidance is required before the workflow can run.
- SKILL.md → transcribe: drives · The agent uses transcription as the first operational step.
- README.md → transcribe: describes · Documentation frames transcription as the foundation for edit decisions.
- install.md → .env.example: configures · Installation expects environment variables and API credentials.
- pyproject.toml → ffmpeg: packages around · Python tooling wraps media work that ultimately relies on ffmpeg.
- .env.example → ElevenLabs Scribe: authenticates · The env template exposes API-key integration for transcription.
- batch transcribe → transcribe: coordinates · Batch transcription orchestrates repeated single-take ingestion.
- transcribe → ElevenLabs Scribe: calls · Transcription is backed by ElevenLabs Scribe.
- transcribe → pack transcripts: feeds · Raw transcript output is compressed into a compact reading surface.
- pack transcripts → timeline view: supplements · Packed text is complemented by targeted visual inspection windows.
- timeline view → render: validates · Timeline inspection is used before and after rendering decisions.
- render → grade: invokes · Finishing may route through a separate grading step.
- render → final.mp4: produces · Rendering writes the finished video deliverable.
- render → edit folder: writes to · Outputs are kept inside the edit working folder.
- edit folder → project.md: stores · The workflow preserves session memory alongside media artifacts.
- timeline view → edit folder: reads from · Inspection composites are built from working edit assets.
- manim-video → manim SKILL: specified by · The sub-skill is governed by its own instructions.
- manim-video → manim README: explained by · The sub-skill has dedicated usage documentation.
- manim-video → reference notes: relies on · Overlay generation is supported by curated reference material.
- manim-video → setup.sh: bootstraps · Setup script prepares the Manim environment.
- manim-video → render: composes with · Overlay artifacts are intended to be inserted into the main edit render.
- yt-dlp → batch transcribe: optionally supplies · Online sources can be fetched before batch transcription.
- render → ffmpeg: executes via · Media assembly and finishing are performed through ffmpeg-based tooling.