# hkuds/videoagent architecture

> Architecture diagram of the GitHub repository hkuds/videoagent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hkuds/videoagent
- Repository: https://github.com/hkuds/videoagent

## Overview

VideoAgent is a conversational, multi-agent toolkit for video understanding, editing, and creative remaking. The documented workflow starts with a natural-language request, maps intent to agent capabilities, and constructs an adaptive tool workflow; the runtime launcher hands control to the multi-agent framework. Major documented branches include Q&A and summarization, clip editing and beat synchronization, commentary and overview creation, and creative remakes such as memes, music videos, and cross-cultural comedy. The tree supports these role paths, and sampled VideoRAG code supports video ingestion and retrieval. Most role internals and dispatch details were not sampled, so those links reflect README-level documentation rather than inspected calls.

Main entry points: [`environment/agents/multi.py`](https://github.com/hkuds/videoagent/blob/main/environment/agents/multi.py) (Multi-agent framework), [`tools/videorag/videoragcontent.py`](https://github.com/hkuds/videoagent/blob/main/tools/videorag/videoragcontent.py) (Video RAG), [`environment/roles/vid_conversion.py`](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_conversion.py) (Video conversion), [`main.py`](https://github.com/hkuds/videoagent/blob/main/main.py) (Application launcher)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Agent orchestration"]
  node_launcher["Application launcher<br/>[main.py]"]
  node_multiagent["Multi-agent framework<br/>[multi.py]"]
  node_agentbase["Agent base<br/>[base.py]"]
  node_llmconfig["LLM configuration<br/>[llm.py]"]
end

subgraph group_understanding["Video understanding"]
  node_qa["Video Q&amp;A<br/>[content_loader.py]"]
  node_summary["Video summaries<br/>[summ_loader.py]"]
  node_preload["Video preparation<br/>[vid_preloader.py]"]
  node_search["Visual search<br/>[vid_searcher.py]"]
end

subgraph group_editing["Editing and storytelling"]
  node_editor["Clip editing<br/>[vid_editor.py]"]
  node_rhythm["Beat detection<br/>[rhythm_detector.py]"]
  node_rhythmstory["Beat-synced edits"]
  node_commentary["Commentary video<br/>[comm_story_gen.py]"]
  node_news["News storytelling<br/>[news_story_gen.py]"]
end

subgraph group_creation["Creative audio video"]
  node_conversion["Video conversion<br/>[vid_conversion.py]"]
  node_crosscomedy["Cross-cultural comedy"]
  node_standup["Stand-up remake"]
  node_voice["Voice generation<br/>[voice_generator.py]"]
  node_audio["Audio processing<br/>[mixer.py]"]
end

subgraph group_retrieval["Video retrieval"]
  node_videorag["Video RAG<br/>[videoragcontent.py]"]
  node_videoindex[("Video segment index")]
  node_videostorage[("Video metadata<br/>[kv_json.py]")]
end

node_user(("User"))

node_user -->|"starts conversation"| node_launcher
node_launcher -->|"starts framework"| node_multiagent
node_multiagent -->|"uses agent base"| node_agentbase
node_multiagent -.->|"dispatches by intent"| node_qa
node_multiagent -.->|"dispatches by intent"| node_summary
node_multiagent -.->|"dispatches by intent"| node_editor
node_multiagent -.->|"dispatches by intent"| node_rhythmstory
node_multiagent -.->|"dispatches by intent"| node_commentary
node_multiagent -.->|"dispatches by intent"| node_news
node_multiagent -.->|"dispatches by intent"| node_crosscomedy
node_multiagent -.->|"dispatches by intent"| node_standup
node_multiagent -.->|"retrieves video context"| node_videorag
node_videorag -->|"writes metadata"| node_videostorage
node_videorag -->|"indexes segments"| node_videoindex
node_videorag -->|"queries features"| node_videoindex
node_preload -.->|"supports retrieval"| node_search
node_rhythmstory -.->|"uses beat analysis"| node_rhythm
node_conversion -.->|"uses generated voice"| node_voice
node_conversion -.->|"processes audio"| node_audio

click node_launcher "https://github.com/hkuds/videoagent/blob/main/main.py"
click node_multiagent "https://github.com/hkuds/videoagent/blob/main/environment/agents/multi.py"
click node_agentbase "https://github.com/hkuds/videoagent/blob/main/environment/agents/base.py"
click node_llmconfig "https://github.com/hkuds/videoagent/blob/main/environment/config/llm.py"
click node_qa "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_qa/content_loader.py"
click node_summary "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_summ/summ_loader.py"
click node_preload "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_preloader.py"
click node_search "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_searcher.py"
click node_editor "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_editor.py"
click node_rhythm "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_rhythm/rhythm_detector.py"
click node_rhythmstory "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_rhythm/rhythm_story_gen.py"
click node_commentary "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_comm/comm_story_gen.py"
click node_news "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_news/news_story_gen.py"
click node_conversion "https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_conversion.py"
click node_crosscomedy "https://github.com/hkuds/videoagent/blob/main/environment/roles/cross_talk/cross_talk_adapter.py"
click node_standup "https://github.com/hkuds/videoagent/blob/main/environment/roles/stand_up/stand_up_adapter.py"
click node_voice "https://github.com/hkuds/videoagent/blob/main/environment/roles/voice_generator.py"
click node_audio "https://github.com/hkuds/videoagent/blob/main/environment/roles/mixer.py"
click node_videorag "https://github.com/hkuds/videoagent/blob/main/tools/videorag/videoragcontent.py"
click node_videoindex "https://github.com/hkuds/videoagent/blob/main/tools/videorag/_storage/vdb_nanovectordb.py"
click node_videostorage "https://github.com/hkuds/videoagent/blob/main/tools/videorag/_storage/kv_json.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_launcher,node_multiagent,node_agentbase,node_llmconfig,node_user toneBlue
class node_qa,node_summary,node_preload,node_search toneAmber
class node_editor,node_rhythm,node_rhythmstory,node_commentary,node_news toneMint
class node_conversion,node_crosscomedy,node_standup,node_voice,node_audio toneRose
class node_videorag,node_videoindex,node_videostorage toneIndigo
```

## Components

### Agent orchestration

- [Application launcher](https://github.com/hkuds/videoagent/blob/main/main.py): `main.py`
- [Multi-agent framework](https://github.com/hkuds/videoagent/blob/main/environment/agents/multi.py): `environment/agents/multi.py`
- [Agent base](https://github.com/hkuds/videoagent/blob/main/environment/agents/base.py): `environment/agents/base.py`
- [LLM configuration](https://github.com/hkuds/videoagent/blob/main/environment/config/llm.py): `environment/config/llm.py`

### Video understanding

- [Video Q&A](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_qa/content_loader.py): `environment/roles/vid_qa/content_loader.py`
- [Video summaries](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_summ/summ_loader.py): `environment/roles/vid_summ/summ_loader.py`
- [Video preparation](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_preloader.py): `environment/roles/vid_preloader.py`
- [Visual search](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_searcher.py): `environment/roles/vid_searcher.py`

### Editing and storytelling

- [Clip editing](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_editor.py): `environment/roles/vid_editor.py`
- [Beat detection](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_rhythm/rhythm_detector.py): `environment/roles/vid_rhythm/rhythm_detector.py`
- [Beat-synced edits](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_rhythm/rhythm_story_gen.py): `environment/roles/vid_rhythm/rhythm_story_gen.py`
- [Commentary video](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_comm/comm_story_gen.py): `environment/roles/vid_comm/comm_story_gen.py`
- [News storytelling](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_news/news_story_gen.py): `environment/roles/vid_news/news_story_gen.py`

### Creative audio video

- [Video conversion](https://github.com/hkuds/videoagent/blob/main/environment/roles/vid_conversion.py): `environment/roles/vid_conversion.py`
- [Cross-cultural comedy](https://github.com/hkuds/videoagent/blob/main/environment/roles/cross_talk/cross_talk_adapter.py): `environment/roles/cross_talk/cross_talk_adapter.py`
- [Stand-up remake](https://github.com/hkuds/videoagent/blob/main/environment/roles/stand_up/stand_up_adapter.py): `environment/roles/stand_up/stand_up_adapter.py`
- [Voice generation](https://github.com/hkuds/videoagent/blob/main/environment/roles/voice_generator.py): `environment/roles/voice_generator.py`
- [Audio processing](https://github.com/hkuds/videoagent/blob/main/environment/roles/mixer.py): `environment/roles/mixer.py`

### Video retrieval

- [Video RAG](https://github.com/hkuds/videoagent/blob/main/tools/videorag/videoragcontent.py): `tools/videorag/videoragcontent.py`
- [Video segment index](https://github.com/hkuds/videoagent/blob/main/tools/videorag/_storage/vdb_nanovectordb.py): `tools/videorag/_storage/vdb_nanovectordb.py`
- [Video metadata](https://github.com/hkuds/videoagent/blob/main/tools/videorag/_storage/kv_json.py): `tools/videorag/_storage/kv_json.py`

### Other components

- **User**

## Connections

- User → Application launcher: starts conversation
- Application launcher → Multi-agent framework: starts framework
- Multi-agent framework → Agent base: uses agent base
- Multi-agent framework → Video Q&A: dispatches by intent
- Multi-agent framework → Video summaries: dispatches by intent
- Multi-agent framework → Clip editing: dispatches by intent
- Multi-agent framework → Beat-synced edits: dispatches by intent
- Multi-agent framework → Commentary video: dispatches by intent
- Multi-agent framework → News storytelling: dispatches by intent
- Multi-agent framework → Cross-cultural comedy: dispatches by intent
- Multi-agent framework → Stand-up remake: dispatches by intent
- Multi-agent framework → Video RAG: retrieves video context
- Video RAG → Video metadata: writes metadata
- Video RAG → Video segment index: indexes segments
- Video RAG → Video segment index: queries features
- Video preparation → Visual search: supports retrieval
- Beat-synced edits → Beat detection: uses beat analysis
- Video conversion → Voice generation: uses generated voice
- Video conversion → Audio processing: processes audio