How ylingfeng/avt works
Project Type and Purpose
• Type: A command-line, data-processing pipeline rather than a web/service application.
• Purpose: Given raw video footage, run a sequence of stages—sectioning, captioning, story composition, video assembly, evaluation—using LLM agents and heuristics to produce trimmed, coherent highlight reels.
File Structure
• Top‐level directories:
– tools/: individual Python scripts (get_section_data.py, get_caption.py, get_story.py, get_video.py, get_evaluation.py) each implements one pipeline stage.
– config/: YAML configuration for each tool, driving parameters, file paths, LLM prompts.
– llms/: adapters for external LLM providers (Gemini, GPT).
– utils/: shared helpers (media I/O, parsing, formatting, YouTube download, metrics).
– data/: input datasets and all intermediate/final outputs organized under data/avt and other corpora.
README for Architectural Insights
• Sequential stages are clearly defined:
1) section data extraction
2) caption generation via LLM
3) narrative/story composition via LLM
4) video assembly (cut & concatenate)
5) evaluation/metrics
• Data flows between stages under data/avt (caption/, section_data/, output/{caption,story,evaluation}).
• External dependencies: LLM APIs (Gemini, OpenAI), YouTube/video files (via utils/youtube.py), ffmpeg or similar (implied by video tools).
Main Components & Their Interactions
a. Components
• CLI Orchestrator (shell commands invoking tools/*.py)
• Configuration Manager (YAML loader in each tool)
• Pipeline Stage Modules (one script per stage under tools/)
• LLM Adapter Layer (llms/gpt.py, llms/gemini.py calling external APIs)
• Utils Library (media I/O, parsing, video processing, printing)
• Data Store (local file system under data/)
• External Services (LLM endpoints, YouTube/video hosting)
b. Relationships & Data Flow
• CLI → Tool Script → Config Loader + Utils + LLM Adapters
• Tool Script reads raw/intermediate files from data/avt → processes → writes outputs back to data/avt
• LLM Adapters call external LLM API → return structured captions or story segments
• Utils/video modules (ffmpeg wrappers) assemble final .mp4 outputs
• Evaluation script reads outputs → computes metrics → writes to data/avt/output/evaluation
c. Architectural Patterns & Principles
• Pipeline pattern with clearly separated stages
• Single-responsibility modules: each tool does one logical step
• Config-driven behavior: YAML files decouple parameters from code
• Adapter pattern for LLM services (llms/ folder)
• Utility library for cross-cutting concerns (utils/)
d. Key Technologies & Libraries
• Python 3.x for scripting
• YAML for configuration (PyYAML)
• External LLM APIs (OpenAI, Gemini)
• ffmpeg or moviepy (via utils/video.py)
• youtube-dl or similar (via utils/youtube.py)
Detail
• Every stage’s inputs, outputs, and dependencies.
• That each tool imports from llms/ and utils/.
• Adapter pattern in llms/ wrapping external API calls.
• Config files driving each module.