# tiger-ai-lab/theoremexplainagent architecture

> Architecture diagram of the GitHub repository tiger-ai-lab/theoremexplainagent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tiger-ai-lab/theoremexplainagent
- Repository: https://github.com/tiger-ai-lab/theoremexplainagent

## Overview

##### Project Type & Purpose
- TEA is an AI system focused on generating animated (Manim) videos that illustrate and explain mathematical and scientific theorems. Its purpose is to leverage large language models (LLMs) and other AI tools (e.g., Litellm, Gemini, Vertex AI) to provide “multimodal” explanations by integrating video rendering, voice-over (via Kokoro TTS), and code generation.
- The README and file tree indicate that the system handles video generation, evaluation (across text, video, image modalities), task generation with prompts, and integration with retrieval-augmented generation (RAG).

##### Analyzing the File Structure
- Top-level Directories:
- data/: Contains JSON files categorizing theorem data by difficulty (thb_easy, thb_medium, thb_hard) and subject (math, physics, chemistry, comp_sci).
- eval_suite/: Contains modules for evaluation (video, text, image modules, prompt parsing, and utilities).
- src/: Contains the core system logic:
▪ config/: Handles configuration (reads .env and configuration variables).
▪ core/: Contains the main modules responsible for code generation, video planning, parsing, and video rendering.
▪ rag/: Provides the integration of RAG functionality along with vector store management.
▪ utils/: Includes common utilities (e.g., allowed models list, Kokoro voiceover integration) used across the system.
- task_generator/: Manages prompt templates and task parsing for generating varied instructions – it stores raw prompt files and the parser that compiles them.
- mllm_tools/: Contains tools to interface with external LLM APIs (Gemini, LiteLLM, Vertex AI, etc.).
- Root-level scripts:
- generate_video.py: Entry point for video generation using selected models, topics, context, and (optionally) RAG integration.
- evaluate.py: Handles automated evaluation by invoking different modalities (text, video, image) evaluation routines.
- Supporting files include configuration (.env.template), requirements for Python packages, and GitHub-related files (CI templates, etc.).

##### Key Components to Highlight
a. Input/Configuration:
- .env & .env.template: Store external API keys and settings.
- Configuration files in src/config enable centralized control over critical settings.
b. Data Layer:
- data/: The repository for theorem datasets (segregated by difficulty and subjects).
- External data sources for RAG (manim_docs, vector DB, etc.) when used.
c. Core Generation Pipeline:
- src/core/:
▪ code_generator.py: Responsible for generating the “scene” codes/instructions based on the provided topic and context.
▪ video_planner.py & parse_video.py: Component modules that plan and sequence the video scenes.
▪ video_renderer.py: Takes the planned scenes and renders/combines them into a final video.
d. Retrieval & Augmentation:
- src/rag: Implements retrieval-augmented generation – connects to a vector store (vector_store.py) and integrates RAG queries (rag_integration.py) to enhance content generation.
e. Task & Prompt Generation:
- The parse_prompt.py script compiles and updates prompt definitions.
f. External LLM & Model Tools:
- mllm_tools/: Acts as an abstraction layer for interfacing with various AI models (Gemini, LiteLLM, Vertex AI).
g. Evaluation Module:
- eval_suite/: Comprises utilities for evaluating the generated videos’ quality (image_utils, text_utils, video_utils) and has its own prompt parsing logic.
h. Additional Utilities:
- src/utils/ and eval_suite/utils.py: Provide common functionalities such as allowed models check, Kokoro TTS voiceover integration, and miscellaneous helper functions.

##### Depicting Relationships & Data Flow
- Input Stage:
- User/Cli provides a “topic” and “context” to generate_video.py.
- Environment variables (from .env) configure API access and settings.
- Generation Pipeline:
- The input is processed through the core generation components (code_generator, video_planner, video_renderer) in src/core, invoking configuration settings from src/config.
- If Retrieval Augmented Generation (RAG) is enabled, the src/rag components (rag_integration, vector_store) fetch and integrate external documentation.
- Task Generation:
- The task_generator uses prompt templates (under task_generator/prompts_raw) to generate varied tasks for scene planning and code generation.
- External AI Services:
- mllm_tools interacts with various external AI models using APIs based on settings in .env ensuring the candidate models (OpenAI, Gemini, Vertex AI) are used properly.
- Voiceover & Rendering:
- The voiceover service (Kokoro TTS, called via src/utils/kokoro_voiceover.py) integrates audio with video rendering.
- Evaluation:
- After generation, evaluate.py and modules in eval_suite evaluate the final video (and supporting SRT subtitle files) using text, video, or image evaluation flows.

##### Architectural Design Principles & Patterns
- Modularity: The project is modular – separate directories for core generation, evaluation, task generation, and model interfacing increase maintainability.
- Pipeline Architecture: The generation of videos follows a clear pipeline (input → planning → rendering → evaluation), with each component passing outputs to the next.
- Extensibility & Configurability: Use of a configuration layer (src/config) and external prompts (task_generator/prompts_raw) that can be altered easily without affecting the rest of the code.
- External Integration: The design accommodates multiple external AI providers, with mllm_tools abstracting these differences.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input Layer
    U["User Input<br/>(Topic & Context)"]:::input
    GV["generate_video.py"]:::input
    ES["evaluate.py"]:::input

    %% Configuration and Data
    CONF["Configuration<br/>(.env, src/config)"]:::config
    DATA["Data Repository<br/>(data/)"]:::data

    %% Core Generation Pipeline
    subgraph "Core Generation Pipeline"
        CG["Code Generation<br/>(src/core/code_generator.py)"]:::core
        VP["Video Planning<br/>(src/core/video_planner.py, src/core/parse_video.py)"]:::core
        VR["Video Rendering<br/>(src/core/video_renderer.py)"]:::core
    end

    %% Retrieval & Augmentation (RAG)
    RAG["RAG Integration<br/>(src/rag/rag_integration.py,<br/>src/rag/vector_store.py)"]:::rag

    %% Task & Prompt Generation
    TASK["Task & Prompt Generation<br/>(task_generator)"]:::task

    %% External LLM & Model Tools
    LLM["LLM & Model Tools<br/>(mllm_tools)"]:::ai

    %% Voiceover & Utilities
    VOX["Voiceover & Utilities<br/>(src/utils)"]:::voice

    %% Evaluation Module
    EVAL["Evaluation Module<br/>(eval_suite)"]:::eval

    %% Connections
    U -->|"provides data"| GV
    GV -->|"reads configuration"| CONF
    CONF -->|"configures processing"| CG
    CONF -->|"fetches theorem data"| DATA

    %% Core Pipeline Flow
    GV -->|"triggers generation"| CG
    CG -->|"sends code/instructions"| VP
    VP -->|"plans scenes"| VR
    VR -->|"integrates TTS audio"| VOX
    VOX -->|"produces final video"| EVAL

    %% Cross Module Integrations
    TASK -->|"supplies prompt templates"| CG
    TASK -->|"guides scene planning"| VP
    CG -->|"augments with retrieval"| RAG
    VP --> RAG
    LLM -->|"supports AI generation"| CG
    LLM -->|"supports task generation"| TASK

    %% Evaluation Script
    ES -->|"evaluates output"| EVAL

    %% Click Events for Configuration
    click CONF "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/.env.template"
    click CONF "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/src/config/config.py"

    %% Click Events for Data Layer
    click DATA "https://github.com/tiger-ai-lab/theoremexplainagent/tree/main/data/"

    %% Click Events for Core Generation Pipeline – Code Generation
    click CG "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/src/core/code_generator.py"

    %% Click Events for Video Planning
    click VP "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/src/core/video_planner.py"
    click VP "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/src/core/parse_video.py"

    %% Click Events for Video Rendering
    click VR "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/src/core/video_renderer.py"

    %% Click Events for RAG Integration
    click RAG "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/src/rag/rag_integration.py"
    click RAG "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/src/rag/vector_store.py"

    %% Click Events for Task & Prompt Generation
    click TASK "https://github.com/tiger-ai-lab/theoremexplainagent/tree/main/task_generator/"
    click TASK "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/task_generator/parse_prompt.py"
    click TASK "https://github.com/tiger-ai-lab/theoremexplainagent/tree/main/task_generator/prompts_raw/"

    %% Click Events for External LLM & Model Tools
    click LLM "https://github.com/tiger-ai-lab/theoremexplainagent/tree/main/mllm_tools/"
    click LLM "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/mllm_tools/gemini.py"
    click LLM "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/mllm_tools/litellm.py"
    click LLM "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/mllm_tools/vertex_ai.py"

    %% Click Events for Evaluation Module
    click EVAL "https://github.com/tiger-ai-lab/theoremexplainagent/tree/main/eval_suite/"
    click EVAL "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/eval_suite/image_utils.py"
    click EVAL "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/eval_suite/text_utils.py"
    click EVAL "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/eval_suite/video_utils.py"
    click EVAL "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/eval_suite/parse_prompt.py"

    %% Click Events for Voiceover & Utilities
    click VOX "https://github.com/tiger-ai-lab/theoremexplainagent/tree/main/src/utils/"
    click VOX "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/src/utils/kokoro_voiceover.py"
    click VOX "https://github.com/tiger-ai-lab/theoremexplainagent/blob/main/src/utils/allowed_models.json"

    %% Styles
    classDef input fill:#FFD580,stroke:#333,stroke-width:2px;
    classDef config fill:#B3E5FC,stroke:#333,stroke-width:2px;
    classDef data fill:#C8E6C9,stroke:#333,stroke-width:2px;
    classDef core fill:#FFF59D,stroke:#333,stroke-width:2px;
    classDef rag fill:#FFCC80,stroke:#333,stroke-width:2px;
    classDef task fill:#D1C4E9,stroke:#333,stroke-width:2px;
    classDef ai fill:#B2EBF2,stroke:#333,stroke-width:2px;
    classDef voice fill:#FFE0B2,stroke:#333,stroke-width:2px;
    classDef eval fill:#E1BEE7,stroke:#333,stroke-width:2px;
```
