How nicglazkov/chalkboard works
1. What the system is
- Chalkboard is a Python CLI that turns a topic plus optional source material into a narrated Manim video.
- The core output path is: topic/context → scripted explainer → fact validation → Manim scene code → code validation → TTS audio → Docker render → merged final.mp4.
2. Entry point and configuration boundary
- main.py is the CLI entry point.
- config.py centralizes environment-driven settings such as model/API keys, default audience/tone/theme/effort, output directory, checkpoint DB path, and TTS backend selection.
- This keeps runtime configuration separate from the pipeline logic and allows the same pipeline to run in interactive, resumed, or scripted modes.
3. Pipeline orchestration layer
- pipeline/graph.py defines the LangGraph state machine that coordinates the whole run.
- pipeline/state.py holds the shared pipeline state and validation result types passed between stages.
- pipeline/agents/orchestrator.py manages stage transitions, retries, and when the system should ask for user guidance after repeated failures.
4. Content generation and validation stages
- pipeline/agents/script_agent.py creates the educational narration script.
- pipeline/agents/fact_validator.py checks the script’s claims for correctness.
- pipeline/agents/manim_agent.py turns the approved script into Manim animation code.
- pipeline/agents/code_validator.py checks the generated code before rendering.
- These are distinct agent boundaries: one produces content, the next approves or rejects it, and the next stage depends on that approval.
5. External model dependency
- The repository is built around Claude/Anthropic for the agentic text-generation and validation steps.
- The README’s “automatic retry logic” and “ask you for guidance” behavior imply the pipeline is resilient to model failure, but still human-in-the-loop when it can’t converge.
6. Context ingestion subsystem
- pipeline/context.py collects and normalizes external source material from local files, directories, URLs, and GitHub README content.
- This context is injected before generation so the script can be grounded in existing material, not just the topic prompt.
- The README notes that context is not stored in checkpoints, so it must be re-supplied when resuming a run.
7. Rendering boundary
- pipeline/render_trigger.py is the bridge from “AI pipeline output” to “actual video artifact.”
- It writes output files for rendering, invokes TTS, and coordinates the Docker/ffmpeg-based render/merge flow.
- This is an important system boundary: the AI pipeline is pure-ish Python state, while rendering is an isolated media build step.
8. Dockerized Manim execution
- docker/Dockerfile builds on manimcommunity/manim:v0.20.1.
- docker/render.sh runs the scene render inside the container.
- Docker is used to isolate Manim and ffmpeg rendering dependencies from the host environment, which matters for reproducibility and for the preview/full-render workflow.
9. Text-to-speech subsystem
- pipeline/tts/base.py defines the TTS abstraction.
- pipeline/tts/openai_tts.py, pipeline/tts/elevenlabs_tts.py, and pipeline/tts/kokoro_tts.py are interchangeable backends.
- The backend choice is an architectural seam: cloud APIs (openai, elevenlabs) versus local inference (kokoro), with speed adjustment handled per backend or via post-processing.
10. Retries, timeouts, and recovery
- pipeline/retry.py contains retry/time-limit logic used by the agents and render flow.
- The system supports resumable runs via --run-id, with checkpoints stored in pipeline_state.db.
- This means the pipeline is designed as a long-running, failure-tolerant workflow rather than a single-shot command.
11. Visual QA feedback loop
- pipeline/visual_qa.py samples frames from the rendered video and asks Claude to detect visual defects like overlap, off-screen text, or readability problems.
- If QA finds issues, Chalkboard can regenerate the Manim scene and re-render, up to a small number of attempts.
- This makes the render stage self-correcting and closes the loop between generated code and visual output.
12. Output artifacts and post-processing
- Outputs live under output/<run-id>/.
- Key artifacts include the rendered video, captions.srt, chapter markers embedded into the MP4, and optional quiz.json.
- Features like --preview, --burn-captions, --quiz, and --no-render are not separate systems; they are different paths through the same pipeline and output boundary.
13. Test and documentation structure
- tests/ mirrors the main modules, with focused tests for agents, retries, context loading, TTS backends, rendering, and visual QA.
- docs/ contains planning/spec documents for major pipeline changes, which suggests the architecture evolved through staged design work and has clear subsystem ownership.
14. How to draw the architecture
- A good diagram would show four layers: CLI/config, LangGraph agent pipeline, render/TTS infrastructure, and output artifacts.
- The central flows are: topic/context into script generation, script into validation and Manim generation, approved scene code into Docker render, audio into merge, and rendered frames back into visual QA with a possible regeneration loop.
Main entry points: render_trigger.py Render Trigger · orchestrator.py Orchestrator · base.py TTS API · visual_qa.py Visual QA
Components 20
CLI & Config
Entry point and runtime settings boundary.
Pipeline Core
LangGraph state machine, shared state, and orchestration.
- Graph · Defines the pipeline state machine and stage wiring.
pipeline/graph.py - State · Pipeline state and validation result types passed between stages.
pipeline/state.py - Orchestrator · Manages stage transitions, retries, and human guidance prompts.
pipeline/agents/orchestrator.py - Retries · Timeout and retry utilities shared by agents and render flow.
pipeline/retry.py
Content Pipeline
Topic ingestion, script generation, and validation stages.
- Context · Loads and normalizes source material from files, URLs, directories, and GitHub README content.
pipeline/context.py - Script · Creates the educational narration script from topic and context.
pipeline/agents/script_agent.py - Fact Check · Checks script claims for correctness.
pipeline/agents/fact_validator.py - Manim Code · Turns the approved script into Manim animation code.
pipeline/agents/manim_agent.py - Code Check · Validates generated Manim code before rendering.
pipeline/agents/code_validator.py
Render & QA
TTS, Docker rendering, and visual feedback loop.
- Render Trigger · Coordinates TTS, scene writing, Docker render, and media merge.
pipeline/render_trigger.py - Visual QA · Samples rendered frames and checks for visual defects.
pipeline/visual_qa.py - TTS API · Common interface for text-to-speech backends.
pipeline/tts/base.py - OpenAI TTS · Cloud text-to-speech backend.
pipeline/tts/openai_tts.py - ElevenLabs TTS · Cloud text-to-speech backend.
pipeline/tts/elevenlabs_tts.py - Kokoro TTS · Local text-to-speech backend.
pipeline/tts/kokoro_tts.py - Docker image · Manim/ffmpeg container image for isolated rendering.
docker/Dockerfile - Render script · Container entry script for rendering the scene.
docker/render.sh
Artifacts
Primary output files and run directory.
- Run output · Per-run output directory containing rendered media and sidecar files.
output
Connections 22
- CLI → Config: loads · CLI reads runtime configuration.
- CLI → Graph: starts · CLI invokes the pipeline graph.
- Graph → Orchestrator: drives · Graph delegates control to the orchestrator.
- Orchestrator → State: updates · Stages share and mutate pipeline state.
- Orchestrator → Retries: uses · Retry and timeout logic protects stage execution.
- Orchestrator → Context: ingests · Context is collected before generation.
- Context → Script: feeds · Normalized source material grounds script generation.
- Script → Fact Check: checks · Generated narration is validated for factual correctness.
- Fact Check → Manim Code: approves · Validated script advances to animation code generation.
- Manim Code → Code Check: checks · Generated Manim code is validated before render.
- Code Check → Render Trigger: releases · Approved code enters the render boundary.
- Render Trigger → TTS API: invokes · Render flow requests narration audio through the TTS abstraction.
- TTS API → OpenAI TTS: implements · One selectable TTS backend.
- TTS API → ElevenLabs TTS: implements · One selectable TTS backend.
- TTS API → Kokoro TTS: implements · One selectable TTS backend.
- Render Trigger → Docker image: builds · Manim rendering runs in an isolated container image.
- Render Trigger → Render script: runs · Containerized render script executes the scene build.
- Render Trigger → Run output: writes · Rendered media and sidecar files are emitted per run.
- Render Trigger → Visual QA: samples · Rendered output is inspected for visual defects.
- Visual QA → Manim Code: regenerates · QA failures can trigger a new scene generation pass.
- Visual QA → Render Trigger: rerenders · QA failures can trigger another render attempt.
- Retries → Orchestrator: guards · Failures can route control back through retries and recovery.