How ianhuang0630/blenderalchemyofficial works
Project Type and Purpose
• This is a specialized “pipeline” tool that automates Blender scene editing via Vision-Language Models (VLMs).
• It orchestrates an iterative edit generator & state evaluator (both LLM-powered) to progressively refine Blender scripts.
Main Components
A. Runner & Orchestration
• main.py: CLI entry point
• refinement_process.py: core loop coordinating edit generation, rendering, evaluation, and selection
B. Configuration & Inputs
• configs/*.yaml: run parameters, prompts, model selections, API key paths
• starter_blends/*.blend: initial Blender scene files
• blender_scripts/*: editable Blender-Python starter scripts by task (materials, lighting, shapekeys, geonodes)
• blender_base/*.py: “rendering scripts” that exec the editable script then invoke Blender render
C. Agents & Prompting
• agents.py: wrapper for edit generator and state evaluator agents
• prompting/*: modules constructing LLM prompts for geonodes, lighting, material, placement, shapekey
D. Execution Environment
• Infinigen-enabled Blender binary (external tool)
• LLM APIs (OpenAI GPT-4V, Claude, Gemini, or local Ollama models)
E. Data & Utilities
• utils/*: code for reading/writing records, images, experiment logs
• metrics.py: evaluation metrics
• output/ (generated at runtime): per-instance directories storing renders, scripts, thought_process JSON
F. External Services
• LLM API endpoints
• (Optional) image-generation model for “imagined” reference images
Relationships & Data Flow
1. User invokes main.py with arguments → loads configs, maps blender_base & blender_script paths
2. refinement_process instantiates agents (edit generator & state evaluator)
3. For each iteration:
a. Edit generator sends current state + prompt → LLM API → candidate Python edits
b. Each candidate edit is saved and fed to blender_base script → Blender binary → rendered image
c. State evaluator sends rendered images + prompt → LLM API → evaluation scores
d. refinement_process selects best candidate → updates “current state” script
4. Iterate until stopping criterion → final output folder with best_of.png and full logs
Architectural Patterns & Principles
• Pipeline pattern with clearly separated stages (generate → render → evaluate)
• Modular “agent” abstraction encapsulating LLM interactions
• Pluggable scripts: users can swap in different blender_scripts or blender_base for new tasks
• Configuration-driven: no hard-coded hyperparameters or model choices