How zai-org/glm-v works
GLM-V is a vision-language model project whose model algorithms are implemented in external Transformers code; this tree chiefly contains inference interfaces, a reward system, and specialized skills. A user-facing workflow is to submit a multimodal request through an inference interface or skill and receive a generated result. The reward subsystem evaluates model answers against ground truth through task-specific verifiers, with optional LLM judging. The README documents grounding and GUI-agent capabilities. Some skill and inference relationships are unclear from the sampled sources, so those connections are omitted.
Main entry points: reward_system.py Reward orchestration · glm_grounding_cli.py Grounding workflow · __init__.py Verifier registry · verifiers Domain verifiers
Components 30
Inference interfaces
- Inference CLI
inference/trans_infer_cli.py - Inference UI
inference/trans_infer_gradio.py - Inference benchmark
inference/trans_infer_bench.py - HTML detector
inference/html_detector.py
Reward evaluation
- Reward orchestration
glmv_reward/src/glmv_reward/reward_system.py - Reward configuration
glmv_reward/src/glmv_reward/configs/reward_system.py - Verifier configs
glmv_reward/src/glmv_reward/configs/verifiers - Verifier registry
glmv_reward/src/glmv_reward/verifiers/__init__.py - Verifier base
glmv_reward/src/glmv_reward/verifiers/_base_verifier.py - Domain verifiers
glmv_reward/src/glmv_reward/verifiers - Text utilities
glmv_reward/src/glmv_reward/utils/text.py - LLM judge client
glmv_reward/src/glmv_reward/utils/llm.py
Vision skills
- Grounding workflow
skills/glmv-grounding/scripts/glm_grounding_cli.py - Box utilities
skills/glmv-grounding/scripts/utils_boxes.py - Detection utilities
skills/glmv-grounding/scripts/utils_detection.py - Video utilities
skills/glmv-grounding/scripts/utils_video.py - Image captioning
skills/glmv-caption/scripts/glmv_caption.py - Android GUI agent
glmv_reward/scripts/gui_agent/AndroidWorld.py - OS GUI agent
glmv_reward/scripts/gui_agent/OSWorld.py - Web GUI agent
glmv_reward/scripts/gui_agent/WebVoyager.py
Task skills
- Document writing
skills/glmv-doc-based-writing/scripts/doc_based_writing.py - PDF to slides
skills/glmv-pdf-to-ppt/scripts/generate_slide.py - PDF to web
skills/glmv-pdf-to-web/scripts/generate_web.py - PRD to app
skills/glmv-prd-to-app/scripts/render_page.py - Prompt generation
skills/glmv-prompt-gen/scripts/prompt_gen.py - Resume screening
skills/glmv-resume-screen/scripts/resume_screen.py - Stock analysis
skills/glmv-stock-analyst/scripts/fetch_all.py - Text charts
skills/glmv-stock-analyst/scripts/text_chart.py
Other components
- User
- GLM-V service
Connections 22
- User → Inference CLI: submits request
- User → Inference UI: submits request
- Inference CLI → GLM-V service: requests inference
- Inference UI → GLM-V service: requests inference
- Inference benchmark → GLM-V service: benchmarks inference
- User → Grounding workflow: requests grounding
- User → Android GUI agent: runs GUI task
- User → OS GUI agent: runs GUI task
- User → Web GUI agent: runs GUI task
- Grounding workflow → Box utilities: uses
skills/glmv-grounding/scripts/glm_grounding_cli.py - Grounding workflow → Detection utilities: uses
skills/glmv-grounding/scripts/glm_grounding_cli.py - Grounding workflow → Video utilities: uses
skills/glmv-grounding/scripts/glm_grounding_cli.py - Stock analysis → Text charts: uses
skills/glmv-stock-analyst/scripts/fetch_all.py - User → Reward orchestration: requests scoring
- Reward orchestration → Reward configuration: loads config
glmv_reward/src/glmv_reward/reward_system.py - Reward orchestration → Verifier configs: resolves settings
glmv_reward/src/glmv_reward/reward_system.py - Reward orchestration → Verifier registry: gets verifier
glmv_reward/src/glmv_reward/reward_system.py - Verifier registry → Verifier base: uses base
glmv_reward/src/glmv_reward/verifiers/__init__.py - Verifier registry → Domain verifiers: registers
glmv_reward/src/glmv_reward/verifiers/__init__.py - Domain verifiers → Text utilities: extracts answers
glmv_reward/src/glmv_reward/verifiers/chart_verifier.py - Domain verifiers → LLM judge client: requests fallback
glmv_reward/src/glmv_reward/verifiers/math_verifier.py - Reward configuration → Verifier configs: types configs
glmv_reward/src/glmv_reward/configs/reward_system.py