How boyuanvisionary/pro-cad works
Purpose and Entry Points
ProCAD is a research pipeline for robust text-to-CAD generation. It identifies ambiguous or contradictory CAD requests, obtains a clarification, generates CadQuery code, executes/evaluates the result, and reports geometry and question-quality metrics. The primary entry point is pipeline.py; dataset construction and analysis flows enter through generate_misleading.py, to_sft.py, batch_label.py, and analyze_results.py.
Clarification-to-Code Orchestration
pipeline.py coordinates batched processing of misleading-prompt samples: load prompt/ground truth, invoke the clarifier, simulate a user answer, synthesize corrected CadQuery, then evaluate against target meshes. Runtime behavior is controlled by environment-selected model IDs, dataset locations, and output directories.
• pipeline.py
• src/ask_agent.py
• src/user.py
Model Inference Boundary
All LLM access is centralized behind a unified inference layer. It supports remote OpenAI-compatible providers for clarification, simulated answering, labeling, and judging, plus local Hugging Face transformer inference for the released Coder checkpoint. Prompt contracts for these roles are separated from orchestration.
• src/inference.py
• config/clarification.py
• config/code_generation.py
Clarifier and User Interaction
The Clarifier detects underspecification or direct conflicts and emits targeted questions rather than immediately generating CAD. The simulated-user stage answers according to configurable expertise/persona, producing a corrected natural-language specification consumed by the Coder.
• src/ask_agent.py
• src/user.py
• config/misleading_prompt.py
CAD Synthesis and Geometry Execution
The Coder turns clarified descriptions into executable CadQuery. Generated code is converted to mesh form for comparison with DeepCAD-derived ground truth; this is a critical execution boundary because LLM output becomes CAD code and geometry artifacts.
• config/code_generation.py
• src/mesh_utils.py
• src/preprocessing.py
Data, Perturbation, and SFT Preparation
Dataset loaders expose Text2CAD/DeepCAD identities, descriptions, code, and meshes. Misleading-data generation creates under-specified or conflicting variants; leakage checking prevents modified prompts from exposing raw code. SFT conversion produces conversation-format data for the two model roles.
• src/data_loader.py
• generate_misleading.py
• to_sft.py
Evaluation and Result Analysis
Geometric success is measured through mesh/point-cloud Chamfer distance against STL ground truth. Additional LLM-assisted metrics judge clarification-question quality and ambiguity resolution. Batch labeling supports broader split-level experiments; analysis aggregates rates and metric statistics.
• src/evaluation.py
• src/processing.py
• analyze_results.py
Runtime, Infrastructure, and External Dependencies
The reproducible GPU environment is defined by cad-agent.def, with Apptainer/Singularity and CadQuery execution support. External dependencies include Hugging Face-hosted checkpoints/dataset, DeepCAD meshes and repository data, Text2CAD splits, LLaMA-Factory for SFT, and OpenAI/Anthropic/OpenRouter-compatible LLM APIs. Training role definitions are supplied as LLaMA-Factory configurations.
• cad-agent.def
• training/configs/qwen2-5_full_sft.yaml
• training/configs/qwen2-5_full_sft_misleading.yaml
Components 20
Runtime Pipeline
- Pipeline · Runs clarification, response simulation, CAD synthesis, and evaluation.
pipeline.py - CAD Execution · Turns generated code into mesh artifacts.
src/mesh_utils.py - Geometry Preprocessing
src/preprocessing.py - GPU CAD Environment · Defines reproducible GPU and CadQuery execution runtime.
cad-agent.def
Model Roles
- Clarifier · Detects ambiguity and asks targeted questions.
src/ask_agent.py - Simulated User · Answers questions using a configured persona.
src/user.py - Unified Inference · Routes remote API and local transformer inference.
src/inference.py - Clarification Prompts
config/clarification.py - Code Generation Prompts
config/code_generation.py - Misleading Prompt Config
config/misleading_prompt.py
Data Preparation
- Text2CAD / DeepCAD Loader · Exposes descriptions, code, identities, and ground-truth meshes.
src/data_loader.py - Misleading Data Generator · Creates underspecified and conflicting prompt variants.
generate_misleading.py - SFT Converter · Builds role-specific conversation training data.
to_sft.py
Evaluation
- Geometry Evaluation · Computes mesh and point-cloud Chamfer distance.
src/evaluation.py - Metric Processing · Processes geometry and LLM-assisted quality metrics.
src/processing.py - Batch Labeling · Labels broader dataset splits with LLM-assisted judgments.
batch_label.py - Result Analysis · Aggregates success rates and metric statistics.
analyze_results.py
Training
- LLaMA-Factory SFT Configs · Defines full-SFT roles for base and misleading data.
training/configs/qwen2-5_full_sft.yaml
Other components
- LLM Providers & Checkpoints · Remote compatible APIs and local Hugging Face Coder checkpoints.
- CAD Datasets & Meshes · Text2CAD splits, DeepCAD data, and STL ground truth.
Connections 24
- Pipeline → Text2CAD / DeepCAD Loader: loads samples
- Pipeline → Clarifier: requests clarification
- Clarifier → Unified Inference: infers via
- Clarification Prompts → Clarifier: defines questions
- Misleading Prompt Config → Simulated User: sets persona
- Clarifier → Simulated User: asks question
- Simulated User → Unified Inference: generates answer via
- Simulated User → Pipeline: corrected specification
- Code Generation Prompts → Pipeline: guides synthesis
- Pipeline → CAD Execution: executes generated CAD
- CAD Execution → Geometry Preprocessing: produces meshes
- Geometry Preprocessing → Geometry Evaluation: generated geometry
- Text2CAD / DeepCAD Loader → Geometry Evaluation: target meshes
- Geometry Evaluation → Metric Processing: geometry metrics
- Metric Processing → Result Analysis: aggregates results
- Batch Labeling → Metric Processing: quality labels
- Text2CAD / DeepCAD Loader → Misleading Data Generator: source prompts
- Misleading Data Generator → Pipeline: misleading samples
- Misleading Data Generator → SFT Converter: training examples
- SFT Converter → LLaMA-Factory SFT Configs: SFT data
- Unified Inference → LLM Providers & Checkpoints: uses
- Text2CAD / DeepCAD Loader → CAD Datasets & Meshes: loads
- GPU CAD Environment → Pipeline: executes
- GPU CAD Environment → CAD Execution: provides runtime