# microsoft/cadfusion architecture

> Architecture diagram of the GitHub repository microsoft/cadfusion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/cadfusion
- Repository: https://github.com/microsoft/cadfusion

## Overview

CADFusion turns text descriptions into CAD command sequences, then improves generation through an alternating visual-feedback loop. The documented workflow prepares and renders CAD data, captions it for sequential learning, generates candidate models, evaluates their rendered images, builds preference pairs, and fine-tunes again. The graph also includes CAD-to-solid conversion and evaluation capabilities. Several links follow the README’s documented workflow; unsampled implementation details are not inferred beyond those statements.

Main entry points: [`src/dpo/make_dpo_dataset.py`](https://github.com/microsoft/cadfusion/blob/main/src/dpo/make_dpo_dataset.py) (Visual scoring), [`src/rendering_utils/parser_visual.py`](https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/parser_visual.py) (Solid conversion), [`src/rendering_utils/parser.py`](https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/parser.py) (Sequence parser), [`src/data_preprocessing/convert.py`](https://github.com/microsoft/cadfusion/blob/main/src/data_preprocessing/convert.py) (SL conversion)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_preparation["Data preparation"]
  node_sl_conversion["SL conversion<br/>[convert.py]"]
  node_provided_conversion["Provided data conversion<br/>[convert.py]"]
  node_captioning["LLM captioning<br/>[captioning.py]"]
  node_openai_client["Azure API client<br/>[call_openai.py]"]
end

subgraph group_cad["CAD rendering"]
  node_sequence_parser["Sequence parser<br/>[parser.py]"]
  node_geometry_parser["OBJ geometry parser<br/>[obj_parser.py]"]
  node_geometry_primitives["Curve primitives"]
  node_image_renderer["Image renderer<br/>[img_renderer.py]"]
  node_solid_export["Solid conversion<br/>[parser_visual.py]"]
  node_obj_reconverter["OBJ reconverter<br/>[obj_reconverter.py]"]
end

subgraph group_learning["Model learning"]
  node_sl_dataset["SL dataset<br/>[CAD_dataset.py]"]
  node_sl_training["Sequential fine-tuning<br/>[llama_finetune.py]"]
  node_generation["CAD generation<br/>[inference.py]"]
  node_dpo_training["DPO fine-tuning<br/>[dpo.py]"]
end

subgraph group_feedback["Feedback and evaluation"]
  node_dpo_scoring["Visual scoring"]
  node_llava["LLaVA scoring<br/>[llava_utils.py]"]
  node_gpt_scoring["GPT scoring<br/>[openai_utils.py]"]
  node_preference_data["Preference pairs"]
  node_evaluation["Generation evaluation<br/>[dist_eval.py]"]
end

node_researcher(("Researcher"))
node_azure_llm["Azure OpenAI"]

node_researcher -->|"converts data"| node_provided_conversion
node_researcher -->|"prepares data"| node_sl_conversion
node_sl_conversion -->|"renders sequences"| node_sequence_parser
node_sequence_parser -->|"renders images"| node_image_renderer
node_sequence_parser -->|"converts CAD"| node_solid_export
node_geometry_parser -->|"uses curves"| node_geometry_primitives
node_solid_export -->|"parses OBJ"| node_geometry_parser
node_solid_export -->|"builds solids"| node_obj_reconverter
node_researcher -->|"captions images"| node_captioning
node_captioning -->|"calls API"| node_openai_client
node_openai_client -->|"requests completions"| node_azure_llm
node_sl_dataset -->|"supplies examples"| node_sl_training
node_researcher -->|"trains model"| node_sl_training
node_researcher -->|"generates samples"| node_generation
node_generation -->|"renders models"| node_solid_export
node_researcher -->|"scores generations"| node_dpo_scoring
node_dpo_scoring -->|"uses local model"| node_llava
node_dpo_scoring -->|"uses GPT option"| node_gpt_scoring
node_gpt_scoring -.->|"requests scores"| node_azure_llm
node_dpo_scoring -->|"builds pairs"| node_preference_data
node_researcher -->|"fine-tunes model"| node_dpo_training
node_preference_data -->|"trains on pairs"| node_dpo_training
node_researcher -->|"evaluates samples"| node_evaluation

click node_sl_conversion "https://github.com/microsoft/cadfusion/blob/main/src/data_preprocessing/convert.py"
click node_provided_conversion "https://github.com/microsoft/cadfusion/blob/main/data/sl_data/convert.py"
click node_sequence_parser "https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/parser.py"
click node_geometry_parser "https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/geometry/obj_parser.py"
click node_geometry_primitives "https://github.com/microsoft/cadfusion/tree/main/src/rendering_utils/geometry"
click node_image_renderer "https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/img_renderer.py"
click node_solid_export "https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/parser_visual.py"
click node_obj_reconverter "https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/utils/obj_reconverter.py"
click node_captioning "https://github.com/microsoft/cadfusion/blob/main/src/data_preprocessing/captioning.py"
click node_openai_client "https://github.com/microsoft/cadfusion/blob/main/src/data_preprocessing/call_openai.py"
click node_sl_dataset "https://github.com/microsoft/cadfusion/blob/main/src/train/CAD_dataset.py"
click node_sl_training "https://github.com/microsoft/cadfusion/blob/main/src/train/llama_finetune.py"
click node_generation "https://github.com/microsoft/cadfusion/blob/main/src/test/inference.py"
click node_dpo_scoring "https://github.com/microsoft/cadfusion/blob/main/src/dpo/make_dpo_dataset.py"
click node_llava "https://github.com/microsoft/cadfusion/blob/main/src/dpo/llava_utils.py"
click node_gpt_scoring "https://github.com/microsoft/cadfusion/blob/main/src/dpo/openai_utils.py"
click node_preference_data "https://github.com/microsoft/cadfusion/blob/main/src/dpo/make_dpo_dataset.py"
click node_dpo_training "https://github.com/microsoft/cadfusion/blob/main/src/train/dpo.py"
click node_evaluation "https://github.com/microsoft/cadfusion/blob/main/src/test/dist_eval.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sl_conversion,node_provided_conversion,node_captioning,node_openai_client toneBlue
class node_sequence_parser,node_geometry_parser,node_geometry_primitives,node_image_renderer,node_solid_export,node_obj_reconverter toneAmber
class node_sl_dataset,node_sl_training,node_generation,node_dpo_training toneMint
class node_dpo_scoring,node_llava,node_gpt_scoring,node_preference_data,node_evaluation toneRose
class node_researcher,node_azure_llm toneIndigo
```

## Components

### Data preparation

- [SL conversion](https://github.com/microsoft/cadfusion/blob/main/src/data_preprocessing/convert.py): `src/data_preprocessing/convert.py`
- [Provided data conversion](https://github.com/microsoft/cadfusion/blob/main/data/sl_data/convert.py): `data/sl_data/convert.py`
- [LLM captioning](https://github.com/microsoft/cadfusion/blob/main/src/data_preprocessing/captioning.py): `src/data_preprocessing/captioning.py`
- [Azure API client](https://github.com/microsoft/cadfusion/blob/main/src/data_preprocessing/call_openai.py): `src/data_preprocessing/call_openai.py`

### CAD rendering

- [Sequence parser](https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/parser.py): `src/rendering_utils/parser.py`
- [OBJ geometry parser](https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/geometry/obj_parser.py): `src/rendering_utils/geometry/obj_parser.py`
- [Curve primitives](https://github.com/microsoft/cadfusion/tree/main/src/rendering_utils/geometry): `src/rendering_utils/geometry`
- [Image renderer](https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/img_renderer.py): `src/rendering_utils/img_renderer.py`
- [Solid conversion](https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/parser_visual.py): `src/rendering_utils/parser_visual.py`
- [OBJ reconverter](https://github.com/microsoft/cadfusion/blob/main/src/rendering_utils/utils/obj_reconverter.py): `src/rendering_utils/utils/obj_reconverter.py`

### Model learning

- [SL dataset](https://github.com/microsoft/cadfusion/blob/main/src/train/CAD_dataset.py): `src/train/CAD_dataset.py`
- [Sequential fine-tuning](https://github.com/microsoft/cadfusion/blob/main/src/train/llama_finetune.py): `src/train/llama_finetune.py`
- [CAD generation](https://github.com/microsoft/cadfusion/blob/main/src/test/inference.py): `src/test/inference.py`
- [DPO fine-tuning](https://github.com/microsoft/cadfusion/blob/main/src/train/dpo.py): `src/train/dpo.py`

### Feedback and evaluation

- [Visual scoring](https://github.com/microsoft/cadfusion/blob/main/src/dpo/make_dpo_dataset.py): `src/dpo/make_dpo_dataset.py`
- [LLaVA scoring](https://github.com/microsoft/cadfusion/blob/main/src/dpo/llava_utils.py): `src/dpo/llava_utils.py`
- [GPT scoring](https://github.com/microsoft/cadfusion/blob/main/src/dpo/openai_utils.py): `src/dpo/openai_utils.py`
- [Preference pairs](https://github.com/microsoft/cadfusion/blob/main/src/dpo/make_dpo_dataset.py): `src/dpo/make_dpo_dataset.py`
- [Generation evaluation](https://github.com/microsoft/cadfusion/blob/main/src/test/dist_eval.py): `src/test/dist_eval.py`

### Other components

- **Researcher**
- **Azure OpenAI**

## Connections

- Researcher → Provided data conversion: converts data (evidence: `README.md`)
- Researcher → SL conversion: prepares data (evidence: `README.md`)
- SL conversion → Sequence parser: renders sequences (evidence: `README.md`)
- Sequence parser → Image renderer: renders images (evidence: `README.md`)
- Sequence parser → Solid conversion: converts CAD (evidence: `README.md`)
- OBJ geometry parser → Curve primitives: uses curves (evidence: `src/rendering_utils/geometry/obj_parser.py`)
- Solid conversion → OBJ geometry parser: parses OBJ (evidence: `src/rendering_utils/parser_visual.py`)
- Solid conversion → OBJ reconverter: builds solids (evidence: `src/rendering_utils/parser_visual.py`)
- Researcher → LLM captioning: captions images (evidence: `README.md`)
- LLM captioning → Azure API client: calls API (evidence: `src/data_preprocessing/captioning.py`)
- Azure API client → Azure OpenAI: requests completions (evidence: `src/data_preprocessing/call_openai.py`)
- SL dataset → Sequential fine-tuning: supplies examples (evidence: `src/train/llama_finetune.py`)
- Researcher → Sequential fine-tuning: trains model (evidence: `README.md`)
- Researcher → CAD generation: generates samples (evidence: `README.md`)
- CAD generation → Solid conversion: renders models (evidence: `README.md`)
- Researcher → Visual scoring: scores generations (evidence: `README.md`)
- Visual scoring → LLaVA scoring: uses local model (evidence: `src/dpo/make_dpo_dataset.py`)
- Visual scoring → GPT scoring: uses GPT option (evidence: `src/dpo/make_dpo_dataset.py`)
- GPT scoring → Azure OpenAI: requests scores
- Visual scoring → Preference pairs: builds pairs (evidence: `src/dpo/make_dpo_dataset.py`)
- Researcher → DPO fine-tuning: fine-tunes model (evidence: `README.md`)
- Preference pairs → DPO fine-tuning: trains on pairs (evidence: `README.md`)
- Researcher → Generation evaluation: evaluates samples (evidence: `README.md`)
