# microsoft/cad-editor architecture

> Architecture diagram of the GitHub repository microsoft/cad-editor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/cad-editor
- Repository: https://github.com/microsoft/cad-editor

## Overview

##### Project type and purpose
- This is a research‐grade machine‐learning system for text‐based CAD editing. It implements a “locate‐then‐infill” framework (two‐stage transformer finetuning on CAD sequences) plus an automated data‐synthesis pipeline and evaluation suite.

##### File structure
Top‐level directories and their roles:
- data/             – raw and intermediate data generation and pairing scripts
- finetune/         – training & sampling scripts, DeepSpeed config, mask/infill tasks
- hnc-cad/          – hooks into external CAD sample generator (customized ac_gen.py)
- prompt/           – zero‐shot and few‐shot GPT‐4 baseline implementations
- utils/            – CAD parsing, conversion, rendering, sampling, evaluation helpers
Other files:
- environment.yaml – conda environment
- README.md        – installation, pipeline, training, inference, evaluation

##### README for additional insights
- Data preparation: 3 sub‐pipelines (paired CAD generation, instruction generation, merge/filter)
- Training: two stages (locate → infill), multi‐GPU via accelerate + DeepSpeed
- Inference: llama_sample.py for mask & infill
- Evaluation: parse_seq2obj, visual_obj, cad_img, sample_points, eval_cad, eval_dclip

##### Main components
A. Data Generation & Preprocessing
- External hnc-cad generator (ac_gen.py)
- obj → sequence parsing (utils/parse_obj2seq.py)
- sequence conversion & pairing (utils/convert.py, data/pair.py)
- Instruction synthesis – image captions (data/caption_image.py ➔ OpenAI API),
sequence captions (data/caption_sequence.py)
- Merging & filtering scripts (data/merge.py, data/filter_sequence.py)

B. Training Pipeline
1. Locate Stage
– create_mask.py ➔ masked sequences
– llama_finetune.py --task_type mask (DeepSpeed config, multi‐GPU)
2. Infill Stage
– llama_finetune.py --task_type infill / infill_selective
– selective data enhancement

C. Inference Pipeline
- llama_sample.py for mask generation
- llama_sample.py for infill generation
- Model checkpoints from HuggingFace (locate_stage, infill_stage)

D. Evaluation Suite
- parse_seq2obj.py ➔ CAD obj
- visual_obj.py + cad_img.py ➔ render & images
- sample_points.py ➔ point clouds
- eval_cad.py ➔ 3D geometry metrics
- eval_dclip.py ➔ directional CLIP score

E. Baseline Prompting
- prompt/gpt4_basic.py, prompt/gpt4_fs3.py

##### Relationships & Data Flow
Data Gen → Preprocessing → Training (Locate → Infill)
Training → Model Registry (HuggingFace checkpoints)
Model Registry + Test Data → Inference → Edited CAD sequences
Edited CAD + GT → Evaluation Suite → Metrics
- External services: OpenAI API (captioning), HuggingFace model hub

##### Architectural patterns & design principles
- Pipeline architecture: sequential ETL → train → infer → evaluate
- Two‐stage modular ML workflow (locate vs. infill)
- Script‐driven orchestration (Python CLIs)
- Reusable utilities under utils/ for parsing, rendering, evaluation

##### Relevant technologies & frameworks
- Python 3, Conda, DeepSpeed/Accelerate
- LLaMA transformer via HuggingFace Transformers
- OpenAI API for text captioning
- OBJ file parsing & geometry utilities
- PyTorch (implied by accelerate)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Generation & Preprocessing
    subgraph "Data Generation & Preprocessing"
        direction TB
        HNC_CAD["External CAD Generator\n(ac_gen.py)"]:::data
        ParseOBJ["OBJ → Sequence Parser\n(parse_obj2seq.py)"]:::data
        ConvertSeq["Sequence Converter\n(convert.py)"]:::data
        PairSeq["Sequence Pairing\n(pair.py)"]:::data
        CaptionImage["Image Captioning\n(caption_image.py)"]:::data
        CaptionSeq["Sequence Captioning\n(caption_sequence.py)"]:::data
        MergeData["Merge Data\n(merge.py)"]:::data
        FilterSeq["Filter Sequences\n(filter_sequence.py)"]:::data
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        direction TB
        subgraph "Locate Stage"
            direction TB
            CreateMask["Create Masked Sequences\n(create_mask.py)"]:::train
            FinetuneMask["Finetune (mask task)\n(llama_finetune.py)"]:::train
            DSConfig["DeepSpeed Config\n(ds_config.yaml)"]:::train
        end
        subgraph "Infill Stage"
            direction TB
            FinetuneInfill["Finetune (infill task)\n(llama_finetune.py)"]:::train
        end
    end

    %% Inference Pipeline
    subgraph "Inference Pipeline"
        direction TB
        Sample["Mask & Infill Sampling\n(llama_sample.py)"]:::infer
    end

    %% Evaluation Suite
    subgraph "Evaluation Suite"
        direction TB
        Seq2OBJ["Sequence → OBJ Converter\n(parse_seq2obj.py)"]:::eval
        VisualObj["3D Renderer\n(visual_obj.py)"]:::eval
        CadImg["CAD Image Export\n(cad_img.py)"]:::eval
        SamplePts["Point Cloud Sampling\n(sample_points.py)"]:::eval
        EvalCAD["Geometry Metrics\n(eval_cad.py)"]:::eval
        EvalDCLIP["Directional CLIP Score\n(eval_dclip.py)"]:::eval
        DCLIPHelper["CLIP Helper\n(directional_clip_score.py)"]:::eval
    end

    %% Baseline Prompting
    subgraph "Baseline Prompting"
        direction TB
        GPT4Basic["Zero-shot GPT-4 Baseline\n(gpt4_basic.py)"]:::baseline
        GPT4FS3["Few-shot GPT-4 Baseline\n(gpt4_fs3.py)"]:::baseline
    end

    %% External Services
    subgraph "External Services"
        direction TB
        OpenAI["OpenAI API"]:::external
        HFHub["HuggingFace Model Hub"]:::external
    end

    %% Data Flow Connections
    HNC_CAD --> ParseOBJ
    ParseOBJ --> ConvertSeq
    ConvertSeq --> PairSeq
    PairSeq --> CaptionImage
    CaptionImage --> CaptionSeq
    CaptionSeq --> MergeData
    MergeData --> FilterSeq
    FilterSeq --> CreateMask
    FilterSeq --> FinetuneMask

    CreateMask --> FinetuneMask
    FinetuneMask --> DSConfig
    FinetuneMask --> FinetuneInfill
    DSConfig --> FinetuneMask

    FinetuneInfill --> HFHub

    HFHub --> Sample
    FilterSeq --> Sample
    Sample --> Seq2OBJ
    Seq2OBJ --> VisualObj
    Seq2OBJ --> CadImg
    Seq2OBJ --> SamplePts
    SamplePts --> EvalCAD
    SamplePts --> EvalDCLIP
    DCLIPHelper --> EvalDCLIP

    GPT4Basic --> EvalCAD
    GPT4FS3 --> EvalCAD

    CaptionImage --> OpenAI
    CaptionSeq --> OpenAI

    %% Click Events
    click HNC_CAD "https://github.com/microsoft/cad-editor/blob/main/hnc-cad/ac_gen.py"
    click ParseOBJ "https://github.com/microsoft/cad-editor/blob/main/utils/parse_obj2seq.py"
    click ConvertSeq "https://github.com/microsoft/cad-editor/blob/main/utils/convert.py"
    click PairSeq "https://github.com/microsoft/cad-editor/blob/main/data/pair.py"
    click CaptionImage "https://github.com/microsoft/cad-editor/blob/main/data/caption_image.py"
    click CaptionSeq "https://github.com/microsoft/cad-editor/blob/main/data/caption_sequence.py"
    click MergeData "https://github.com/microsoft/cad-editor/blob/main/data/merge.py"
    click FilterSeq "https://github.com/microsoft/cad-editor/blob/main/data/filter_sequence.py"
    click CreateMask "https://github.com/microsoft/cad-editor/blob/main/finetune/create_mask.py"
    click FinetuneMask "https://github.com/microsoft/cad-editor/blob/main/finetune/llama_finetune.py"
    click DSConfig "https://github.com/microsoft/cad-editor/blob/main/finetune/ds_config.yaml"
    click FinetuneInfill "https://github.com/microsoft/cad-editor/blob/main/finetune/llama_finetune.py"
    click Sample "https://github.com/microsoft/cad-editor/blob/main/finetune/llama_sample.py"
    click Seq2OBJ "https://github.com/microsoft/cad-editor/blob/main/utils/parse_seq2obj.py"
    click VisualObj "https://github.com/microsoft/cad-editor/blob/main/utils/visual_obj.py"
    click CadImg "https://github.com/microsoft/cad-editor/blob/main/utils/cad_img.py"
    click SamplePts "https://github.com/microsoft/cad-editor/blob/main/utils/sample_points.py"
    click EvalCAD "https://github.com/microsoft/cad-editor/blob/main/utils/eval_cad.py"
    click EvalDCLIP "https://github.com/microsoft/cad-editor/blob/main/utils/eval_dclip.py"
    click DCLIPHelper "https://github.com/microsoft/cad-editor/blob/main/utils/directional_clip_score.py"
    click GPT4Basic "https://github.com/microsoft/cad-editor/blob/main/prompt/gpt4_basic.py"
    click GPT4FS3 "https://github.com/microsoft/cad-editor/blob/main/prompt/gpt4_fs3.py"

    %% Styles
    classDef data fill:#BBE1FA,stroke:#1B262C,color:#0F4C75
    classDef train fill:#D0F4DE,stroke:#1B262C,color:#2D6A4F
    classDef infer fill:#FFD8A9,stroke:#1B262C,color:#E85D04
    classDef eval fill:#E5D1FA,stroke:#1B262C,color:#6A4C93
    classDef baseline fill:#FFF3B0,stroke:#1B262C,color:#997B55
    classDef external fill:#CCCCCC,stroke:#1B262C,color:#5A5A5A
```
