How weiliansong/blueprint-vectorizer works
Main Components
• Data Ingestion & Preprocessing: Represented by the “00_preprocess” directory and the “data” folder structure where raw images and GT labels reside.
• Instance Segmentation Module: Found in the “01_instance_seg” folder. This module contains both training and inference scripts, as well as variants of U-Net-based models (under “models/unet” and “models/unet_coord”) used to segment floorplans into separate instances.
• Semantic Classification Module: Shown in the “02_type_class” directory, where each segmented instance is categorized (e.g., door, window, wall, etc.) based on the provided class mapping.
• Frame Detection Component: Under “03_frame_detect”, this stage trains a CNN to understand the topology or arrangement of frame symbols.
• Frame Correction Component: Within the “04_frame_correct” folder, this stage uses a Generative Adversarial Network (GAN) for correcting errors in the predicted frame boundaries. It also leverages the topology classifier from the previous stage.
• Heuristic Simplification Module: In “05_simplification”, the segmentation output is refined to smooth out spurious details to produce a cleaner vectorization.
• Evaluation (and future post-processing): The “06_eval” directory contains scripts for evaluating various aspects of the model’s output.
• External Resources and Data: Include the “data” directory (with subfolders for images, segmentation masks, splits, etc.) and the “ckpts” folder (from pretrained models as mentioned in the README).
Relationships and Data Flow
• These then pass into the preprocessing stage (00_preprocess), which likely prepares masks and fixes annotation inconsistencies.
• Processed images are fed into the Instance Segmentation module (01_instance_seg). This component outputs segmented instances.
• The segmented outputs then pass to the Semantic Classification module (02_type_class) to assign a type/class label to each instance.
• In parallel or as a side chain, the feature maps (or segmentation outputs) feed into the Frame Detection module (03_frame_detect) which is used by the Frame Correction module (04_frame_correct) to adjust any topological errors.
• The outputs of the segmentation and correction components are then refined by the Heuristic Simplification module (05_simplification).
• Optionally, the final outputs can be evaluated using the scripts in (06_eval).
Architectural Patterns and Technologies
• That this is a pipelined architecture with clear modular stages. Each stage is independently trainable and produces intermediate outputs for the subsequent stages.
• That the system leverages deep learning frameworks (PyTorch for the segmentation and GAN models) and utilizes configurations (YAML files) for setting parameters.
• The use of conventional file organization (sequential directories as numbered stages) hints at a batch-style pipeline where outputs are stored in specified folders and used as inputs for the next steps.