# sail-sg/sewformer architecture

> Architecture diagram of the GitHub repository sail-sg/sewformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sail-sg/sewformer
- Repository: https://github.com/sail-sg/sewformer

## Overview

##### Project Type and Purpose
- Sewformer is a research‐oriented deep learning system for reconstructing garment sewing patterns from single images.
- It comprises two major subsystems:
a) SewFactory – a data‐generation and simulation toolkit that assembles synthetic garment meshes, patterns, and scene materials via Maya‐based scripts.
b) Sewformer – a PyTorch‐based training and inference pipeline implementing the transformer‐based model described in SIGGRAPH Asia 2023.

##### High‐Level Architecture Components
A. SewFactory (Data Generation & Simulation)
- Data Generators (data_generator/) – Python scripts (data_sim.py, deepfashion_sim.py, pattern_gen.py, fbx_anime.py) that produce 2D panels, 3D meshes, animations, and simulation inputs.
- Meta-Information (meta_infos/) – JSON/YAML catalogs defining sewing patterns, material configs, FBX metadata, scene materials, and simulation parameters.
- MayaQLTools & Pattern Packages (packages/) – reusable Python modules wrapping Maya commands, Qualoth cloth simulator, panel‐rotation logic, core pattern operations, and utility functions.
- Output Dataset (SewFactory folder) – Stores synthetic images, ground‐truth pattern files, and Maya scene exports for training and evaluation.

B. Sewformer (Model Training & Inference)
- Configuration (configs/, system.json, environment.yaml, requirements.txt) – Defines training and testing hyperparameters (train.yaml, test.yaml), dataset paths, environment dependencies, and checkpoint locations.
- Data Layer (data/) – Dataset wrappers, SMPL body model integration (human_body_prior/), data transforms, panel‐class converters, and unified data loader exposing batches to the model.
- Models (models/) – Backbone CNNs, transformer encoder/decoder, positional encodings, garment‐specific detection heads (garment_detr_2d.py), and utility functions.
- Training Engine (train.py, trainer.py, metrics/) – Training loop orchestrating forward/backward passes, loss computation (composed_loss.py, eval_detr_metrics.py), scheduler (warm_cosine_scheduler.py), logging, and checkpoint saving.
- Inference & Evaluation (inference.py, experiment.py, metrics/) – Scripts to run inference on real/synthetic images, compute evaluation metrics, and export predicted panels for downstream simulation.

##### Component Relationships and Data Flow
- SewFactory → Produces a structured dataset on disk (images + pattern labels + simulation inputs).
- Sewformer Data Layer → Reads synthetic/real data using paths from system.json and YAML configs.
- Model → Receives image batch → Backbone (CNN) → Transformer → Panel detection & parameter regression heads → Output pattern predictions.
- Trainer → Calculates losses, updates model weights, stores checkpoints.
- Inference → Loads trained checkpoint, runs on new images, dumps predicted patterns.
- Simulation (optional) → Feeds predicted patterns back into SewFactory’s Maya‐based simulators (deepfashion_sim.py) to generate rendered results.

##### Key Architectural Patterns and Design Principles
- Modular separation: Data generation (SewFactory) vs. modeling (Sewformer) lives in separate directories.
- Config‐driven workflows: Almost every script reads from YAML/JSON configs, enabling reproducibility and hyperparameter management.
- Layered design:
- Data layer abstracts dataset formats (SMPL, panels, images).
- Model layer encapsulates neural architecture.
- Training/inference layer orchestrates execution and metrics.
- Reusability: MayaQLTools and Pattern packages serve as internal SDKs for garment operations.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% SewFactory Cluster
    subgraph "SewFactory" 
        style SewFactory fill:#D0E6FF,stroke:#004080,stroke-width:2px
        DF1["Data Generators\n(data_sim.py,\ndeepfashion_sim.py,\nfbx_anime.py,\npattern_gen.py)"] 
        DF2["Meta-Information Store\n(JSON/YAML catalogs)"] 
        DF3["MayaQLTools SDK\n(packages/mayaqltools)"] 
        DF4["Pattern Package\n(packages/pattern)"] 
        DS[(Generated Dataset)] 
    end

    %% Sewformer Cluster
    subgraph "Sewformer"
        style Sewformer fill:#D6FFD0,stroke:#207520,stroke-width:2px
        CFG["Configuration Service\n(configs/, system.json,\nenvironment.yaml,\nrequirements.txt)"] 
        DL["Data Layer\n(dataset.py,\nwrapper.py,\ntransforms.py,\npanel_classes.py,\npattern_converter.py,\nutils.py,\nhuman_body_prior/)"] 
        MOD["Model\n(backbone.py,\ngarment_backbone.py,\ntransformer.py,\ngarment_detr_2d.py,\nposition_encoding.py,\nutils/misc.py)"] 
        TRN["Training Engine\n(train.py,\ntrainer.py,\nwarm_cosine_scheduler.py,\nmetrics/)"] 
        CKP[(Checkpoint Store)] 
        INF["Inference & Evaluation\n(inference.py,\nexperiment.py,\nmetrics/)"] 
    end

    %% External Dependencies
    subgraph "External Dependencies"
        style ED fill:#F5D0FF,stroke:#80004F,stroke-width:2px
        EXT1["Conda/PyTorch/\nHuggingFace"] 
        EXT2["Maya/Qualoth"] 
        EXT3["RSC-Net (SMPL)"] 
    end

    %% Primary Data Flows
    DF1 --> DF2
    DF2 --> DF3
    DF3 --> DF4
    DF4 --> DS
    DS --> CFG
    CFG --> DL
    DL --> MOD
    MOD --> TRN
    TRN --> CKP
    CKP --> INF
    INF --> DS

    %% Optional Flows
    EXT3 -.-> DF1
    EXT2 -.-> DF3
    EXT1 -.-> MOD
    EXT1 -.-> TRN
    EXT1 -.-> INF

    %% Click Events for SewFactory
    click DF1 "https://github.com/sail-sg/sewformer/tree/main/SewFactory/data_generator/"
    click DF2 "https://github.com/sail-sg/sewformer/tree/main/SewFactory/meta_infos/"
    click DF3 "https://github.com/sail-sg/sewformer/tree/main/SewFactory/packages/mayaqltools/"
    click DF4 "https://github.com/sail-sg/sewformer/tree/main/SewFactory/packages/pattern/"
    click DS "https://github.com/sail-sg/sewformer/tree/main/SewFactory/"

    %% Click Events for Sewformer
    click CFG "https://github.com/sail-sg/sewformer/tree/main/Sewformer/configs/"
    click DL "https://github.com/sail-sg/sewformer/blob/main/Sewformer/data/dataset.py"
    click DL "https://github.com/sail-sg/sewformer/blob/main/Sewformer/data/wrapper.py"
    click DL "https://github.com/sail-sg/sewformer/blob/main/Sewformer/data/transforms.py"
    click DL "https://github.com/sail-sg/sewformer/blob/main/Sewformer/data/panel_classes.py"
    click DL "https://github.com/sail-sg/sewformer/blob/main/Sewformer/data/pattern_converter.py"
    click DL "https://github.com/sail-sg/sewformer/blob/main/Sewformer/data/utils.py"
    click DL "https://github.com/sail-sg/sewformer/tree/main/Sewformer/data/human_body_prior/"
    click MOD "https://github.com/sail-sg/sewformer/blob/main/Sewformer/models/backbone.py"
    click MOD "https://github.com/sail-sg/sewformer/blob/main/Sewformer/models/garment_backbone.py"
    click MOD "https://github.com/sail-sg/sewformer/blob/main/Sewformer/models/transformer.py"
    click MOD "https://github.com/sail-sg/sewformer/blob/main/Sewformer/models/garment_detr_2d.py"
    click MOD "https://github.com/sail-sg/sewformer/blob/main/Sewformer/models/position_encoding.py"
    click MOD "https://github.com/sail-sg/sewformer/blob/main/Sewformer/models/utils/misc.py"
    click TRN "https://github.com/sail-sg/sewformer/blob/main/Sewformer/train.py"
    click TRN "https://github.com/sail-sg/sewformer/blob/main/Sewformer/trainer.py"
    click TRN "https://github.com/sail-sg/sewformer/blob/main/Sewformer/warm_cosine_scheduler.py"
    click TRN "https://github.com/sail-sg/sewformer/blob/main/Sewformer/metrics/composed_loss.py"
    click TRN "https://github.com/sail-sg/sewformer/blob/main/Sewformer/metrics/eval_detr_metrics.py"
    click TRN "https://github.com/sail-sg/sewformer/blob/main/Sewformer/metrics/losses.py"
    click TRN "https://github.com/sail-sg/sewformer/blob/main/Sewformer/metrics/metrics.py"
    click INF "https://github.com/sail-sg/sewformer/blob/main/Sewformer/inference.py"
    click INF "https://github.com/sail-sg/sewformer/blob/main/Sewformer/experiment.py"

    %% Styles
    classDef factory fill:#D0E6FF,stroke:#004080
    classDef sewo fill:#D6FFD0,stroke:#207520
    classDef external fill:#F5D0FF,stroke:#80004F
    class DF1,DF2,DF3,DF4,DS factory
    class CFG,DL,MOD,TRN,CKP,INF sewo
    class EXT1,EXT2,EXT3 external
```
