# kchen92/text2shape architecture

> Architecture diagram of the GitHub repository kchen92/text2shape, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kchen92/text2shape
- Repository: https://github.com/kchen92/text2shape

## Overview

Main Components to Include:

##### Data Processing Pipeline
- Input Sources:
- ShapeNet voxelizations
- Primitives dataset
- Text descriptions/captions
- Processing Components:
- NRRD file handling (nrrd_rw.py)
- Data preprocessing (preprocess.py)
- Various data processing modules (data_process*.py files)

##### Core Architecture Components
a. Text Encoder:
- Text embedding generation
- Learning joint embeddings
- Located in models/encoder_component.py

- Learning-by-association (LBA) components
- Located in models/lba_component.py and models/lba_models.py

c. Conditional Wasserstein GAN (CWGAN):
- Generator (t2s_generator_component.py)
- Critic (t2s_critic_component.py)
- WGAN training logic (cwgan.py)

- Classification models (classifier_models.py)
- Training and evaluation components

##### Training Pipeline
- Loss functions (losses.py)
- Optimizers (optimizers.py)
- Solvers for different components (solver*.py files)
- Network components (network.py, net_component.py)

##### Use distinct sections with clear boundaries
- Data Input & Processing (blue section)
- Model Architecture (green section)
- Training Pipeline (orange section)
- Output Generation (purple section)

##### Flow Direction
- Main data flow should move left to right
- Feedback loops should be shown for GAN training

##### Component Relationships
- Where embeddings merge

##### Specific Elements
- Embedding space visualization
- GAN architecture with generator and critic
- Classification pathway
- Output visualization component

##### Technical Details to Note
- Configuration parameters from improved_wgan.yaml
- Use of NRRD format for voxel data
- Integration points with SmartScenes Toolkit
- Batch sizes and learning rates

Visualization Guidelines:

##### Use standardized symbols
- Rectangles: Processing modules
- Diamonds: Decision points

##### Layer Organization
- Top layer: Data input and preprocessing
- Middle layer: Core neural network architecture
- Bottom layer: Training and optimization
- Side panel: Configuration and utilities

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Input
        TD["Text Descriptions"]
        SN["ShapeNet Voxels"]
        PD["Primitives Dataset"]
    end

    subgraph "Data Processing"
        NR["NRRD Handler"]:::file
        DP["Data Preprocessor"]:::file
        DPM["Main Processing"]:::file
        DPC["Classifier Processing"]:::file
        DPE["Encoder Processing"]:::file
        DPG["GAN Processing"]:::file
    end

    subgraph "Core Architecture"
        subgraph "Encoder Components"
            TE["Text Encoder"]:::file
            SE["Shape Encoder"]:::file
            LBA["LBA Models"]:::file
        end

        subgraph "GAN Components"
            GEN["Generator"]:::file
            CRI["Critic"]:::file
            CWG["CWGAN Core"]:::file
        end

        CLS["Shape Classifier"]:::file
    end

    subgraph "Training Pipeline"
        LF["Loss Functions"]:::file
        OPT["Optimizers"]:::file
        MS["Main Solver"]:::file
        CS["Classifier Solver"]:::file
        ES["Encoder Solver"]:::file
        GS["GAN Solver"]:::file
        NC["Network Components"]:::file
    end

    subgraph "Output & Evaluation"
        VR["Voxel Renderer"]:::file
        TEE["Text Encoder Eval"]:::file
        TEG["Text Embedding Gen"]:::file
        NG["NRRD Generator"]:::file
    end

    TD --> DPM
    SN --> NR
    PD --> NR
    NR --> DP
    DP --> DPM
    DPM --> DPC & DPE & DPG

    DPE --> TE
    DPE --> SE
    SE --> LBA
    TE --> LBA

    DPG --> GEN
    DPG --> CRI
    GEN --> CWG
    CRI --> CWG

    DPC --> CLS

    LBA & CWG & CLS --> LF
    LF --> OPT
    OPT --> MS & CS & ES & GS
    MS & CS & ES & GS --> NC

    NC --> VR
    TE --> TEE
    TEE --> TEG
    TEG --> NG

    classDef file fill:#f9f,stroke:#333,stroke-width:2px
    classDef default fill:#ddd,stroke:#333,stroke-width:2px

    click NR "https://github.com/kchen92/text2shape/blob/master/lib/nrrd_rw.py"
    click DP "https://github.com/kchen92/text2shape/blob/master/lib/preprocess.py"
    click DPM "https://github.com/kchen92/text2shape/blob/master/lib/data_process.py"
    click DPC "https://github.com/kchen92/text2shape/blob/master/lib/data_process_classifier.py"
    click DPE "https://github.com/kchen92/text2shape/blob/master/lib/data_process_encoder.py"
    click DPG "https://github.com/kchen92/text2shape/blob/master/lib/data_process_gan.py"
    click TE "https://github.com/kchen92/text2shape/blob/master/models/encoder_component.py"
    click SE "https://github.com/kchen92/text2shape/blob/master/models/lba_component.py"
    click LBA "https://github.com/kchen92/text2shape/blob/master/models/lba_models.py"
    click GEN "https://github.com/kchen92/text2shape/blob/master/models/t2s_generator_component.py"
    click CRI "https://github.com/kchen92/text2shape/blob/master/models/t2s_critic_component.py"
    click CWG "https://github.com/kchen92/text2shape/blob/master/lib/cwgan.py"
    click CLS "https://github.com/kchen92/text2shape/blob/master/models/classifier_models.py"
    click LF "https://github.com/kchen92/text2shape/blob/master/lib/losses.py"
    click OPT "https://github.com/kchen92/text2shape/blob/master/lib/optimizers.py"
    click MS "https://github.com/kchen92/text2shape/blob/master/lib/solver.py"
    click CS "https://github.com/kchen92/text2shape/blob/master/lib/solver_classifier.py"
    click ES "https://github.com/kchen92/text2shape/blob/master/lib/solver_encoder.py"
    click GS "https://github.com/kchen92/text2shape/blob/master/lib/solver_gan.py"
    click NC "https://github.com/kchen92/text2shape/blob/master/lib/network.py"
    click VR "https://github.com/kchen92/text2shape/blob/master/lib/visualization/voxel_rendering.py"
    click TEE "https://github.com/kchen92/text2shape/blob/master/tools/eval/eval_text_encoder.py"
    click TEG "https://github.com/kchen92/text2shape/blob/master/tools/scripts/generate_text_embeddings.sh"
    click NG "https://github.com/kchen92/text2shape/blob/master/tools/scripts/generate_nrrd.py"
```
