# ziangcao0312/physx-3d architecture

> Architecture diagram of the GitHub repository ziangcao0312/physx-3d, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ziangcao0312/physx-3d
- Repository: https://github.com/ziangcao0312/physx-3d

## Overview

##### Project Type and Purpose
- PhysX-3D is a research-grade machine-learning codebase for “Physical-Grounded 3D Asset Generation.” It integrates data-preprocessing toolkits, model definitions (VAE + diffusion/flow), training/inference pipelines, and rendering utilities.
- Core features include dataset annotation scripts (`dataset_toolkits`, `tools`), model architectures and modules under `trellis`, training entry point (`train.py`), inference example (`example.py`), and external‐dependency management (`setup.sh`, `download_pretrain.sh`).

##### High-Level Architectural Components
a. Data Layer
- Dataset Toolkit (`dataset_toolkits/`) – ingestion, preprocessing, metadata extraction, Blender integration for textures.
- External Datasets – PhysXNet (via Hugging Face), ShapeNet for textures.
b. Configuration & Environment
- Config files (`configs/vae`, `configs/generation`) – JSON experiment parameters.
- Setup scripts (`setup.sh`, `download_pretrain.sh`) – conda env, dependencies, pre-trained weights.
c. Core ML Library (Treillis Submodule)
- `trellis/datasets/` – PyTorch Dataset classes, samplers.
- `trellis/models/` – VAE, flow/diffusion architectures (encoders, decoders, mixins).
- `trellis/modules/` – attention, sparse and dense building blocks.
- `trellis/pipelines/` – end-to-end training & inference orchestration (samplers, text/image->3D).
- `trellis/trainers/` – training loops for VAE and flow matching, guidance mixins.
- `trellis/renderers/` & `trellis/utils/` – mesh/octree/gaussian rendering, loss, post-processing.
d. Voxel2Seq Library (Optional Component)
- `vox2seq/` – CUDA/C++ extensions for voxel encoding (Hilbert, Z-order), PyTorch bindings.
e. Entry Points
- `train.py` – unified training CLI that loads config, initializes `trellis` pipelines/trainers, dispatches to GPU/distributed.
- `example.py` – inference demo: loads pre-trained models, runs generation, invokes renderers.

##### Component Interactions
- Data Toolkit → Preprocessed dataset on disk → Trellis Dataset classes
- Config Loader (train.py) → Instantiates appropriate Trainer (VAE or Flow) → Builds Model & Modules → Batches of data from Dataset → GPU Training
- After training → Model checkpoints → Inference pipeline (`example.py`) → Sampler pipelines → Renderers → Exported 3D assets
- Tools scripts (`tools/`) integrate with dataset or serve as annotation/visualization utilities.

##### Architectural Patterns & Principles
- Modularization: clear separation of data, models, modules, pipelines, trainers, and utilities.
- Pipeline Pattern: `trellis/pipelines` orchestrates data→model→sampler→renderer.
- Configuration-Driven: experiments and hyperparameters in JSON, enabling reproducibility.
- Mixins & Inheritance: sharing behaviors (classifier-free guidance, text/image conditioning) across samplers and trainers.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Dependencies (grey)
    subgraph "External Dependencies"
        direction TB
        HF["PhysXNet Dataset"]:::external
        SN["ShapeNet Textures"]:::external
        BL["Blender"]:::external
        GPU["GPU Cluster"]:::external
    end

    %% Data & Preprocessing Layer (blue)
    subgraph "Data & Preprocessing" 
        direction TB
        DT["Data Toolkit"]:::data
        PM["Preprocessed Dataset"]:::data
        AU["Annotation & Visualization Utilities"]:::data
        VS["Optional Vox2Seq Module"]:::data
    end

    %% Configuration & Environment (orange)
    subgraph "Config & Environment"
        direction TB
        CM["Config Manager"]:::entry
        ES["Environment & Dependency Setup"]:::entry
    end

    %% Core ML Library - Trellis (green)
    subgraph "Core ML Library (trellis)"
        direction TB
        TD["Trellis Datasets"]:::core
        TM["Trellis Models"]:::core
        TMo["Trellis Modules"]:::core
        TP["Trellis Pipelines"]:::core
        TT["Trellis Trainers"]:::core
        TRU["Renderers & Utils"]:::core
    end

    %% Entry Points (orange)
    subgraph "Entry Points"
        direction TB
        TR["Training CLI (train.py)"]:::entry
        IE["Inference Example (example.py)"]:::entry
    end

    %% Data Flow
    HF -->|ingest| DT
    SN -->|provide textures| DT
    BL -->|integrate via Blender scripts| DT
    DT -->|outputs| PM
    VS -->|feature encoding| DT
    AU -->|annotation scripts| DT

    %% Config & Setup
    CM --> TR
    ES --> TR
    CM --> IE
    ES --> IE

    %% Training Flow
    TR -->|loads data| TD
    TD --> TM
    TM --> TMo
    TMo --> TP
    TP --> TT
    TT -->|GPU training| GPU
    TT -->|saves checkpoints| PM

    %% Inference Flow
    IE --> TD
    TD --> TP
    TP --> TRU
    TRU -->|generate assets| PM

    %% Click Events
    click DT "https://github.com/ziangcao0312/physx-3d/tree/main/dataset_toolkits/"
    click DT "https://github.com/ziangcao0312/physx-3d/tree/main/dataset_toolkits/"
    click DT "https://github.com/ziangcao0312/physx-3d/tree/main/dataset_toolkits/blender_script/"
    click DT "https://github.com/ziangcao0312/physx-3d/blob/main/dataset_toolkits/render.py"
    click DT "https://github.com/ziangcao0312/physx-3d/blob/main/dataset_toolkits/render_cond.py"
    click DT "https://github.com/ziangcao0312/physx-3d/blob/main/dataset_toolkits/build_metadata.py"
    click DT "https://github.com/ziangcao0312/physx-3d/blob/main/dataset_toolkits/gen_csv.py"
    click DT "https://github.com/ziangcao0312/physx-3d/blob/main/dataset_toolkits/merge_property.py"
    click DT "https://github.com/ziangcao0312/physx-3d/blob/main/dataset_toolkits/tools.py"
    click DT "https://github.com/ziangcao0312/physx-3d/blob/main/dataset_toolkits/datasets/PhysXNet.py"
    click CM "https://github.com/ziangcao0312/physx-3d/tree/main/configs/vae/"
    click CM "https://github.com/ziangcao0312/physx-3d/tree/main/configs/generation/"
    click ES "https://github.com/ziangcao0312/physx-3d/blob/main/setup.sh"
    click ES "https://github.com/ziangcao0312/physx-3d/blob/main/download_pretrain.sh"
    click TR "https://github.com/ziangcao0312/physx-3d/blob/main/train.py"
    click IE "https://github.com/ziangcao0312/physx-3d/blob/main/example.py"
    click AU "https://github.com/ziangcao0312/physx-3d/tree/main/tools/"
    click TD "https://github.com/ziangcao0312/physx-3d/tree/main/trellis/datasets/"
    click TM "https://github.com/ziangcao0312/physx-3d/tree/main/trellis/models/"
    click TMo "https://github.com/ziangcao0312/physx-3d/tree/main/trellis/modules/"
    click TP "https://github.com/ziangcao0312/physx-3d/tree/main/trellis/pipelines/"
    click TT "https://github.com/ziangcao0312/physx-3d/tree/main/trellis/trainers/"
    click TRU "https://github.com/ziangcao0312/physx-3d/tree/main/trellis/renderers/"
    click TRU "https://github.com/ziangcao0312/physx-3d/tree/main/trellis/utils/"
    click VS "https://github.com/ziangcao0312/physx-3d/tree/main/vox2seq/"

    %% Styles
    classDef data fill:#B3D4FC,stroke:#333,stroke-width:1px
    classDef core fill:#C2FABC,stroke:#333,stroke-width:1px
    classDef entry fill:#FFDCA8,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px
```
