# alanyoung123/physworld architecture

> Architecture diagram of the GitHub repository alanyoung123/physworld, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alanyoung123/physworld
- Repository: https://github.com/alanyoung123/physworld

## Overview

##### Project type and purpose
- PhysWorld is an open-source research framework for learning physics-consistent world models of deformable objects from real video. It combines a simulator-based data synthesis pipeline with a graph neural network (GNN) world model for fast, accurate future state prediction.

##### Main architectural components
a. Video Input & Preprocessing
- Ingest raw video frames (RGB or depth)
- Preprocess for segmentation, object tracking, and mesh/voxel extraction
b. Physics-Aware Digital Twin Constructor
- Constitutive Model Selector (chooses material models in MPM)
- Global-to-Local Physical Property Optimizer (fits simulator parameters to real video)
c. Demonstration Synthesis Pipeline
- Part-Aware Perturbation Module (perturbs local properties to generate diverse behaviors)
- MPM Simulator (runs forward simulations, renders 4D trajectories)
d. World Model Training
- Dataset Assembler (packages synthetic demonstrations)
- GNN-Based Predictor (implements node/edge features, message passing in PyTorch/PyG)
- Training Engine (handles loss functions, batching, early stopping)
e. Real-Video Refinement Loop
- Refinement Optimizer (fine-tunes physical properties using real video loops)
- Data Fusion Layer (melds real and synthetic data)
f. Inference Engine & Export
- Fast GNN Inference Module (performs rollouts at ~47× speedup)
- API/CLI Interface (exposes model for downstream tasks in robotics/VR/AR)
g. Supporting Infrastructure
- Data Storage (synthetic trajectories, trained checkpoints)
- Configuration & Experiment Scripts (hyperparameters, job submission)
- Visualization & Logging (TensorBoard, matplotlib figures under /figures)

##### Architectural patterns & design principles
- Pipeline Pattern: clear stages from data ingestion to inference
- Modular Separation: simulation, data synthesis, ML training, and refinement are decoupled
- Plugin/Extension Points: constitutive model selector and perturbation modules can be extended
- Data-Driven Simulation: tight loop between real data calibration and physics simulator

##### Relevant technologies & libraries
- MPM Simulator (e.g., custom C++/CUDA code or Taichi MPM)
- PyTorch & PyTorch Geometric for GNN implementation
- Optimization libraries (e.g., SciPy for global-to-local fitting)
- Experiment management (Hydra, YAML config files)
- Visualization (matplotlib, TensorBoard)

## Diagram (Mermaid)

```mermaid
graph TD
    %% Input Layer
    RawVideo["Raw Video Frames"]:::data
    subgraph "Video Input & Preprocessing"
        Preprocessor["Preprocessor\n(segmentation, tracking,\nmesh/voxel extraction)"]:::compute
    end

    %% Digital Twin Constructor
    subgraph "Physics-Aware Digital Twin Constructor"
        CMSelector["Constitutive Model Selector"]:::simulation
        PropOptimizer["Global-to-Local Property Optimizer"]:::simulation
    end

    %% Demonstration Synthesis Pipeline
    subgraph "Demonstration Synthesis Pipeline"
        Perturbation["Part-Aware Perturbation Module"]:::simulation
        MPM["MPM Simulator\n(4D trajectory rendering)"]:::simulation
        SynthData["Synthetic Demonstrations"]:::data
    end

    %% World Model Training
    subgraph "World Model Training"
        Assembler["Dataset Assembler"]:::ml
        GNN["GNN-Based Predictor"]:::ml
        Trainer["Training Engine"]:::ml
        Checkpoints["Trained Checkpoints"]:::data
    end

    %% Refinement Loop
    subgraph "Real-Video Refinement Loop"
        RealVideo["Real Video"]:::data
        RefineOpt["Refinement Optimizer"]:::ml
        Fusion["Data Fusion Layer"]:::ml
    end

    %% Inference & Export
    subgraph "Inference Engine & Export"
        Inference["Fast GNN Inference Module"]:::ml
        API["API/CLI Interface"]:::io
        Predictions["Predicted Future States"]:::data
    end

    %% Supporting Infrastructure
    subgraph "Supporting Infrastructure"
        Storage["Data Storage"]:::data
        Configs["Config & Experiment Scripts"]:::data
        Logging["Visualization & Logging"]:::data
    end

    %% Connections
    RawVideo -->|raw frames| Preprocessor
    Preprocessor -->|mesh + features| CMSelector
    CMSelector --> PropOptimizer
    PropOptimizer --> Perturbation
    Perturbation --> MPM
    MPM -->|4D trajectories| SynthData

    SynthData --> Assembler
    Assembler --> GNN
    GNN --> Trainer
    Trainer -->|model checkpoints| Checkpoints

    RealVideo --> RefineOpt
    Checkpoints --> RefineOpt
    RefineOpt --> Fusion
    Fusion --> PropOptimizer

    Checkpoints --> Inference
    Inference --> API
    API --> Predictions

    %% Supporting Infra connections
    SynthData --- Storage
    Checkpoints --- Storage
    Configs --- Trainer
    Logging --- Trainer

    %% Click events
    click Logging "https://github.com/alanyoung123/physworld/tree/main/figures/"
    click API "https://github.com/alanyoung123/physworld/blob/main/README.md"

    %% Styles
    classDef simulation fill:#a8d0e6,stroke:#1f78b4,stroke-width:1px
    classDef ml fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px
    classDef data fill:#ffe0b2,stroke:#ef6c00,stroke-width:1px
    classDef compute fill:#d1c4e9,stroke:#5e35b1,stroke-width:1px
    classDef io fill:#ffccbc,stroke:#bf360c,stroke-width:1px
```
