How kartik13-op/world-model works
AI World Model V3 turns source video into frame/action training data, learns an autoencoder plus action-conditioned latent dynamics, and provides interactive camera-driven playback with optional idle-time scene reconstruction. The README and sampled code support the main pipeline and GUI branches; playback internals are only partially excerpted, so some documented relationships are cited to the README rather than inferred from unseen code.
Main entry points: main.py CLI Commands · gui.py Desktop GUI · play.py Interactive Playback · train.py World Model Training
Components 17
User Interfaces
- CLI Commands
main.py - Desktop GUI
gui.py
Video Preparation
- Video Preprocessing
world_model/preprocess.py
Model Learning
- World Model Training
world_model/train.py - World Model
world_model/model.py - Reconstructor Training
world_model/reconstructor.py - Reconstruction Engine
world_model/upscaler.py
Interactive Runtime
- Interactive Playback
world_model/play.py - Camera Controls
world_model/play.py - Display Output
world_model/play.py
Project State
- Frames and Actions
world_model/config.py - Project Paths
world_model/config.py - World Checkpoint
world_model/config.py - Reconstructor Checkpoint
world_model/config.py - Pipeline Verification
world_model/verify.py
Other components
- User
- Source Video
Connections 32
- User → CLI Commands: runs commands
README.md - User → Desktop GUI: opens interface
README.md - CLI Commands → Desktop GUI: launches
main.py - Desktop GUI → Video Preprocessing: starts preprocessing
gui.py - Desktop GUI → World Model Training: starts training
gui.py - Desktop GUI → Reconstructor Training: starts training
gui.py - Desktop GUI → Interactive Playback: starts playback
gui.py - Desktop GUI → Pipeline Verification: runs checks
gui.py - CLI Commands → Video Preprocessing: dispatches to
main.py - CLI Commands → World Model Training: dispatches to
README.md - CLI Commands → Reconstructor Training: dispatches to
main.py - CLI Commands → Interactive Playback: dispatches to
main.py - CLI Commands → Pipeline Verification: dispatches to
README.md - Source Video → Video Preprocessing: supplies footage
README.md - Video Preprocessing → Project Paths: creates project
world_model/preprocess.py - Video Preprocessing → Frames and Actions: writes frames/actions
world_model/preprocess.py - World Model Training → Frames and Actions: reads training data
world_model/train.py - World Model Training → World Model: trains
world_model/train.py - World Model Training → World Checkpoint: saves checkpoint
README.md - Reconstructor Training → Reconstruction Engine: delegates training
world_model/reconstructor.py - Reconstructor Training → Reconstructor Checkpoint: saves checkpoint
README.md - Interactive Playback → Camera Controls: reads input
README.md - Interactive Playback → World Model: steps and decodes
README.md - Interactive Playback → Reconstructor Checkpoint: loads optional model
world_model/play.py - Interactive Playback → Reconstruction Engine: reconstructs idle scene
world_model/play.py - Interactive Playback → Display Output: renders frames
README.md - Pipeline Verification → Frames and Actions: checks data
world_model/verify.py - Pipeline Verification → World Checkpoint: checks checkpoint
world_model/verify.py - Pipeline Verification → Reconstructor Checkpoint: checks checkpoint
world_model/verify.py - Project Paths → Frames and Actions: defines locations
world_model/config.py - Project Paths → World Checkpoint: defines location
world_model/config.py - Project Paths → Reconstructor Checkpoint: defines location
world_model/config.py