How hardmaru/worldmodelsexperiments works
Main Components
• Environment Module: Represented by files like env.py in both experiments and additional modules (e.g., render_env.py, render_model.py) for visualization.
• World Model Components:
• VAE Component: In both directories, there is a “vae” folder (and tf_models/vae for doomrnn) that contains model definitions, configuration (JSON files), and training scripts (vae_train.py, vae_test.ipynb).
• RNN Component: Similarly, there is an “rnn” folder (or tf_models/rnn in doomrnn) which offers the sequential dynamics modeling via rnn.py and training scripts (rnn_train.py).
• Dream Model/Environment: Files such as dream_model.py and dream_env.py capture the “dreaming” aspect where the learned models are used to simulate future time steps and are critical to the planning.
• Optimization/Training Module:
• Training scripts (train.py, extract.py, extract.bash, gce_train.bash, gpu_jobs.bash) organize the training pipeline.
• The “es.py” file implements the evolutionary strategies optimization that evolves the policy based on the world model’s predictions.
• Data Management & Logging:
• Log folders with JSON logs and training progress visualizations (plot_training_progress.ipynb) capture experiment details.
• Files such as “initial_z.json” store initial latent representations.
• Experiment Differentiation:
Relationships and Interactions
• Cyclical feedback loop: the RNN predictions can be used to “dream” future states (through dream_env.py or dream_model.py) which are then used by the optimization module (e.g., via es.py) to evolve a controller.
• Logging/data flow paths between training scripts and storage (the “log” and “trainlog” directories) along with data extraction steps (extract.py, extract.bash).
Architectural Patterns and Design Principles
• The design follows a modular pattern where each logical component (environment simulation, unsupervised learning and sequence modeling, and policy evolution) is encapsulated into separate modules.
• There is a clear separation of concerns: model training modules (VAE and RNN) are distinct from the policy optimization modules, and these are both separately interfaced with the environment.
• Notice that there are specialized scripts for GPU job management – this hints at distributed or parallel training practices.