# hardmaru/worldmodelsexperiments architecture

> Architecture diagram of the GitHub repository hardmaru/worldmodelsexperiments, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hardmaru/worldmodelsexperiments
- Repository: https://github.com/hardmaru/worldmodelsexperiments

## Overview

##### Main Components
- Environment Module: Represented by files like env.py in both experiments and additional modules (e.g., render_env.py, render_model.py) for visualization.
- World Model Components:
- VAE Component: In both directories, there is a “vae” folder (and tf_models/vae for doomrnn) that contains model definitions, configuration (JSON files), and training scripts (vae_train.py, vae_test.ipynb).
- RNN Component: Similarly, there is an “rnn” folder (or tf_models/rnn in doomrnn) which offers the sequential dynamics modeling via rnn.py and training scripts (rnn_train.py).
- Dream Model/Environment: Files such as dream_model.py and dream_env.py capture the “dreaming” aspect where the learned models are used to simulate future time steps and are critical to the planning.
- Optimization/Training Module:
- Training scripts (train.py, extract.py, extract.bash, gce_train.bash, gpu_jobs.bash) organize the training pipeline.
- The “es.py” file implements the evolutionary strategies optimization that evolves the policy based on the world model’s predictions.
- Data Management & Logging:
- Log folders with JSON logs and training progress visualizations (plot_training_progress.ipynb) capture experiment details.
- Files such as “initial_z.json” store initial latent representations.
- Experiment Differentiation:

##### Relationships and Interactions
- Cyclical feedback loop: the RNN predictions can be used to “dream” future states (through dream_env.py or dream_model.py) which are then used by the optimization module (e.g., via es.py) to evolve a controller.
- Logging/data flow paths between training scripts and storage (the “log” and “trainlog” directories) along with data extraction steps (extract.py, extract.bash).

##### Architectural Patterns and Design Principles
- The design follows a modular pattern where each logical component (environment simulation, unsupervised learning and sequence modeling, and policy evolution) is encapsulated into separate modules.
- There is a clear separation of concerns: model training modules (VAE and RNN) are distinct from the policy optimization modules, and these are both separately interfaced with the environment.
- Notice that there are specialized scripts for GPU job management – this hints at distributed or parallel training practices.

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Carracing Experiment"
        C_ENV["Environment Interface (env.py, render_env.py)"]:::env
        C_VAE["VAE Module (vae/)"]:::model
        C_RNN["RNN Module (rnn/)"]:::model
        C_DREAM["Dream Simulation (dream_env.py, dream_model.py)"]:::dream
        C_ES["Evolutionary Strategy (es.py, train.py)"]:::opt
        C_PIPE["Training Pipeline (extract.py, extract.bash, gpu_jobs.bash)"]:::train
        C_LOG["Logging & Data (log/, initial_z.json, plot_training_progress.ipynb)"]:::log
    end

    subgraph "Doomrnn Experiment"
        D_ENV["Environment Interface (env.py, doomreal.py)"]:::env
        D_VAE["VAE Module (tf_models/vae.json)"]:::model
        D_RNN["RNN Module (tf_models/rnn.json)"]:::model
        D_ES["Evolutionary Strategy (es.py, train.py)"]:::opt
        D_PIPE["Training Pipeline (extract.py, extract.bash, gpu_jobs.bash)"]:::train
        D_LOG["Logging & Data (log/, trainlog, log files)"]:::log
    end

    %% Carracing Flow
    C_ENV -->|"observation_data"| C_VAE
    C_VAE -->|"latent_representation"| C_RNN
    C_RNN -->|"dynamics_prediction"| C_DREAM
    C_DREAM -->|"simulated_future"| C_ES
    C_ES -->|"control_signal"| C_ENV
    C_PIPE -->|"training_data"| C_ES
    C_PIPE -->|"logs"| C_LOG

    %% Doomrnn Flow
    D_ENV -->|"observation_data"| D_VAE
    D_VAE -->|"latent_representation"| D_RNN
    D_RNN -->|"dynamics_prediction"| D_ES
    D_ES -->|"control_signal"| D_ENV
    D_PIPE -->|"training_data"| D_ES
    D_PIPE -->|"logs"| D_LOG

    %% Click Events for Carracing
    click C_ENV "https://github.com/hardmaru/worldmodelsexperiments/blob/master/carracing/env.py"
    click C_VAE "https://github.com/hardmaru/worldmodelsexperiments/tree/master/carracing/vae/"
    click C_RNN "https://github.com/hardmaru/worldmodelsexperiments/tree/master/carracing/rnn/"
    click C_DREAM "https://github.com/hardmaru/worldmodelsexperiments/blob/master/carracing/dream_env.py"
    click C_ES "https://github.com/hardmaru/worldmodelsexperiments/blob/master/carracing/es.py"
    click C_PIPE "https://github.com/hardmaru/worldmodelsexperiments/blob/master/carracing/extract.py"
    click C_LOG "https://github.com/hardmaru/worldmodelsexperiments/blob/master/carracing/plot_training_progress.ipynb"

    %% Click Events for Doomrnn
    click D_ENV "https://github.com/hardmaru/worldmodelsexperiments/blob/master/doomrnn/env.py"
    click D_VAE "https://github.com/hardmaru/worldmodelsexperiments/blob/master/doomrnn/tf_models/vae.json"
    click D_RNN "https://github.com/hardmaru/worldmodelsexperiments/blob/master/doomrnn/tf_models/rnn.json"
    click D_ES "https://github.com/hardmaru/worldmodelsexperiments/blob/master/doomrnn/es.py"
    click D_PIPE "https://github.com/hardmaru/worldmodelsexperiments/blob/master/doomrnn/extract.py"
    click D_LOG "https://github.com/hardmaru/worldmodelsexperiments/tree/master/doomrnn/log"

    %% Styles
    classDef env fill:#FAD7A0,stroke:#333,stroke-width:2px;
    classDef model fill:#AED6F1,stroke:#333,stroke-width:2px;
    classDef dream fill:#A9DFBF,stroke:#333,stroke-width:2px;
    classDef opt fill:#F9E79F,stroke:#333,stroke-width:2px;
    classDef train fill:#D2B4DE,stroke:#333,stroke-width:2px;
    classDef log fill:#F5CBA7,stroke:#333,stroke-width:2px;
```
