# alec-tschantz/vime architecture

> Architecture diagram of the GitHub repository alec-tschantz/vime, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alec-tschantz/vime
- Repository: https://github.com/alec-tschantz/vime

## Overview

##### Main Components
- Experiment Runner:
- Files: run_experiment.py, experiment.py
- Role: Orchestration of experiments, setting up training loops, managing experiment configurations and execution.
- Policy/Agent:
- File: policy.py
- Role: Implements the agent’s action selection and optimization components. For SAC integration, this module likely contains the logic for the actor-critic network interactions and policy updates.
- Dynamics / Model Training:
- File: train_dynamics.py
- Role: Handles the training of the model dynamics or the Bayesian neural network components that estimate uncertainty within the environment. This is central to the VIME approach.
- Replay Buffer:
- File: replay_pool.py
- Role: Manages storage and sampling of experiences to stabilize learning, ensuring the model trains on diverse past experiences.
- Sampling & Data Collection:
- File: sampler.py
- Role: Implements the mechanism for interacting with the environment (or simulated environment) and collecting trajectories or samples for training.
- Model Definition:
- File: feedforward.py, bnn.py
- Role: Contains network architectures – feedforward networks for policy/value approximations and Bayesian neural network (BNN) implementations for capturing uncertainty and information gain.
- Utilities & Logging:
- Files: utils.py, logger.py
- Role: Provides helper functions, common utilities, and logging functionality to track experiment progress and debug issues.

##### Relationships and Interactions
- The Experiment Runner (run_experiment.py / experiment.py) is the entry point. It initializes other modules, sets experiment configurations, and controls the overall flow.
- The Sampler invokes policy decisions (from policy.py) and gathers data from the environment. Collected data is then stored in the Replay Pool.
- The Agent/Policy component consumes data from the Replay Pool for training. Simultaneously, the Dynamics Model (train_dynamics.py) is updated, possibly by leveraging feedforward architectures (feedforward.py) and Bayesian methods (bnn.py) to accurately model uncertainty.
- Logging and utility modules provide support to all these components ensuring that operations are traceable and errors can be diagnosed efficiently.

##### Architectural Patterns and Design Principles
- Modularity: Each file represents a distinct module (e.g., dynamics, policy, replay buffer) that communicates with others through well-defined interfaces.
- Separation of Concerns: The project separates experiment orchestration, data management, model training, and utility functions.
- Data Flow: There is a clear flow from environment-sampling (sampler.py) to data storage (replay_pool.py), followed by training (train_dynamics.py & policy.py) and orchestration (run_experiment.py).
- Integration of Advanced Techniques: Bayesian methods implemented in bnn.py for uncertainty and exploration; integration of SAC for policy optimization.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Experiment Orchestrator"
        EXP1["experiment.py"]:::orchestrator
        EXP2["run_experiment.py"]:::orchestrator
    end

    S["sampler.py"]:::sampler
    RB["replay_pool.py"]:::buffer
    POL["policy.py"]:::policy
    DT["train_dynamics.py"]:::dynamics

    subgraph "Model Definitions"
        MD1["feedforward.py"]:::model
        MD2["bnn.py"]:::model
    end

    subgraph "Utilities & Logging"
        UT1["utils.py"]:::utilities
        UT2["logger.py"]:::utilities
    end

    %% Main Data Flow
    EXP2 -->|"initiate"| S
    S -->|"collects_experience"| RB
    RB -->|"provides_data"| POL
    RB -->|"provides_data"| DT
    EXP2 -->|"triggers_policy"| POL
    EXP2 -->|"triggers_training"| DT

    %% Model Definitions supporting Policy and Dynamics
    MD1 -->|"supports"| POL
    MD1 -->|"supports"| DT
    MD2 -->|"supports"| POL
    MD2 -->|"supports"| DT

    %% Utilities and Logging connections
    EXP1 -->|"logs"| UT1
    EXP2 -->|"logs"| UT2
    S    -->|"logs"| UT1
    RB   -->|"logs"| UT1
    POL  -->|"logs"| UT2
    DT   -->|"logs"| UT2

    %% Click Events
    click EXP1 "https://github.com/alec-tschantz/vime/blob/master/experiment.py"
    click EXP2 "https://github.com/alec-tschantz/vime/blob/master/run_experiment.py"
    click S "https://github.com/alec-tschantz/vime/blob/master/sampler.py"
    click RB "https://github.com/alec-tschantz/vime/blob/master/replay_pool.py"
    click POL "https://github.com/alec-tschantz/vime/blob/master/policy.py"
    click DT "https://github.com/alec-tschantz/vime/blob/master/train_dynamics.py"
    click MD1 "https://github.com/alec-tschantz/vime/blob/master/feedforward.py"
    click MD2 "https://github.com/alec-tschantz/vime/blob/master/bnn.py"
    click UT1 "https://github.com/alec-tschantz/vime/blob/master/utils.py"
    click UT2 "https://github.com/alec-tschantz/vime/blob/master/logger.py"

    %% Styles
    classDef orchestrator fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef sampler fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
    classDef buffer fill:#D5F5E3,stroke:#1E8449,stroke-width:2px;
    classDef policy fill:#A9DFBF,stroke:#229954,stroke-width:2px;
    classDef dynamics fill:#FADBD8,stroke:#C0392B,stroke-width:2px;
    classDef model fill:#D2B4DE,stroke:#6C3483,stroke-width:2px;
    classDef utilities fill:#F5CBA7,stroke:#873600,stroke-width:2px;
```
