# google/dopamine architecture

> Architecture diagram of the GitHub repository google/dopamine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/google/dopamine
- Repository: https://github.com/google/dopamine

## Overview

##### Project type and purpose
- This is an open-source research framework for rapid prototyping of reinforcement-learning (RL) agents, called “Dopamine.”
- It provides battle-tested implementations of algorithms (DQN, Rainbow, IQN, SAC, PPO) in both TensorFlow (legacy) and JAX, plus experiment runners, replay buffers, metrics, plotting utilities, Colab notebooks, baselines data, Docker containers, and extensive documentation.

##### File structure
Top-level directories and purposes:
- dopamine/        → core Python package
– continuous_domains/ discrete_domains/   → train & run scripts for Gym/ALE/MuJoCo
– jax/             → JAX-based agents, networks, replay memory, checkpointers
– tf/              → TensorFlow-based agents, replay memory
– labs/            → experimental extensions (Atari-100k, MOEs, offline-RL, etc.)
– colab/           → Jupyter notebooks & helpers for interactive use
– metrics/         → collectors, dispatchers for console, TensorBoard, pickle
– utils/           → plotting, visualization, test utilities
- baselines/        → JSON baseline results & static plots for Atari & MuJoCo
- docs/             → Sphinx/Markdown source for API docs & user guides
- docker/           → Dockerfiles for core, Atari, and MuJoCo environments
- tests/            → unit/integration tests covering all subpackages

##### README for insights
- Confirms design goals (easy experimentation, compactness, reproducibility).
- Notes both TF (legacy) and JAX (actively maintained) implementations.
- Highlights use of Gin configuration files (.gin) to drive experiments.
- Provides pointers to Colab notebooks, baselines data, and Docker usage.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interfaces
    subgraph "User Interfaces"
        direction TB
        Discrete_Run["Discrete CLI: run_experiment.py"]:::ui
        Discrete_Train["Discrete CLI: train.py"]:::ui
        Continuous_Run["Continuous CLI: run_experiment.py"]:::ui
        Continuous_Train["Continuous CLI: train.py"]:::ui
        Colab["Jupyter/Colab Notebooks"]:::ui
    end

    %% Core Framework
    subgraph "Dopamine Library"
        direction TB
        GinLoader["Gin Configuration Loader"]:::core
        Runner["Experiment Runner"]:::core

        subgraph "Environment API"
            direction TB
            AtariLib["Atari/Gym API"]:::core
            GymLib["Gym API"]:::core
        end

        subgraph "Agent Implementations"
            direction TB
            JAXAgents["JAX-based Agents"]:::agent
            TFAgents["TF-based Agents"]:::agent
            Labs["Experimental Labs"]:::labs
        end

        subgraph "Networks & Replay Memory"
            direction TB
            JAXNetworks["JAX Networks"]:::core
            TFNetworks["TF Networks"]:::core
            JAXReplay["JAX Replay Buffers"]:::core
            TFReplay["TF Replay Buffers"]:::core
        end

        Trainer["Trainer & Optimizer"]:::core

        subgraph "Checkpointing & Metrics"
            direction TB
            DiscreteCP["Discrete Checkpointer"]:::core
            JAXCP["JAX Checkpointers"]:::core
            Metrics["Metrics & Logging"]:::util
        end

        Plotter["Baselines & Plotting"]:::util
    end

    %% External Resources
    Baselines["Baselines Data"]:::ext
    subgraph "Docker & Deployment"
        direction TB
        CoreDocker["Core Dockerfile"]:::ext
        AtariDocker["Atari Dockerfile"]:::ext
        MujocoDocker["MuJoCo Dockerfile"]:::ext
    end
    Docs["Documentation"]:::ext
    Tests["Tests"]:::ext

    %% Connections
    Discrete_Run -->|load config| GinLoader
    Discrete_Train -->|load config| GinLoader
    Continuous_Run -->|load config| GinLoader
    Continuous_Train -->|load config| GinLoader
    Colab -->|interactive| GinLoader

    GinLoader -->|orchestrate| Runner
    Runner -->|step| AtariLib
    Runner -->|step| GymLib
    Runner -->|action/obs| JAXAgents
    Runner -->|action/obs| TFAgents
    Runner -->|action/obs| Labs

    JAXAgents -->|use network| JAXNetworks
    TFAgents -->|use network| TFNetworks

    Runner -->|store| JAXReplay
    Runner -->|store| TFReplay

    Runner -->|train| Trainer
    Trainer -->|sample| JAXReplay
    Trainer -->|sample| TFReplay
    Trainer -->|update| JAXAgents
    Trainer -->|update| TFAgents

    Runner -->|save| DiscreteCP
    Runner -->|save| JAXCP
    Runner -->|log| Metrics

    Runner -->|generate| Plotter
    Plotter -->|compare| Baselines

    Runner -->|deploy| CoreDocker
    Runner -->|deploy| AtariDocker
    Runner -->|deploy| MujocoDocker

    Runner -->|docs| Docs
    Runner -->|tests| Tests

    %% Click Events
    click Discrete_Run "https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/run_experiment.py"
    click Discrete_Train "https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/train.py"
    click Continuous_Run "https://github.com/google/dopamine/blob/master/dopamine/continuous_domains/run_experiment.py"
    click Continuous_Train "https://github.com/google/dopamine/blob/master/dopamine/continuous_domains/train.py"
    click Colab "https://github.com/google/dopamine/tree/master/dopamine/colab/"
    click AtariLib "https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/atari_lib.py"
    click GymLib "https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/gym_lib.py"
    click JAXAgents "https://github.com/google/dopamine/tree/master/dopamine/jax/agents/"
    click TFAgents "https://github.com/google/dopamine/tree/master/dopamine/tf/agents/"
    click Labs "https://github.com/google/dopamine/tree/master/dopamine/labs/"
    click JAXReplay "https://github.com/google/dopamine/tree/master/dopamine/jax/replay_memory/"
    click TFReplay "https://github.com/google/dopamine/tree/master/dopamine/tf/replay_memory/"
    click DiscreteCP "https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/checkpointer.py"
    click JAXCP "https://github.com/google/dopamine/blob/master/dopamine/jax/checkpointers.py"
    click Metrics "https://github.com/google/dopamine/blob/master/dopamine/metrics/collector.py"
    click Plotter "https://github.com/google/dopamine/blob/master/dopamine/utils/atari_plotter.py"
    click CoreDocker "https://github.com/google/dopamine/tree/master/docker/core/Dockerfile"
    click AtariDocker "https://github.com/google/dopamine/tree/master/docker/atari/Dockerfile"
    click MujocoDocker "https://github.com/google/dopamine/tree/master/docker/mujoco/Dockerfile"

    %% Styles
    classDef ui fill:#f9f,stroke:#333,stroke-width:1px
    classDef core fill:#bbf,stroke:#333,stroke-width:1px
    classDef agent fill:#bfb,stroke:#333,stroke-width:1px
    classDef util fill:#fb8,stroke:#333,stroke-width:1px
    classDef labs fill:#d8b,stroke:#333,stroke-width:1px
    classDef ext fill:#eee,stroke:#333,stroke-width:1px
```
