# congyewang/reinforcement-learning-for-adaptive-mcmc architecture

> Architecture diagram of the GitHub repository congyewang/reinforcement-learning-for-adaptive-mcmc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/congyewang/reinforcement-learning-for-adaptive-mcmc
- Repository: https://github.com/congyewang/reinforcement-learning-for-adaptive-mcmc

## Overview

##### Project Type and Purpose
- It is a research-oriented project that combines reinforcement learning with adaptive Markov Chain Monte Carlo (MCMC) sampling.
- The README indicates dependencies spanning MATLAB, Python, and C/C++ (GCC) that support experiments, demos, and core algorithm implementations.
- The purpose is to implement reinforcement learning algorithms that adaptively adjust MCMC approaches, likely for exploring complex target distributions.

##### File Structure – Key Directories and Their Roles
- Source (src/rlmcmc):
- Contains the core implementation modules such as:
⋅ actor: Defines components like TwinNetworkLayer*.m files used for deep networks.
⋅ agent: Contains scripts (make_actor, make_critic, etc.) to build or train the agent.
⋅ env: Houses environment definitions (e.g., RLMHEnv*.m) that likely simulate MCMC proposals.
⋅ learn: Provides the learning loop/template (template_learning.m) to connect the agent with the environment.
⋅ target: Contains scripts that define various target distributions (e.g., Gaussian, mixture densities) and functions to compute log probability densities.
⋅ utils: Offers helper functions—for visualization, plotting, data export, and other supportive tasks.

- Demo:
- Provides multiple example experiments (with directories such as different_nn, mixture_gaussian_1D, mixture_gaussian_2D, etc.), each including run scripts, MATLAB learning scripts, and target definitions (often in Stan files). This directory shows how different configurations or network architectures (e.g., different number of hidden nodes) are tested.

- Experiments:
- Contains scripts and Makefiles to automate the scientific experiments: running multiple baselines, batch experiments, and integrating with external tools like Bridgestan and posteriordb.

- Root Level:
- Contains support files (e.g., .gitignore, CITATION.cff, README) and Git configuration (.gitmodules) that reflect version control and referencing.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Top-level nodes
    EXP("Experiment Runner"):::exp
    DEMO("Demo"):::demo
    EXT("External Tools"):::external

    %% Core RL-MCMC Library Subgraph
    subgraph "Core RL-MCMC Library"
        AGENT("Agent"):::core
        ACTOR("Actor"):::core
        LEARN("Learning Module"):::core
        ENV("Environment"):::core
        TARGET("Target Distribution"):::core
        UTILS("Utilities"):::core
    end

    %% Interactions between top-level and core modules
    EXP -->|"initiates"| LEARN
    DEMO -->|"initiates"| LEARN
    EXP -->|"integrates"| EXT

    %% Internal Core Library interactions
    AGENT -->|"builds"| ACTOR
    AGENT -->|"updates"| LEARN
    LEARN -->|"simulates"| ENV
    LEARN -->|"obtains"| TARGET

    %% Utility feedback loop
    UTILS -->|"visualizes"| EXP

    %% Click events for component mapping
    click ACTOR "https://github.com/congyewang/reinforcement-learning-for-adaptive-mcmc/tree/main/src/rlmcmc/actor"
    click AGENT "https://github.com/congyewang/reinforcement-learning-for-adaptive-mcmc/tree/main/src/rlmcmc/agent"
    click ENV "https://github.com/congyewang/reinforcement-learning-for-adaptive-mcmc/tree/main/src/rlmcmc/env"
    click LEARN "https://github.com/congyewang/reinforcement-learning-for-adaptive-mcmc/tree/main/src/rlmcmc/learn"
    click TARGET "https://github.com/congyewang/reinforcement-learning-for-adaptive-mcmc/tree/main/src/rlmcmc/target"
    click UTILS "https://github.com/congyewang/reinforcement-learning-for-adaptive-mcmc/tree/main/src/rlmcmc/utils"
    click DEMO "https://github.com/congyewang/reinforcement-learning-for-adaptive-mcmc/tree/main/demo"
    click EXP "https://github.com/congyewang/reinforcement-learning-for-adaptive-mcmc/tree/main/experiments"

    %% Style definitions
    classDef core fill:#cce5ff,stroke:#003366,stroke-width:2px;
    classDef demo fill:#d1e7dd,stroke:#0f5132,stroke-width:2px;
    classDef exp fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef external fill:#f8d7da,stroke:#721c24,stroke-width:2px;
```
