How congyewang/reinforcement-learning-for-adaptive-mcmc works
Project Type and Purpose
• It is a research-oriented project that combines reinforcement learning with adaptive Markov Chain Monte Carlo (MCMC) sampling.
• The README indicates dependencies spanning MATLAB, Python, and C/C++ (GCC) that support experiments, demos, and core algorithm implementations.
• The purpose is to implement reinforcement learning algorithms that adaptively adjust MCMC approaches, likely for exploring complex target distributions.
File Structure – Key Directories and Their Roles
• Source (src/rlmcmc):
• Contains the core implementation modules such as:
⋅ actor: Defines components like TwinNetworkLayer*.m files used for deep networks.
⋅ agent: Contains scripts (make_actor, make_critic, etc.) to build or train the agent.
⋅ env: Houses environment definitions (e.g., RLMHEnv*.m) that likely simulate MCMC proposals.
⋅ learn: Provides the learning loop/template (template_learning.m) to connect the agent with the environment.
⋅ target: Contains scripts that define various target distributions (e.g., Gaussian, mixture densities) and functions to compute log probability densities.
⋅ utils: Offers helper functions—for visualization, plotting, data export, and other supportive tasks.
• Demo:
• Provides multiple example experiments (with directories such as different_nn, mixture_gaussian_1D, mixture_gaussian_2D, etc.), each including run scripts, MATLAB learning scripts, and target definitions (often in Stan files). This directory shows how different configurations or network architectures (e.g., different number of hidden nodes) are tested.
• Experiments:
• Contains scripts and Makefiles to automate the scientific experiments: running multiple baselines, batch experiments, and integrating with external tools like Bridgestan and posteriordb.
• Root Level:
• Contains support files (e.g., .gitignore, CITATION.cff, README) and Git configuration (.gitmodules) that reflect version control and referencing.