How sinzlab/energy-guided-diffusion works
Project type and purpose
• This repository is an open‑source research tool implementing “Energy Guided Diffusion” (EGG) on top of OpenAI’s guided‑diffusion framework. Its goal is to generate neurally exciting images by combining a pre‑trained diffusion model with a user‑defined energy function (e.g. neural response predictor).
File structure
Top‑level directories and files:
• Dockerfile, docker‑compose.yml, .env‑placeholder: define a reproducible environment.
• README.md, docs/: user documentation and experiment instructions.
• assets/: static images (e.g. teaser).
• download_requirements.sh: fetches large external packages.
• setup.py: installs this project as a Python package.
Core library “egg/”:
• egg/diffusion.py: the EGG wrapper around guided diffusion.
• egg/guided_diffusion/: a vendored copy of OpenAI’s guided‑diffusion code (data sampling, model definitions, training utilities).
• egg/models/: model definitions split into “data_driven” and “task_driven” cores (e.g. grayscale model implementations).
• egg/optimizers/: plug‑in optimizers (e.g. gradient_ascent).
• egg/palettes.py: visualization helpers.
Auxiliary code:
• lib/: pip‑installable local packages (nnvision, mei) providing neural‐vision utilities.
• scripts/: CLI scripts to run full experiments (e.g. meis_egg.py, menis.py, reconstruct_gd.py).
• notebooks/: Jupyter notebooks for analysis and plotting results.
README for additional insights
• Describes usage: how to instantiate EGG(diffusion_artefact, num_steps), supply an energy_fn (neural predictor), and call .sample().
• Explains how to build the Docker image (docker compose build base) and run experiments via docker compose run --rm python scripts/....
• Emphasizes reproducibility: containerization + pip editable installs.