How facebookresearch/all-atom-diffusion-transformer works
Project type and purpose
• This is an ML research codebase built on the Lightning-Hydra template.
• It implements a two-stage generative pipeline:
a) A Variational Autoencoder (VAE) module mapping atomic graphs to latent space.
b) A Latent Diffusion Transformer (DiT) module generating new latent codes, decoded back to molecules/materials.
File structure
• Top-level directories:
• configs/ (Hydra YAML configs for every component)
• src/ (Python implementation of callbacks, data modules, models, training/evaluation scripts, tools)
• data/ (raw and processed datasets: QM9, MP20, QMOF)
• slurm/ (cluster launch scripts)
• Entry scripts in src/:
• train_autoencoder.py
• train_diffusion.py
• eval_autoencoder.py / eval_diffusion.py
• Key patterns:
• Hydra for configuration-driven instantiation
• PyTorch Lightning for training loops and callbacks
• Modular ML components (DataModule, LitModules for VAE & DiT)
• Separation of concerns via directories: models/components, data/components, callbacks, utils.
Extract additional insights from README
• The pipeline downloads/ processes datasets automatically via PyG/HuggingFace.
• VAE is trained first; its checkpoint and hyperparameters feed into DiT training.
• Logging is via W&B; SLURM scripts orchestrate multi-GPU jobs.
• Evaluation scripts sample new structures, saving CIF/PDB outputs.
Components and relationships
a. Main components:
1. User / CLI & Slurm scripts
2. Hydra Config System (configs/)
3. Entrypoint Scripts (train_autoencoder.py / train_diffusion.py / eval_*.py)
4. PyTorch Lightning Trainer
5. DataModule (data loading & preprocessing)
6. VAE LitModule (encoder + decoder)
7. DiT LitModule (latent diffusion transformer)
8. Logging & Callbacks (W&B, checkpointing, LR monitor, etc.)
9. Data Storage (raw → processed under data/)
10. External Services (PyG Datasets, HuggingFace datasets, W&B, HuggingFace Hub for weights)
11. Visualization Tools (ASE notebook under src/tools)
b. Relationships / data flow:
• CLI / Slurm → pass command to entrypoint script
• Entrypoint + Hydra → instantiate Trainer + DataModule + LitModule + Callbacks
• DataModule → loads raw/processed data → batches
• For VAE training: batches → VAE module → Trainer → checkpoints & logs
• For DiT training: Trainer loads frozen VAE checkpoint → DataModule yields VAE-encoded latents → DiT module → Trainer → logs & checkpoints
• Evaluation: saved checkpoints → eval scripts → generate samples → save CIF/PDB → logs
• W&B receives metrics & artifacts; HuggingFace Hub serves pretrained weights & datasets.
c. Architectural patterns:
• Configuration-driven instantiation (Hydra)
• Modular ML design (Lightning modules, DataModule)
• Two-stage pipeline (autoencoder → latent diffusion)
• Plugin callbacks for logging, early stopping, checkpoints
d. Key technologies:
• Python, PyTorch, PyTorch Lightning, Hydra
• PyTorch Geometric, HuggingFace datasets
• WandB, SLURM for multi-GPU training
• ASE notebook for visualization