# facebookresearch/all-atom-diffusion-transformer architecture

> Architecture diagram of the GitHub repository facebookresearch/all-atom-diffusion-transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/facebookresearch/all-atom-diffusion-transformer
- Repository: https://github.com/facebookresearch/all-atom-diffusion-transformer

## Overview

##### Project type and purpose
- This is an ML research codebase built on the Lightning-Hydra template.
- It implements a two-stage generative pipeline:
a) A Variational Autoencoder (VAE) module mapping atomic graphs to latent space.
b) A Latent Diffusion Transformer (DiT) module generating new latent codes, decoded back to molecules/materials.

##### File structure
- Top-level directories:
- configs/ (Hydra YAML configs for every component)
- src/ (Python implementation of callbacks, data modules, models, training/evaluation scripts, tools)
- data/ (raw and processed datasets: QM9, MP20, QMOF)
- slurm/ (cluster launch scripts)
- Entry scripts in src/:
- train_autoencoder.py
- train_diffusion.py
- eval_autoencoder.py / eval_diffusion.py
- Key patterns:
- Hydra for configuration-driven instantiation
- PyTorch Lightning for training loops and callbacks
- Modular ML components (DataModule, LitModules for VAE & DiT)
- Separation of concerns via directories: models/components, data/components, callbacks, utils.

##### Extract additional insights from README
- The pipeline downloads/ processes datasets automatically via PyG/HuggingFace.
- VAE is trained first; its checkpoint and hyperparameters feed into DiT training.
- Logging is via W&B; SLURM scripts orchestrate multi-GPU jobs.
- Evaluation scripts sample new structures, saving CIF/PDB outputs.

##### Components and relationships
a. Main components:
1. User / CLI & Slurm scripts
2. Hydra Config System (configs/)
3. Entrypoint Scripts (train_autoencoder.py / train_diffusion.py / eval_*.py)
4. PyTorch Lightning Trainer
5. DataModule (data loading & preprocessing)
6. VAE LitModule (encoder + decoder)
7. DiT LitModule (latent diffusion transformer)
8. Logging & Callbacks (W&B, checkpointing, LR monitor, etc.)
9. Data Storage (raw → processed under data/)
10. External Services (PyG Datasets, HuggingFace datasets, W&B, HuggingFace Hub for weights)
11. Visualization Tools (ASE notebook under src/tools)
b. Relationships / data flow:
- CLI / Slurm → pass command to entrypoint script
- Entrypoint + Hydra → instantiate Trainer + DataModule + LitModule + Callbacks
- DataModule → loads raw/processed data → batches
- For VAE training: batches → VAE module → Trainer → checkpoints & logs
- For DiT training: Trainer loads frozen VAE checkpoint → DataModule yields VAE-encoded latents → DiT module → Trainer → logs & checkpoints
- Evaluation: saved checkpoints → eval scripts → generate samples → save CIF/PDB → logs
- W&B receives metrics & artifacts; HuggingFace Hub serves pretrained weights & datasets.
c. Architectural patterns:
- Configuration-driven instantiation (Hydra)
- Modular ML design (Lightning modules, DataModule)
- Two-stage pipeline (autoencoder → latent diffusion)
- Plugin callbacks for logging, early stopping, checkpoints
d. Key technologies:
- Python, PyTorch, PyTorch Lightning, Hydra
- PyTorch Geometric, HuggingFace datasets
- WandB, SLURM for multi-GPU training
- ASE notebook for visualization

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Orchestration Layer
    subgraph "Orchestration" 
        direction TB
        UserCLI["User / CLI"]:::orchestration
        subgraph "Slurm Scripts"
            direction TB
            TrainVAE["train_ddp_vae.sh"]:::orchestration
            TrainLDM["train_ddp_ldm.sh"]:::orchestration
            SweepGPU["sweep_gpu.sh"]:::orchestration
        end
        subgraph "Entrypoint Scripts"
            direction TB
            TrainAE["train_autoencoder.py"]:::orchestration
            TrainDiff["train_diffusion.py"]:::orchestration
            EvalAE["eval_autoencoder.py"]:::orchestration
            EvalDiff["eval_diffusion.py"]:::orchestration
        end
        Hydra["Hydra Config System"]:::orchestration
        Trainer["PyTorch Lightning Trainer"]:::orchestration
    end

    %% Data Components
    subgraph "Data Components" 
        direction TB
        DataModule["DataModule"]:::data
        CrystalDS["crystal_dataset.py"]:::data
        MP20DS["mp20_dataset.py"]:::data
        QMOFDS["qmof150_dataset.py"]:::data
        Preproc["preprocessing_utils.py"]:::data
    end

    %% Model Modules
    subgraph "Model Modules"
        direction TB
        subgraph "VAE Module" 
            direction TB
            VAE["VAE LitModule"]:::model
            EquiEnc["equiformer.py"]:::model
            TransEnc["transformer.py"]:::model
            EquiDec["equivariant_feedforward.py"]:::model
            TransDec["transformer.py"]:::model
        end
        subgraph "DiT Module"
            direction TB
            DiT["DiT LitModule"]:::model
            Denoiser["dit.py"]:::model
        end
    end

    %% Logging & Callbacks
    subgraph "Logging & Callbacks"
        direction TB
        EMA["ema.py"]:::callback
        CheckpointCfg["model_checkpoint.yaml"]:::callback
        LRMonCfg["learning_rate_monitor.yaml"]:::callback
        WandBCfg["wandb.yaml"]:::callback
        LoggingUtils["logging_utils.py"]:::callback
        PyLogger["pylogger.py"]:::callback
    end

    %% Data Storage
    subgraph "Data Storage"
        direction TB
        QM9Raw["qm9/raw"]:::storage
        QM9Proc["qm9/processed"]:::storage
        MP20Raw["mp_20/raw"]:::storage
        MP20Proc["mp_20/processed"]:::storage
        QMOFRaw["qmof/raw"]:::storage
        QMOFProc["qmof/processed"]:::storage
    end

    %% External Services
    subgraph "External Services"
        direction TB
        PyG["PyG Datasets"]:::external
        HFDS["HuggingFace Datasets"]:::external
        WnB["Weights & Biases"]:::external
        HFHub["HuggingFace Hub"]:::external
    end

    %% Visualization Tools
    subgraph "Visualization Tools"
        direction TB
        ASENotebook["ASE Notebook"]:::tool
        AtomInfo["atom_info.py"]:::tool
        Viewer["viewer.py"]:::tool
    end

    %% Relationships
    UserCLI -->|submits| TrainVAE
    UserCLI -->|submits| TrainLDM
    UserCLI -->|submits| SweepGPU
    TrainVAE --> TrainAE
    TrainLDM --> TrainDiff
    Hydra -->|configures| TrainAE
    Hydra -->|configures| TrainDiff
    TrainAE -->|instantiates| Trainer
    TrainDiff -->|instantiates| Trainer
    Trainer -->|uses| DataModule
    DataModule --> CrystalDS
    DataModule --> MP20DS
    DataModule --> QMOFDS
    DataModule --> Preproc
    Trainer -->|trains| VAE
    VAE --> EquiEnc
    VAE --> TransEnc
    VAE --> EquiDec
    VAE --> TransDec
    Trainer -->|trains| DiT
    DiT --> Denoiser
    Trainer -->|logs| WnB
    Trainer -->|saves ckpt| CheckpointCfg
    Trainer -->|monitors| LRMonCfg
    Trainer -->|callbacks| EMA
    Trainer -->|callbacks| LoggingUtils
    Trainer -->|callbacks| PyLogger
    DataModule -->|reads| QM9Raw
    DataModule -->|writes| QM9Proc
    DataModule -->|reads| MP20Raw
    DataModule -->|writes| MP20Proc
    DataModule -->|reads| QMOFRaw
    DataModule -->|writes| QMOFProc
    PyG -->|provides| CrystalDS
    HFDS -->|provides| QMOFDS
    HFDS -->|provides| MP20DS
    WnB -->|receives logs| WandBCfg
    HFHub -->|hosts weights| TrainDiff
    EvalAE -->|loads ckpt| CheckpointCfg
    EvalDiff -->|loads ckpt| CheckpointCfg
    EvalAE -->|generates samples| ASENotebook
    EvalDiff -->|generates samples| ASENotebook
    ASENotebook --> AtomInfo
    ASENotebook --> Viewer

    %% Click Events
    click TrainVAE "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/slurm/train_ddp_vae.sh"
    click TrainLDM "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/slurm/train_ddp_ldm.sh"
    click SweepGPU "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/slurm/sweep_gpu.sh"
    click TrainAE "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/train_autoencoder.py"
    click TrainDiff "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/train_diffusion.py"
    click EvalAE "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/eval_autoencoder.py"
    click EvalDiff "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/eval_diffusion.py"
    click Hydra "https://github.com/facebookresearch/all-atom-diffusion-transformer/tree/main/configs/"
    click CrystalDS "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/data/components/crystal_dataset.py"
    click MP20DS "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/data/components/mp20_dataset.py"
    click QMOFDS "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/data/components/qmof150_dataset.py"
    click Preproc "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/data/components/preprocessing_utils.py"
    click VAE "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/models/vae_module.py"
    click EquiEnc "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/models/encoders/equiformer.py"
    click TransEnc "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/models/encoders/transformer.py"
    click EquiDec "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/models/decoders/equivariant_feedforward.py"
    click TransDec "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/models/decoders/transformer.py"
    click DiT "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/models/ldm_module.py"
    click Denoiser "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/models/denoisers/dit.py"
    click EMA "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/callbacks/ema.py"
    click CheckpointCfg "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/configs/callbacks/model_checkpoint.yaml"
    click LRMonCfg "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/configs/callbacks/learning_rate_monitor.yaml"
    click WandBCfg "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/configs/logger/wandb.yaml"
    click LoggingUtils "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/utils/logging_utils.py"
    click PyLogger "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/utils/pylogger.py"
    click ASENotebook "https://github.com/facebookresearch/all-atom-diffusion-transformer/tree/main/src/tools/ase_notebook/"
    click AtomInfo "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/tools/ase_notebook/atom_info.py"
    click Viewer "https://github.com/facebookresearch/all-atom-diffusion-transformer/blob/main/src/tools/ase_notebook/viewer.py"

    %% Styles
    classDef orchestration fill:#FFA500,stroke:#333,stroke-width:1px
    classDef data fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef model fill:#98FB98,stroke:#333,stroke-width:1px
    classDef callback fill:#FFE4B5,stroke:#333,stroke-width:1px
    classDef storage fill:#B0C4DE,stroke:#333,stroke-width:1px
    classDef external fill:#D8BFD8,stroke:#333,stroke-width:1px
    classDef tool fill:#DDA0DD,stroke:#333,stroke-width:1px
```
