# bytedance/dplm architecture

> Architecture diagram of the GitHub repository bytedance/dplm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bytedance/dplm
- Repository: https://github.com/bytedance/dplm

## Overview

##### Project Type & Purpose
- This repository implements DPLM as described in the ICML’24 paper “Diffusion Language Models Are Versatile Protein Learners.”
- It is a full-scale deep learning project working with protein sequences – including functionalities for unconditional generation, motif scaffolding, inverse folding, and representation learning.
- Key aspects include model training (using PyTorch Lightning, Hydra for configuration, and distributed training), data processing (with custom datamodules for protein datasets like UniRef50 and CATH), and diverse evaluation pipelines (using analysis scripts and notebooks).

##### File Structure Analysis & Main Architectural Areas
- “configs/”: Contains various YAML configuration files – for callbacks, data modules, experiments, hydra configuration, and trainer setups. This indicates a heavily configurable architecture.
- “src/byprot/”: Holds the core implementation in a modular format. Within this folder:
- “datamodules/” handles data loading and preprocessing for different datasets.
- “models/lm/” defines model variants (dplm, cond_dplm, esm_dplm) and modules (adapter modules, transformer encoders).
- “tasks/lm/” contains scripts defining different training/inference tasks such as conditional and unconditional generation.
- “utils/” provides configurations, I/O, logging, learning rate scheduling, optimization, and strategy registries.
- “modules/” includes common components like cross entropy, metrics, and protein-specific metrics.
- “analysis/”: Holds scripts and notebooks for downstream analysis and evaluation (e.g., for generating plots, calculating scores).
- “scripts/” and “run/”: Provide auxiliary deployment and running scripts for downloading datasets and executing generation pipelines.
- “train.py” and “test.py”: Script entry points for training and testing pipelines.
- “generate.py” and “scaffold_generate.py”: Dedicated to protein sequence generation tasks.

a. Main Components:
- Data Ingestion and Preprocessing:
- Data modules for UniRef50 and CATH (located in “configs/datamodule” and “src/byprot/datamodules”).
- Download scripts and auxiliary preprocessed data (in “scripts/” & “data-bin/”).
- Model Definition & Modules:
- Core diffusion language model (DPLM) implementations (in “src/byprot/models/lm”).
- Submodules such as adapter modules and transformer encoders (in “src/byprot/models/lm/modules”).
- Training & Task Pipelines:
- Training pipeline script (“train.py”) and different task definitions (conditional/unconditional generation, inverse folding) in “src/byprot/tasks/lm/”.
- Utility functions (in “src/byprot/utils”).
- Configuration Management:
- Extensive configurations managed via Hydra (files in “configs/hydra”, “configs/experiment”, “configs/trainer”, etc.).
- Evaluation & Analysis:
- Analysis scripts for metrics calculation (e.g., pLDDT evaluation, motif RMSD) in “analysis/”.
- Notebooks for visualization (motif_analysis.ipynb, uncond_analysis.ipynb).
- Generation Modules:
- Scripts for generating protein sequences (“generate.py”, “scaffold_generate.py”, “run/” directory scripts).

b. Relationships and Interactions:
- The configuration management layer interacts with every part (data modules, model structure, training strategy, logging).
- Model definitions in “src/byprot/models/” are used by both the training tasks and by generation scripts.
- Utility modules provide common functionality (I/O, logging, strategies) and are accessed by multiple components.
- Analysis modules take generation outputs and trained model checkpoints as inputs for evaluation metrics and downstream assessments.
- External dependencies like PyTorch Lightning, Hydra, and ESMFold (for structure prediction in analysis) should be noted as integrations.

c. Architectural Patterns & Design Principles:
- Modular design: Clear separation of concerns (data, model, training, evaluation).
- Configuration-driven development: Hydra configuration files allow customization across experiments.
- Reusability: Shared utilities and module components are designed for multiple tasks (generation, representation learning).
- Distributed training: Configurations and scripts supporting multi-GPU/multi-node training.

d. Relevant Technologies & Frameworks:
- PyTorch and PyTorch Lightning for training and model implementation.
- Hydra for configuration management.
- ESMFold for secondary evaluation of generated protein structures.
- External tools for data downloading and processing (scripts in “scripts/”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing"
        D1["Data Configurations (configs/datamodule)"]:::data
        D2["Data Modules (src/byprot/datamodules)"]:::data
        D3a["download_uniref50.sh (scripts)"]:::data
        D3b["download_cath.sh (scripts)"]:::data
        D3c["Data Store: uniref50 (data-bin)"]:::data
        D3d["Data Store: scaffolding-pdbs (data-bin)"]:::data
    end

    %% Model Definition & Modules
    subgraph "Model Definition & Modules"
        M1["Core Models (src/byprot/models/lm)"]:::model
        M2["Model Submodules (src/byprot/models/lm/modules)"]:::model
    end

    %% Training & Task Pipelines
    subgraph "Training & Task Pipelines"
        T1["Entry Point: train.py"]:::training
        T2["Training Pipeline (src/byprot/training_pipeline.py)"]:::training
        T3["Task Definitions (src/byprot/tasks/lm)"]:::training
        T4["Utility Functions (src/byprot/utils)"]:::training
    end

    %% Configuration Management
    subgraph "Configuration Management"
        C1["Hydra & Experiment Configs (configs/hydra, configs/experiment)"]:::config
        C2["Trainer Configs (configs/trainer)"]:::config
        C3["General Configs (configs/config.yaml, configs/paths)"]:::config
    end

    %% Evaluation & Analysis
    subgraph "Evaluation & Analysis"
        E1["Analysis Scripts & Notebooks (analysis)"]:::analysis
    end

    %% Generation Modules
    subgraph "Generation Modules"
        G1["Generation Scripts (generate.py, scaffold_generate.py)"]:::generation
        G2["Auxiliary Generation Pipelines (run)"]:::generation
    end

    %% External Dependencies
    subgraph "External Dependencies"
        X1["PyTorch Lightning"]:::external
        X2["Hydra"]:::external
        X3["ESMFold"]:::external
        X4["PyTorch"]:::external
    end

    %% Connections - Data Ingestion to Training Pipeline
    D1 -->|"feeds"| T2
    D2 -->|"feeds"| T2
    D3a -->|"provides raw data"| T2
    D3b -->|"provides raw data"| T2
    D3c -->|"data source"| T2
    D3d -->|"data source"| T2

    %% Connections - Training Pipeline and Tasks
    T2 -->|"executes tasks"| T3
    T4 -->|"supports"| T2
    T4 -->|"supports"| T3
    T1 -->|"initiates"| T2

    %% Connections - Model Definition integrated into Training and Generation
    M1 -->|"used by"| T2
    M1 -->|"used by"| G1
    M2 -->|"complements"| M1

    %% Connections - Configuration Management influencing various components
    C1 -->|"configures"| D1
    C1 -->|"configures"| T2
    C1 -->|"configures"| M1
    C2 -->|"configures"| T2
    C3 -->|"provides paths to"| T2

    %% Connections - Generation Modules
    T1 -->|"outputs trained models to"| G1
    T2 -->|"outputs checkpoints to"| G1
    G1 -->|"produces outputs for"| E1

    %% Connections - Evaluation & Analysis using training & generation outputs
    T1 -->|"provides models for evaluation"| E1

    %% External Dependencies
    X1 -->|"orchestrates training"| T2
    X4 -->|"underpins"| M1
    X4 -->|"underpins"| T2
    X2 -->|"configures via Hydra"| C1
    X3 -->|"evaluates structure"| E1

    %% Click Events for Component Mapping
    click D1 "https://github.com/bytedance/dplm/tree/main/configs/datamodule"
    click D2 "https://github.com/bytedance/dplm/tree/main/src/byprot/datamodules"
    click D3a "https://github.com/bytedance/dplm/blob/main/scripts/download_uniref50.sh"
    click D3b "https://github.com/bytedance/dplm/blob/main/scripts/download_cath.sh"
    click D3c "https://github.com/bytedance/dplm/tree/main/data-bin/uniref50"
    click D3d "https://github.com/bytedance/dplm/tree/main/data-bin/scaffolding-pdbs"
    click M1 "https://github.com/bytedance/dplm/tree/main/src/byprot/models/lm"
    click M2 "https://github.com/bytedance/dplm/tree/main/src/byprot/models/lm/modules"
    click T1 "https://github.com/bytedance/dplm/blob/main/train.py"
    click T2 "https://github.com/bytedance/dplm/blob/main/src/byprot/training_pipeline.py"
    click T3 "https://github.com/bytedance/dplm/tree/main/src/byprot/tasks/lm"
    click T4 "https://github.com/bytedance/dplm/tree/main/src/byprot/utils"
    click C1 "https://github.com/bytedance/dplm/tree/main/configs/hydra"
    click C2 "https://github.com/bytedance/dplm/tree/main/configs/trainer"
    click C3 "https://github.com/bytedance/dplm/blob/main/configs/config.yaml"
    click E1 "https://github.com/bytedance/dplm/tree/main/analysis"
    click G1 "https://github.com/bytedance/dplm/blob/main/generate.py"
    click G2 "https://github.com/bytedance/dplm/tree/main/run"

    %% Styles
    classDef data fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef model fill:#A9DFBF,stroke:#1E8449,stroke-width:2px;
    classDef training fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef config fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
    classDef analysis fill:#D2B4DE,stroke:#6C3483,stroke-width:2px;
    classDef generation fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
    classDef external fill:#E8DAEF,stroke:#8E44AD,stroke-width:2px;
```
