# junjie-zhu/idpfold architecture

> Architecture diagram of the GitHub repository junjie-zhu/idpfold, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/junjie-zhu/idpfold
- Repository: https://github.com/junjie-zhu/idpfold

## Overview

1. **Overall shape**
   IDPFold is a PyTorch Lightning project wrapped in Hydra configuration. The repository is organized around two executable entrypoints, `src/train.py` for model training and `src/eval.py` for ensemble generation/evaluation, with `src/read_seqs.py` handling sequence preprocessing. The codebase is aimed at protein conformation generation from sequence, specifically intrinsically disordered proteins.

2. **Top-level workflow**
   The main data flow is: FASTA sequence input → sequence embedding extraction with ESM → model inference or training data loading → diffusion-based structure sampling → output analysis/metrics/checkpoints. The repo is built to support both offline training on curated protein datasets and standalone inference from user-provided sequences.

3. **Configuration and experiment management**
   The `configs/` tree is the control plane. Hydra config groups cover data, model, trainer, logging, callbacks, paths, debugging, and hyperparameter search. This means most runtime behavior is selected declaratively rather than hardcoded, and the architecture is split between reusable Python modules and experiment-specific YAML composition.

4. **Data ingestion boundary**
   `src/data/` contains the dataset and Lightning datamodule layer. `src/data/components/dataset.py` likely defines dataset objects, while `src/data/protein_datamodule.py` bridges raw protein data into Lightning-ready loaders. The config files `configs/data/protein.yaml` and `configs/data/sampling.yaml` suggest separate paths for training data versus sampling/inference data.

5. **Sequence preprocessing**
   `src/read_seqs.py` is a distinct preprocessing stage for input sequences. The README says it extracts sequence embeddings before inference, and the repo includes `src/utils/esm_extract.py`, indicating ESM is used as an external sequence encoder. This is an important boundary: the model does not consume raw FASTA directly at inference time; it consumes derived sequence features.

6. **Core generative model**
   `src/models/diffusion_module.py` is the main LightningModule for training and inference. It likely wraps the denoising network, loss computation, sampling logic, and optimizer configuration. The model stack is organized so that training orchestration lives in the diffusion module, while the neural architecture itself is split into subpackages.

7. **Network and geometric modeling**
   `src/models/net/` contains the denoising architecture, including `denoising_ipa.py`, `ipa.py`, and `layers.py`. This points to an invariant-point-attention style protein model operating over geometric representations. The model is structured to predict or refine 3D conformations rather than simple sequence labels.

8. **Score/geometry subsystems**
   `src/models/score/` holds frame, SO(3), and R3 scoring utilities. This indicates the diffusion process is geometry-aware and explicitly models rigid-body/rotation components. That separation matters architecturally: the network predicts denoising updates in protein coordinate space, and the score code handles the mathematical representation of rotations and translations.

9. **Protein geometry and common utilities**
   `src/common/` is the shared protein-geometry foundation: atom-level utilities, rigid transforms, residue constants, PDB handling, rotation helpers, and coordinate/data transforms. These modules are infrastructure for both training and inference, and they define the repository’s internal protein representation boundary.

10. **Losses and training objectives**
   `src/models/loss.py` centralizes the optimization objective. In a diffusion-based protein generator, this typically combines geometric denoising terms and structural supervision. Keeping losses separate from the model module suggests the repo is designed for experimentation with alternative objectives.

11. **Training, evaluation, and runtime orchestration**
   `src/train.py` and `src/eval.py` are the executable orchestration layers. They likely instantiate configs, datamodules, models, loggers, callbacks, and checkpoint loading. The structure follows the Lightning-Hydra template pattern, so training/eval are thin wrappers around reusable components rather than monolithic scripts.

12. **Logging, checkpoints, and debugging**
   `src/utils/` contains cross-cutting runtime support: checkpoint utilities, logging helpers, rich terminal output, tensor helpers, multiprocessing helpers, plotting, and instantiation helpers. The `configs/logger/` and `configs/callbacks/` folders show support for multiple experiment trackers and training callbacks such as checkpointing, early stopping, summaries, and progress bars.

13. **External dependencies and environment**
   The repo depends materially on PyTorch, Lightning, Hydra, and `fair-esm`. It also uses an `.env`-based path setup initialized by `initialize.py`, which means dataset and artifact locations are treated as environment-specific infrastructure rather than hardcoded paths. This is important for diagramming deployment boundaries.

14. **Artifact and output boundaries**
   The repository includes example input data in `data/example.fasta` and expects pretrained checkpoints from external storage. Outputs are likely generated conformational ensembles plus logs/metrics/plots depending on the selected logger and callbacks. So the system boundary is sequence-in plus checkpoint-in, ensemble-out, with optional experiment tracking side effects.

15. **What is not in scope**
   There is no evidence of a web server, API service, or interactive UI in this repository. The architecture is a research/ML pipeline centered on offline training and inference, with the main components being preprocessing, geometric diffusion modeling, and experiment management.

Main entry points: [`src/models/diffusion_module.py`](https://github.com/junjie-zhu/idpfold/blob/main/src/models/diffusion_module.py) (Diffusion module), [`src/eval.py`](https://github.com/junjie-zhu/idpfold/blob/main/src/eval.py) (Eval), [`src/models/net/denoising_ipa.py`](https://github.com/junjie-zhu/idpfold/blob/main/src/models/net/denoising_ipa.py) (Denoising IPA), [`src/train.py`](https://github.com/junjie-zhu/idpfold/blob/main/src/train.py) (Train)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control["Control plane"]
  node_train["Train<br/>entrypoint<br/>[train.py]"]
  node_eval["Eval<br/>entrypoint<br/>[eval.py]"]
  node_configs["Configs<br/>hydra config"]
end

subgraph group_data["Data flow"]
  node_read_seqs["Read seqs<br/>preprocess<br/>[read_seqs.py]"]
  node_esm_extract["ESM extract<br/>embedding util<br/>[esm_extract.py]"]
  node_dataset[("Dataset<br/>data source<br/>[dataset.py]")]
  node_datamodule["Protein DM<br/>lightning datamodule"]
end

subgraph group_model["Model stack"]
  node_diffusion_module["Diffusion module<br/>lightning module"]
  node_loss["Losses<br/>objective<br/>[loss.py]"]
  node_denoising_ipa["Denoising IPA<br/>network<br/>[denoising_ipa.py]"]
  node_ipa{{"IPA<br/>attention core<br/>[ipa.py]"}}
  node_layers["Layers<br/>[layers.py]"]
  node_frame{{"Frame score<br/>geometry score<br/>[frame.py]"}}
  node_r3{{"R3 score<br/>geometry score<br/>[r3.py]"}}
  node_so3{{"SO3 score<br/>geometry score<br/>[so3.py]"}}
end

subgraph group_infra["Shared infra"]
  node_protein_common["Protein core<br/>geometry foundation<br/>[protein.py]"]
  node_rigid_utils["Rigid utils<br/>transform math<br/>[rigid_utils.py]"]
  node_data_transforms["Transforms<br/>preprocessing<br/>[data_transforms.py]"]
  node_runtime_utils["Runtime utils<br/>support"]
end

node_configs -->|"configures"| node_train
node_configs -->|"configures"| node_eval
node_read_seqs -->|"uses"| node_esm_extract
node_read_seqs -->|"feeds"| node_eval
node_esm_extract -->|"features"| node_datamodule
node_dataset -->|"loads"| node_datamodule
node_datamodule -->|"provides data"| node_train
node_datamodule -->|"provides data"| node_eval
node_train -->|"instantiates"| node_diffusion_module
node_eval -->|"loads"| node_diffusion_module
node_diffusion_module -->|"wraps"| node_denoising_ipa
node_diffusion_module -->|"optimizes"| node_loss
node_denoising_ipa -->|"builds on"| node_ipa
node_denoising_ipa -->|"uses"| node_layers
node_diffusion_module -->|"scores"| node_frame
node_diffusion_module -->|"scores"| node_r3
node_diffusion_module -->|"scores"| node_so3
node_denoising_ipa -->|"operates on"| node_protein_common
node_protein_common -->|"relies on"| node_rigid_utils
node_protein_common -->|"relies on"| node_data_transforms
node_train -->|"uses"| node_runtime_utils
node_eval -->|"uses"| node_runtime_utils

click node_train "https://github.com/junjie-zhu/idpfold/blob/main/src/train.py"
click node_eval "https://github.com/junjie-zhu/idpfold/blob/main/src/eval.py"
click node_read_seqs "https://github.com/junjie-zhu/idpfold/blob/main/src/read_seqs.py"
click node_esm_extract "https://github.com/junjie-zhu/idpfold/blob/main/src/utils/esm_extract.py"
click node_dataset "https://github.com/junjie-zhu/idpfold/blob/main/src/data/components/dataset.py"
click node_datamodule "https://github.com/junjie-zhu/idpfold/blob/main/src/data/protein_datamodule.py"
click node_diffusion_module "https://github.com/junjie-zhu/idpfold/blob/main/src/models/diffusion_module.py"
click node_loss "https://github.com/junjie-zhu/idpfold/blob/main/src/models/loss.py"
click node_denoising_ipa "https://github.com/junjie-zhu/idpfold/blob/main/src/models/net/denoising_ipa.py"
click node_ipa "https://github.com/junjie-zhu/idpfold/blob/main/src/models/net/ipa.py"
click node_layers "https://github.com/junjie-zhu/idpfold/blob/main/src/models/net/layers.py"
click node_frame "https://github.com/junjie-zhu/idpfold/blob/main/src/models/score/frame.py"
click node_r3 "https://github.com/junjie-zhu/idpfold/blob/main/src/models/score/r3.py"
click node_so3 "https://github.com/junjie-zhu/idpfold/blob/main/src/models/score/so3.py"
click node_protein_common "https://github.com/junjie-zhu/idpfold/blob/main/src/common/protein.py"
click node_rigid_utils "https://github.com/junjie-zhu/idpfold/blob/main/src/common/rigid_utils.py"
click node_data_transforms "https://github.com/junjie-zhu/idpfold/blob/main/src/common/data_transforms.py"
click node_configs "https://github.com/junjie-zhu/idpfold/tree/main/configs"
click node_runtime_utils "https://github.com/junjie-zhu/idpfold/tree/main/src/utils"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train,node_eval,node_configs toneBlue
class node_read_seqs,node_esm_extract,node_dataset,node_datamodule toneAmber
class node_diffusion_module,node_loss,node_denoising_ipa,node_ipa,node_layers,node_frame,node_r3,node_so3 toneMint
class node_protein_common,node_rigid_utils,node_data_transforms,node_runtime_utils toneRose
```

## Components

### Control plane

Hydra configs and runtime entrypoints

- [Train](https://github.com/junjie-zhu/idpfold/blob/main/src/train.py): entrypoint · PyTorch Lightning training entrypoint · `src/train.py`
- [Eval](https://github.com/junjie-zhu/idpfold/blob/main/src/eval.py): entrypoint · Ensemble generation and evaluation entrypoint · `src/eval.py`
- [Configs](https://github.com/junjie-zhu/idpfold/tree/main/configs): hydra config · Hydra composition for data, model, trainer, logging, callbacks, and paths · `configs`

### Data flow

Sequence input, datasets, and dataloading

- [Read seqs](https://github.com/junjie-zhu/idpfold/blob/main/src/read_seqs.py): preprocess · Sequence preprocessing and feature preparation · `src/read_seqs.py`
- [ESM extract](https://github.com/junjie-zhu/idpfold/blob/main/src/utils/esm_extract.py): embedding util · Sequence embedding extraction using ESM · `src/utils/esm_extract.py`
- [Dataset](https://github.com/junjie-zhu/idpfold/blob/main/src/data/components/dataset.py): data source · Protein dataset objects and sample assembly · `src/data/components/dataset.py`
- [Protein DM](https://github.com/junjie-zhu/idpfold/blob/main/src/data/protein_datamodule.py): lightning datamodule · Lightning data module bridging raw protein data to loaders · `src/data/protein_datamodule.py`

### Model stack

Diffusion model, network, and geometric score components

- [Diffusion module](https://github.com/junjie-zhu/idpfold/blob/main/src/models/diffusion_module.py): lightning module · Training, inference, loss, and sampling orchestration · `src/models/diffusion_module.py`
- [Losses](https://github.com/junjie-zhu/idpfold/blob/main/src/models/loss.py): objective · Diffusion and structural training objectives · `src/models/loss.py`
- [Denoising IPA](https://github.com/junjie-zhu/idpfold/blob/main/src/models/net/denoising_ipa.py): network · Main denoising architecture for protein structure refinement · `src/models/net/denoising_ipa.py`
- [IPA](https://github.com/junjie-zhu/idpfold/blob/main/src/models/net/ipa.py): attention core · Invariant-point-attention building blocks · `src/models/net/ipa.py`
- [Layers](https://github.com/junjie-zhu/idpfold/blob/main/src/models/net/layers.py): nn layers · Shared neural network layers and blocks · `src/models/net/layers.py`
- [Frame score](https://github.com/junjie-zhu/idpfold/blob/main/src/models/score/frame.py): geometry score · Rigid-frame scoring utilities · `src/models/score/frame.py`
- [R3 score](https://github.com/junjie-zhu/idpfold/blob/main/src/models/score/r3.py): geometry score · Translation-space scoring utilities · `src/models/score/r3.py`
- [SO3 score](https://github.com/junjie-zhu/idpfold/blob/main/src/models/score/so3.py): geometry score · Rotation-space scoring utilities · `src/models/score/so3.py`

### Shared infra

Protein geometry, utilities, and runtime support

- [Protein core](https://github.com/junjie-zhu/idpfold/blob/main/src/common/protein.py): geometry foundation · Atom, residue, rigid-body, and coordinate utilities · `src/common/protein.py`
- [Rigid utils](https://github.com/junjie-zhu/idpfold/blob/main/src/common/rigid_utils.py): transform math · Rigid transforms and rotation helpers · `src/common/rigid_utils.py`
- [Transforms](https://github.com/junjie-zhu/idpfold/blob/main/src/common/data_transforms.py): preprocessing · Coordinate and data transformation helpers · `src/common/data_transforms.py`
- [Runtime utils](https://github.com/junjie-zhu/idpfold/tree/main/src/utils): support · Logging, checkpoints, plotting, instantiation, and multiprocessing helpers · `src/utils`

## Connections

- Configs → Train: configures · Hydra composes training runtime settings
- Configs → Eval: configures · Hydra composes evaluation runtime settings
- Read seqs → ESM extract: uses · Preprocessing derives embeddings from sequences
- Read seqs → Eval: feeds · Sequence preprocessing prepares inference inputs
- ESM extract → Protein DM: features · Embedded sequence features enter the data pipeline
- Dataset → Protein DM: loads · Datasets are wrapped by the Lightning data module
- Protein DM → Train: provides data · Training consumes batches from the data module
- Protein DM → Eval: provides data · Evaluation and sampling consume prepared inputs
- Train → Diffusion module: instantiates · Training creates the Lightning diffusion module
- Eval → Diffusion module: loads · Evaluation restores the model for inference
- Diffusion module → Denoising IPA: wraps · The Lightning module delegates to the denoising network
- Diffusion module → Losses: optimizes · Training objective is computed from diffusion losses
- Denoising IPA → IPA: builds on · The denoiser is centered on IPA blocks
- Denoising IPA → Layers: uses · Network internals reuse shared layers
- Diffusion module → Frame score: scores · Geometric diffusion uses frame-space scoring
- Diffusion module → R3 score: scores · Geometric diffusion uses translation scoring
- Diffusion module → SO3 score: scores · Geometric diffusion uses rotation scoring
- Denoising IPA → Protein core: operates on · The model consumes protein geometry primitives
- Protein core → Rigid utils: relies on · Core geometry uses rigid transform math
- Protein core → Transforms: relies on · Protein representations are transformed for modeling
- Train → Runtime utils: uses · Training uses logging, checkpoints, and instantiation helpers
- Eval → Runtime utils: uses · Evaluation uses logging and output utilities
