How atomicarchitects/equiformer works
Main Components
• Data Sources and Preprocessing:
• Datasets folder (with subdirectories like datasets/pyg) where data for QM9, MD17, and OC20 reside.
• Automatic dataset download and symbolic links (such as for OC20) as part of the data ingestion layer.
• Model Architectures (Nets):
• “nets” directory as a core module, containing various transformer implementations (e.g., dp_attention_transformer, graph_attention_transformer) which include variants for different datasets (QM9, MD17, OC20).
• That many network files include components such as attention, normalization layers, activation functions, and radial functions that contribute to the equivariant nature.
• Training and Evaluation Pipelines:
• For each dataset, include separate entry points:
• QM9: main_qm9.py and its corresponding training scripts in scripts/train/qm9 (and dp_equiformer variants).
• MD17: main_md17.py and main_md17_dens.py for the standard and DeNS-enhanced training, with corresponding directory structures under md17/configs and scripts/train/md17.
• OC20: main_oc20.py and oc20/trainer with its configurations under oc20/configs.
• These components also integrate environment configuration (such as env/env_equiformer.yml) for reproducible setups.
• Configuration and Utility Components:
• Configuration files (YAML files in md17/configs and oc20/configs) that define model hyperparameters, training settings, and dataset specifics.
• Utility modules like engine.py, logger.py, and optim_factory.py that manage training loops, logging, and optimizer configurations.
• Scripts directory is also important as it provides shell scripts that kick off training experiments for different datasets and model variants.
Relationships and Interactions
• Training scripts (from scripts/) and main entry points (main_qm9.py, main_md17.py, main_oc20.py) to the “Model Architectures” defined in nets.
• How configurations (from md17/configs and oc20/configs) are loaded by these training programs.
• That training routines call utility functions (from engine.py, logger.py, etc.) and interact with external libraries like PyTorch, PyG, and e3nn (as noted in the acknowledgment section).
• For OC20 in particular, show additional components such as trainer modules (in oc20/trainer) that handle specialized training routines (like learning rate scheduling and weight decay adjustments).
Architectural Patterns and Design Principles
• The separation of data, model definitions, configurations, and training scripts suggests a modular design that facilitates testing and experimentation.
• Multi-experiment architecture: separate configurations and scripts for different tasks or settings (e.g., standard training vs. DeNS).
• That the project follows a “configuration over hardcoding” approach by using YAML config files to abstract hyperparameter settings.
Relevant Technologies and Frameworks
• PyTorch as the main deep learning framework.
• Dependencies like PyG (PyTorch Geometric), e3nn (for equivariant networks), and timm (PyTorch Image Models) which are referenced in the acknowledgments.
• External dataset specifications (like linking to OC20 data) and the use of environment files (env/env_equiformer.yml).