How microsoft/graphormer works
Main Components
• Core Library (Graphormer):
– This component includes fundamental modules (e.g., criterions, data handling, models, modules/multihead attention, training tasks, pretrain utilities, and evaluation scripts). It is used as the backbone for all subprojects.
– Highlight supporting functionalities that integrate with external frameworks (e.g., fairseq, PyG, DGL, and OGB).
• Subproject Variants (Distributional Graphormer):
– Catalyst-Adsorption: Contains specialized scripts, models, and training procedures specifically tailored for modeling catalyst adsorption phenomena.
– Property-Guided: Another branch that contains bespoke models (such as m3gnet utilities) and task definitions for material property prediction.
– Protein–Ligand: Focused on tasks like protein–ligand interactions with dedicated evaluation, preprocessing (using environment files, sample pdb scripts), and diffusion models.
– Protein: A branch that appears oriented toward general protein modeling, with a separate data provider and a suite of model definitions.
• Auxiliary Components:
– Examples: Scripted examples and custom dataset handlers for demonstrating how to set up experiments.
– Documentation and CI/CD: GitHub workflows, docs (readthedocs, tutorials, guides), and installation scripts (like install.sh) that manage deployment and user onboarding.
Out Relationships and Interactions
• Within the Core Library, delineate subdivisions such as:
– Data Management: Handling datasets (algos, collators, dataset loaders, and wrappers).
– Model Components: The structure of graphormer layers, 3D modules, multihead attention, encoder layers, etc.
– Training and Tasks: Custom training scripts, pre-training processes, and criterions (loss functions).
Architectural Patterns and Design Principles
• Modularity: The repository is clearly structured in modular components—with separate directories for data, models, modules, and tasks—which makes it easier to extend for different molecular modeling tasks.
• Reusability & Extensibility: The core library is designed to be adapted for multiple domain-specific projects.
• Separation of Concerns: Division between core functionalities (model definitions, utility functions) and task-specific logic (specific training scripts, data pipelines for each subproject).
Relevant Technologies
• Deep Learning and Pre-training: The modules and models that implement transformer-based architectures adapted for graph data.
• External Libraries and Frameworks: Support for fairseq, PyG (PyTorch Geometric), DGL (Deep Graph Library), and OGB.
• Scripting and Lifecycle Management: Shell scripts (install.sh, train.sh) for installation and training, plus CI/CD configs from GitHub workflows that ensure consistent builds and tests.
Elements
– Graphormer Core (with submodules: Data, Models, Modules, Tasks, Utils)
– Catalyst-Adsorption extension
– Property-Guided extension
– Protein–Ligand extension
– Protein extension
– External Framework Integrations (fairseq, PyG, DGL, OGB)
– Data flow (from dataset processing, through model layers, to evaluation)
– Dependency flows (from core library components to specialized subprojects)
– Core components vs. task-specific components
– External integrations vs. internal modules
• Optionally, include deployment/CI components (GitHub Actions, installation scripts) on the periphery if relevant.
Final Assembly
– Start with a high-level view with the “Graphormer Core” at the center.
– Link external libraries and examples to the relevant core modules.