How duyhominhnguyen/conan-fgw works
Project Type and Purpose
• This is a research-focused machine-learning library implementing “Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networks” (ConAN-FGW).
• It is not a web or microservice application but a data-driven ML pipeline built on PyTorch Lightning and PyTorch Geometric, orchestrated by Bash scripts and YAML configs.
High-Level Architectural Components
a. Configuration Layer
• YAML files under conan_fgw/config/<model>/<task>/<dataset>/…
• environment.yml, pyproject.toml, setup.cfg
b. Orchestration Layer (CLI & Scripts)
• conan_fgw/script/run.sh and variants
• Makefile (build/test targets)
c. Parser & Experiment Entry Point
• conan_fgw/src/config_parser.py
• conan_fgw/src/train_val.py (accepts CLI args & config path)
• conan_fgw/src/experiments.py (defines SOTAExperiment, SOTAExperimentBaryCenter)
d. Data Layer
• Raw data location: /data (user-provided)
• Data pipeline modules in conan_fgw/src/data/:
– datasets.py (dataset abstractions)
– datamodules.py (Lightning DataModule wrappers)
– conformers/generators.py & features.py (conformer generation & graph‐feature extraction)
– splitters.py (train/val/test splits)
e. Model Layer
• Core neural modules in conan_fgw/src/model/:
– Graph-embedding backbones:
· schnet_no_sum.py (SchNet extension)
· visnet.py (ViSNet extension)
– Aggregation & FGW algorithms:
· model/fgw/ (Barycenter, Bregman, Sinkhorn, utils)
– Attention & baseline components: attention_layer.py, baseline_models.py
– Common utilities: common.py, utils.py, features.py
f. Training & Validation
• conan_fgw/src/train_val.py dispatches to PyTorch Lightning Trainer using Experiment classes
• Distributed DataParallel for stage “conan_fgw”
• Logging via local outputs/ and optional Weights & Biases
g. Artifacts & Reports
• Model checkpoints: models/
• Logs (tensorboard/Lightning logs): outputs/
• Figures and reports: figs/, reports/
• Notebooks for demo: notebooks/fgw.ipynb
Interactions & Data Flow
1. User invokes run.sh → sets environment variables → calls train_val.py.
2. train_val.py loads YAML config via config_parser.py.
3. config_parser → instantiates a Lightning DataModule (datamodules.py) → raw .pt or dataset files under /data.
4. DataModule pipelines conformer generation → feature extraction → batching.
5. train_val.py also instantiates the Experiment class (SOTAExperiment or SOTAExperimentBaryCenter)
6. Experiment builds the model: selects backbone (SchNetNoSum or ViSNet), wraps FGW aggregator, attention, etc.
7. Lightning Trainer executes training loops: forward passes→loss computation→backprop.
8. Checkpoints written to models/…, logs to outputs/…, optional W&B calls.
9. Post-training artifacts produce figures under reports/ and figs/ for analysis.
Important Architectural Patterns
• Configuration-Driven: All hyperparams and experiment settings defined in YAML and loaded at runtime.
• Modular ML Pipeline: Clear separation between DataModule, Model, Training loop (Experiment + Trainer).
• Pluggable Backbones & Aggregators: The Experiment class swaps between SchNet/ViSNet and different FGW stages (pretrain vs. barycenter).
• PyTorch Lightning Pattern: Standard Trainer–LightningModule–DataModule triad.
Elements
• Clear Component Labels (e.g., “SchNetNoSum” vs. “FGW Barycenter Module”)
• Blue = Configuration & Orchestration
• Green = Data Layer
• Orange = Model Layer
• Purple = Training & Logging
• Gray = Artifacts & Reports