How recursionpharma/mole_public works
MolE is a Python package centered on two user-facing workflows: training molecular property models and running inference/embedding generation from trained checkpoints. The repository is organized as a small public API plus a training stack, with packaged data/config assets to support local experimentation.
High-level architecture
The codebase breaks into these main layers:
1. Command-line entrypoints
- mole/cli/mole_train.py
- mole/cli/mole_predict.py
These are the operational front door for the project. They expose the training and inference workflows described in the README.
2. Training subsystem
- mole/training/
- Hydra-based configuration under mole/training/configs/
This is the largest part of the repository. It contains:
- data loading and preprocessing
- model definitions
- neural network building blocks
- training utilities such as metrics and schedulers
3. Packaged datasets and vocabularies
- data/*.parquet
- mole/training/data/vocabularies/*.pkl
These are example benchmark datasets and a serialized vocabulary artifact used by the molecular representation pipeline.
4. Packaging and environment management
- setup.py, setup.cfg
- requirements/
- Dockerfile
- tox.ini
These support installation, reproducibility, and testing across Python versions and CPU/GPU environments.
Main runtime flow
Training flow
The training path is:
• mole_train CLI
• Hydra config resolution
• dataset/data module setup
• model instantiation
• training loop with metrics and scheduler support
• checkpointing of learned weights
The README shows that training is intended to support both:
• fine-tuning on smaller labeled datasets
• starting from a pretrained checkpoint or from scratch
The mole/training/configs/model/finetune.yaml and related config files indicate that model selection and hyperparameters are driven by configuration rather than hardcoded CLI flags.
Inference / prediction flow
The prediction path is:
• mole_predict CLI or from mole import mole_predict
• load trained checkpoint
• normalize input SMILES strings
• run the molecular encoder/model
• return either:
- task predictions
- embeddings
The README shows both property prediction and embedding extraction use the same underlying model family, with embeddings exposed as a reusable representation for downstream tasks.
Core subsystems
1) CLI boundary
The mole/cli package is the thin operational layer around the rest of the system.
• mole_train.py likely wires command-line arguments into Hydra and the training stack.
• mole_predict.py exposes both command-line prediction and the importable helper used in notebooks.
This boundary is important because it separates user interaction from model/data internals.
2) Configuration system
The repository uses Hydra for experiment configuration.
Relevant files:
• mole/training/configs/base_config.yaml
• mole/training/configs/model/finetune.yaml
• mole/training/configs/deberta_config/base.yaml
This suggests a layered configuration approach:
• a base training config
• model-specific overrides
• possibly architecture-specific settings for a DeBERTa-like transformer variant
For architecture diagrams, this is the main control plane: it determines dataset paths, task type, learning rate, dropout, checkpoint initialization, and other run parameters.
3) Data subsystem
Files under mole/training/data/ define the data boundary:
• datasets.py — dataset objects
• data_modules.py — likely PyTorch Lightning-style orchestration of train/validation/test loaders
• utils.py — preprocessing helpers
• vocabularies/ — token/fragment/environment vocabulary artifact
The packaged parquet files under data/ are example splits for two benchmark tasks:
• TDC_Half_Life_Obach_*
• TDC_HIA_Hou_*
These indicate the repository is designed around labeled molecular property prediction from tabular datasets containing at least:
• smiles
• one or more target columns
The presence of a vocabulary pickle strongly suggests the input pipeline converts molecules into a discrete representation before feeding the model. That vocabulary is part of the model’s data boundary and should be shown separately from raw SMILES inputs.
4) Model subsystem
The model code is split across several layers:
• mole/training/models/base.py
• mole/training/models/encoder.py
• mole/training/models/mole.py
This indicates a separation between:
• a base model abstraction
• an encoder that produces molecular representations
• the task-facing MolE model that attaches prediction heads or exposes embeddings
Given the README description, the model combines:
• geometric deep learning
• transformer-style attention
The low-level neural network pieces live in:
• mole/training/nn/bert.py
• mole/training/nn/disentangled_attention.py
This suggests the core representation engine is transformer-like, with a BERT/DeBERTa-inspired attention mechanism and a specialized attention variant for molecular structure. The deberta_config directory reinforces that the architecture is related to disentangled attention ideas rather than a plain vanilla transformer.
For diagramming, treat this as the central compute block:
raw molecular representation in -> encoder/attention stack -> latent embedding -> task head or embedding output
5) Training utilities
Supporting components live under mole/training/utils/:
• metrics.py — evaluation metrics for regression/classification
• schedulers.py — learning-rate scheduling, including the warmup behavior mentioned in the README
These are not standalone services, but they materially affect training behavior and should be shown as part of the training loop rather than the model itself.
6) Masking and representation helpers
mole/training/models/utils/masks.py indicates the model operates on masked structured inputs, which is typical for sequence/graph hybrid models. This is likely used to control attention over valid molecular tokens or graph neighborhoods.
Data and artifact boundaries
There are three important artifact types in the repository:
1. Example labeled datasets
- stored as parquet in data/
- used for fine-tuning and evaluation examples
2. Vocabulary artifact
- mole/training/data/vocabularies/vocabulary_207atomenvs_radius0_ZINC_guacamole.pkl
- likely a serialized mapping used to convert molecules into model-ready discrete units
3. Model checkpoints
- not stored in the repo, but required for prediction and fine-tuning
- loaded via checkpoint_path / pretrained_model parameters in the CLI and API
This means the model has an explicit separation between:
• repository-provided assets
• user-provided training data
• externally stored pretrained weights
External technology that matters architecturally
A few technologies materially shape the repo structure:
• PyTorch: the model and training stack are built around neural network modules.
• Hydra: experiment and CLI configuration management.
• Packaging via pip/setuptools: the project is intended to be installed as a library and used from the command line.
• Docker: included for reproducible runtime environments.
• Python 3.9/3.10 and separate CPU/GPU dependency sets: the repo supports both local CPU usage and CUDA-enabled training/inference.
How the major pieces interact
A useful architectural diagram would show these boundaries and flows:
• User / notebook / CLI
-> mole_train or mole_predict
• Hydra config
-> selects model variant, hyperparameters, data paths, checkpoint behavior
• Data module / dataset layer
-> reads parquet/CSV-like molecular tables
-> uses vocabulary/preprocessing helpers
-> emits model-ready batches
• Model layer
-> encoder + attention blocks
-> produces embeddings
-> prediction head for regression/classification or embedding output
• Training utilities
-> metrics, schedulers, warmup, validation logic
• Artifacts
-> checkpoints saved/loaded externally
-> example datasets and vocabulary bundled in repo
Repository character
This is not a generic web service or application server. It is a research/modeling library packaged for:
• training molecular foundation/fine-tuned models
• running checkpoint-based inference
• extracting molecular embeddings for downstream use
The architecture is centered on the model/data boundary and on configuration-driven experimentation rather than on request/response APIs or persistent backend services.
Main entry points: mole.py MolE model · mole_train.py Train CLI · encoder.py Encoder · data_modules.py Data module
Components 21
CLI
Command-line entrypoints for training and prediction.
- Train CLI · Hydra-driven training entrypoint.
mole/cli/mole_train.py - Predict CLI · Inference and embedding entrypoint.
mole/cli/mole_predict.py
Training
Core model, data, and optimization stack.
- Base config · Top-level training configuration.
mole/training/configs/base_config.yaml - Finetune config · Model and fine-tuning overrides.
mole/training/configs/model/finetune.yaml - Attention config · Architecture settings for disentangled attention.
mole/training/configs/deberta_config/base.yaml - Data module · Builds loaders and dataset splits.
mole/training/data/data_modules.py - Datasets · Dataset objects and split handling.
mole/training/data/datasets.py - Preprocess · Input normalization and molecule preprocessing.
mole/training/data/utils.py - MolE model · Task-facing model for prediction and embeddings.
mole/training/models/mole.py - Encoder · Produces molecular latent representations.
mole/training/models/encoder.py - Base model · Shared model abstraction and checkpoint behavior.
mole/training/models/base.py - Masks · Attention and structure masking utilities.
mole/training/models/utils/masks.py - BERT blocks · Transformer-style neural building blocks.
mole/training/nn/bert.py - Disentangled attn · Specialized molecular attention mechanism.
mole/training/nn/disentangled_attention.py - Metrics · Training and validation metrics.
mole/training/utils/metrics.py - Schedulers · Learning-rate scheduling and warmup.
mole/training/utils/schedulers.py
Assets
Packaged datasets and vocabulary artifacts.
- Vocabulary · Serialized molecular vocabulary used by the input pipeline.
mole/training/data/vocabularies/vocabulary_207atomenvs_radius0_ZINC_guacamole.pkl - Bench data · Example labeled molecular datasets in parquet format.
data/TDC_HIA_Hou_train_seed0.parquet
Connections 21
- Train CLI → Base config: loads · Training entrypoint resolves experiment config.
- Train CLI → Finetune config: selects · Chooses model and finetuning settings.
- Train CLI → Data module: starts · Builds the data pipeline for training.
- Train CLI → MolE model: trains · Instantiates and optimizes the model.
- Predict CLI → MolE model: loads · Inference uses a trained checkpoint.
- Predict CLI → Preprocess: normalizes · Prepares input SMILES or molecular strings.
- Base config → Attention config: includes · Base config can delegate architecture details.
- Finetune config → MolE model: configures · Selects model variant and hyperparameters.
- Data module → Datasets: wraps · Orchestrates dataset objects and splits.
- Datasets → Preprocess: uses · Applies preprocessing and tokenization steps.
- Preprocess → Vocabulary: maps to · Converts molecules into discrete vocabulary units.
- Data module → Bench data: reads · Consumes packaged benchmark splits for local runs.
- MolE model → Base model: extends · Task model builds on shared base abstractions.
- MolE model → Encoder: encodes · Produces molecular embeddings through the encoder stack.
- Encoder → BERT blocks: uses · Encoder relies on transformer-style blocks.
- Encoder → Disentangled attn: uses · Encoder uses disentangled attention components.
- Encoder → Masks: applies · Attention over valid molecular structure is masked.
- MolE model → Metrics: evaluates · Training and validation metrics are computed on outputs.
- MolE model → Schedulers: optimizes · Learning-rate scheduling supports training runs.
- Setup → Deps: declares · Packaging depends on pinned environment sets.
- Setup → Docker: supports · Container build mirrors the package environment.