# recursionpharma/mole_public architecture

> Architecture diagram of the GitHub repository recursionpharma/mole_public, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/recursionpharma/mole_public
- Repository: https://github.com/recursionpharma/mole_public

## Overview

MolE is a Python package centered on two user-facing workflows: training molecular property models and running inference/embedding generation from trained checkpoints. The repository is organized as a small public API plus a training stack, with packaged data/config assets to support local experimentation.

#### High-level architecture

The codebase breaks into these main layers:

1. **Command-line entrypoints**
   - `mole/cli/mole_train.py`
   - `mole/cli/mole_predict.py`

   These are the operational front door for the project. They expose the training and inference workflows described in the README.

2. **Training subsystem**
   - `mole/training/`
   - Hydra-based configuration under `mole/training/configs/`

   This is the largest part of the repository. It contains:
   - data loading and preprocessing
   - model definitions
   - neural network building blocks
   - training utilities such as metrics and schedulers

3. **Packaged datasets and vocabularies**
   - `data/*.parquet`
   - `mole/training/data/vocabularies/*.pkl`

   These are example benchmark datasets and a serialized vocabulary artifact used by the molecular representation pipeline.

4. **Packaging and environment management**
   - `setup.py`, `setup.cfg`
   - `requirements/`
   - `Dockerfile`
   - `tox.ini`

   These support installation, reproducibility, and testing across Python versions and CPU/GPU environments.

#### Main runtime flow

##### Training flow
The training path is:

- `mole_train` CLI
- Hydra config resolution
- dataset/data module setup
- model instantiation
- training loop with metrics and scheduler support
- checkpointing of learned weights

The README shows that training is intended to support both:
- **fine-tuning** on smaller labeled datasets
- starting from a **pretrained checkpoint** or from scratch

The `mole/training/configs/model/finetune.yaml` and related config files indicate that model selection and hyperparameters are driven by configuration rather than hardcoded CLI flags.

##### Inference / prediction flow
The prediction path is:

- `mole_predict` CLI or `from mole import mole_predict`
- load trained checkpoint
- normalize input SMILES strings
- run the molecular encoder/model
- return either:
  - task predictions
  - embeddings

The README shows both property prediction and embedding extraction use the same underlying model family, with embeddings exposed as a reusable representation for downstream tasks.

#### Core subsystems

##### 1) CLI boundary
The `mole/cli` package is the thin operational layer around the rest of the system.

- `mole_train.py` likely wires command-line arguments into Hydra and the training stack.
- `mole_predict.py` exposes both command-line prediction and the importable helper used in notebooks.

This boundary is important because it separates user interaction from model/data internals.

##### 2) Configuration system
The repository uses **Hydra** for experiment configuration.

Relevant files:
- `mole/training/configs/base_config.yaml`
- `mole/training/configs/model/finetune.yaml`
- `mole/training/configs/deberta_config/base.yaml`

This suggests a layered configuration approach:
- a base training config
- model-specific overrides
- possibly architecture-specific settings for a DeBERTa-like transformer variant

For architecture diagrams, this is the main control plane: it determines dataset paths, task type, learning rate, dropout, checkpoint initialization, and other run parameters.

##### 3) Data subsystem
Files under `mole/training/data/` define the data boundary:

- `datasets.py` — dataset objects
- `data_modules.py` — likely PyTorch Lightning-style orchestration of train/validation/test loaders
- `utils.py` — preprocessing helpers
- `vocabularies/` — token/fragment/environment vocabulary artifact

The packaged parquet files under `data/` are example splits for two benchmark tasks:
- `TDC_Half_Life_Obach_*`
- `TDC_HIA_Hou_*`

These indicate the repository is designed around labeled molecular property prediction from tabular datasets containing at least:
- `smiles`
- one or more target columns

The presence of a vocabulary pickle strongly suggests the input pipeline converts molecules into a discrete representation before feeding the model. That vocabulary is part of the model’s data boundary and should be shown separately from raw SMILES inputs.

##### 4) Model subsystem
The model code is split across several layers:

- `mole/training/models/base.py`
- `mole/training/models/encoder.py`
- `mole/training/models/mole.py`

This indicates a separation between:
- a **base model abstraction**
- an **encoder** that produces molecular representations
- the **task-facing MolE model** that attaches prediction heads or exposes embeddings

Given the README description, the model combines:
- **geometric deep learning**
- **transformer-style attention**

The low-level neural network pieces live in:
- `mole/training/nn/bert.py`
- `mole/training/nn/disentangled_attention.py`

This suggests the core representation engine is transformer-like, with a BERT/DeBERTa-inspired attention mechanism and a specialized attention variant for molecular structure. The `deberta_config` directory reinforces that the architecture is related to disentangled attention ideas rather than a plain vanilla transformer.

For diagramming, treat this as the central compute block:
raw molecular representation in -> encoder/attention stack -> latent embedding -> task head or embedding output

##### 5) Training utilities
Supporting components live under `mole/training/utils/`:

- `metrics.py` — evaluation metrics for regression/classification
- `schedulers.py` — learning-rate scheduling, including the warmup behavior mentioned in the README

These are not standalone services, but they materially affect training behavior and should be shown as part of the training loop rather than the model itself.

##### 6) Masking and representation helpers
`mole/training/models/utils/masks.py` indicates the model operates on masked structured inputs, which is typical for sequence/graph hybrid models. This is likely used to control attention over valid molecular tokens or graph neighborhoods.

#### Data and artifact boundaries

There are three important artifact types in the repository:

1. **Example labeled datasets**
   - stored as parquet in `data/`
   - used for fine-tuning and evaluation examples

2. **Vocabulary artifact**
   - `mole/training/data/vocabularies/vocabulary_207atomenvs_radius0_ZINC_guacamole.pkl`
   - likely a serialized mapping used to convert molecules into model-ready discrete units

3. **Model checkpoints**
   - not stored in the repo, but required for prediction and fine-tuning
   - loaded via `checkpoint_path` / `pretrained_model` parameters in the CLI and API

This means the model has an explicit separation between:
- repository-provided assets
- user-provided training data
- externally stored pretrained weights

#### External technology that matters architecturally

A few technologies materially shape the repo structure:

- **PyTorch**: the model and training stack are built around neural network modules.
- **Hydra**: experiment and CLI configuration management.
- **Packaging via pip/setuptools**: the project is intended to be installed as a library and used from the command line.
- **Docker**: included for reproducible runtime environments.
- **Python 3.9/3.10** and separate CPU/GPU dependency sets: the repo supports both local CPU usage and CUDA-enabled training/inference.

#### How the major pieces interact

A useful architectural diagram would show these boundaries and flows:

- **User / notebook / CLI**
  -> `mole_train` or `mole_predict`
- **Hydra config**
  -> selects model variant, hyperparameters, data paths, checkpoint behavior
- **Data module / dataset layer**
  -> reads parquet/CSV-like molecular tables
  -> uses vocabulary/preprocessing helpers
  -> emits model-ready batches
- **Model layer**
  -> encoder + attention blocks
  -> produces embeddings
  -> prediction head for regression/classification or embedding output
- **Training utilities**
  -> metrics, schedulers, warmup, validation logic
- **Artifacts**
  -> checkpoints saved/loaded externally
  -> example datasets and vocabulary bundled in repo

#### Repository character

This is not a generic web service or application server. It is a **research/modeling library** packaged for:
- training molecular foundation/fine-tuned models
- running checkpoint-based inference
- extracting molecular embeddings for downstream use

The architecture is centered on the model/data boundary and on configuration-driven experimentation rather than on request/response APIs or persistent backend services.

Main entry points: [`mole/training/models/mole.py`](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/models/mole.py) (MolE model), [`mole/cli/mole_train.py`](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/cli/mole_train.py) (Train CLI), [`mole/training/models/encoder.py`](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/models/encoder.py) (Encoder), [`mole/training/data/data_modules.py`](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/data/data_modules.py) (Data module)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph "CLI"
  mole_train["Train CLI<br/>entrypoint<br/>[mole_train.py]"]
  mole_predict["Predict CLI<br/>entrypoint<br/>[mole_predict.py]"]
end

subgraph "Training"
  base_config["Base config<br/>hydra config<br/>[base_config.yaml]"]
  model_config["Finetune config<br/>hydra config<br/>[finetune.yaml]"]
  deberta_config["Attention config<br/>hydra config<br/>[base.yaml]"]
  data_module["Data module<br/>batch orchestration<br/>[data_modules.py]"]
  datasets["Datasets<br/>dataset layer<br/>[datasets.py]"]
  data_utils["Preprocess<br/>data utils<br/>[utils.py]"]
  mole_model["MolE model<br/>task model<br/>[mole.py]"]
  encoder["Encoder<br/>representation model<br/>[encoder.py]"]
  base_model["Base model<br/>model base<br/>[base.py]"]
  masks["Masks<br/>masking helpers<br/>[masks.py]"]
  bert_nn["BERT blocks<br/>attention blocks<br/>[bert.py]"]
  da_nn["Disentangled attn<br/>attention mechanism"]
  metrics["Metrics<br/>evaluation<br/>[metrics.py]"]
  schedulers["Schedulers<br/>optimization<br/>[schedulers.py]"]
end

subgraph "Assets"
  vocabulary[("Vocabulary<br/>serialized artifact")]
  parquet_data[("Bench data<br/>packaged datasets")]
end

subgraph "Packaging"
  setup["Setup<br/>packaging<br/>[setup.py]"]
  requirements["Deps<br/>environment<br/>[main.in]"]
  docker["Docker<br/>container"]
end

mole_train -->|"loads"| base_config
mole_train -->|"selects"| model_config
mole_train -->|"starts"| data_module
mole_train -->|"trains"| mole_model
mole_predict -->|"loads"| mole_model
mole_predict -->|"normalizes"| data_utils
base_config -->|"includes"| deberta_config
model_config -->|"configures"| mole_model
data_module -->|"wraps"| datasets
datasets -->|"uses"| data_utils
data_utils -->|"maps to"| vocabulary
data_module -->|"reads"| parquet_data
mole_model -->|"extends"| base_model
mole_model -->|"encodes"| encoder
encoder -->|"uses"| bert_nn
encoder -->|"uses"| da_nn
encoder -->|"applies"| masks
mole_model -->|"evaluates"| metrics
mole_model -->|"optimizes"| schedulers
setup -->|"declares"| requirements
setup -->|"supports"| docker

click mole_train "https://github.com/recursionpharma/mole_public/blob/trunk/mole/cli/mole_train.py"
click mole_predict "https://github.com/recursionpharma/mole_public/blob/trunk/mole/cli/mole_predict.py"
click base_config "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/configs/base_config.yaml"
click model_config "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/configs/model/finetune.yaml"
click deberta_config "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/configs/deberta_config/base.yaml"
click data_module "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/data/data_modules.py"
click datasets "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/data/datasets.py"
click data_utils "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/data/utils.py"
click vocabulary "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/data/vocabularies/vocabulary_207atomenvs_radius0_ZINC_guacamole.pkl"
click mole_model "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/models/mole.py"
click encoder "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/models/encoder.py"
click base_model "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/models/base.py"
click masks "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/models/utils/masks.py"
click bert_nn "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/nn/bert.py"
click da_nn "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/nn/disentangled_attention.py"
click metrics "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/utils/metrics.py"
click schedulers "https://github.com/recursionpharma/mole_public/blob/trunk/mole/training/utils/schedulers.py"
click parquet_data "https://github.com/recursionpharma/mole_public/blob/trunk/data/TDC_HIA_Hou_train_seed0.parquet"
click setup "https://github.com/recursionpharma/mole_public/blob/trunk/setup.py"
click requirements "https://github.com/recursionpharma/mole_public/blob/trunk/requirements/main.in"
click docker "https://github.com/recursionpharma/mole_public/tree/trunk/Dockerfile"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class mole_train,mole_predict toneBlue
class base_config,model_config,deberta_config,data_module,datasets,data_utils,mole_model,encoder,base_model,masks,bert_nn,da_nn,metrics,schedulers toneAmber
class vocabulary,parquet_data toneMint
class setup,requirements,docker toneRose
```

## Components

### CLI

Command-line entrypoints for training and prediction.

- [Train CLI](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/cli/mole_train.py): entrypoint · Hydra-driven training entrypoint. · `mole/cli/mole_train.py`
- [Predict CLI](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/cli/mole_predict.py): entrypoint · Inference and embedding entrypoint. · `mole/cli/mole_predict.py`

### Training

Core model, data, and optimization stack.

- [Base config](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/configs/base_config.yaml): hydra config · Top-level training configuration. · `mole/training/configs/base_config.yaml`
- [Finetune config](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/configs/model/finetune.yaml): hydra config · Model and fine-tuning overrides. · `mole/training/configs/model/finetune.yaml`
- [Attention config](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/configs/deberta_config/base.yaml): hydra config · Architecture settings for disentangled attention. · `mole/training/configs/deberta_config/base.yaml`
- [Data module](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/data/data_modules.py): batch orchestration · Builds loaders and dataset splits. · `mole/training/data/data_modules.py`
- [Datasets](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/data/datasets.py): dataset layer · Dataset objects and split handling. · `mole/training/data/datasets.py`
- [Preprocess](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/data/utils.py): data utils · Input normalization and molecule preprocessing. · `mole/training/data/utils.py`
- [MolE model](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/models/mole.py): task model · Task-facing model for prediction and embeddings. · `mole/training/models/mole.py`
- [Encoder](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/models/encoder.py): representation model · Produces molecular latent representations. · `mole/training/models/encoder.py`
- [Base model](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/models/base.py): model base · Shared model abstraction and checkpoint behavior. · `mole/training/models/base.py`
- [Masks](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/models/utils/masks.py): masking helpers · Attention and structure masking utilities. · `mole/training/models/utils/masks.py`
- [BERT blocks](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/nn/bert.py): attention blocks · Transformer-style neural building blocks. · `mole/training/nn/bert.py`
- [Disentangled attn](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/nn/disentangled_attention.py): attention mechanism · Specialized molecular attention mechanism. · `mole/training/nn/disentangled_attention.py`
- [Metrics](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/utils/metrics.py): evaluation · Training and validation metrics. · `mole/training/utils/metrics.py`
- [Schedulers](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/utils/schedulers.py): optimization · Learning-rate scheduling and warmup. · `mole/training/utils/schedulers.py`

### Assets

Packaged datasets and vocabulary artifacts.

- [Vocabulary](https://github.com/recursionpharma/mole_public/blob/HEAD/mole/training/data/vocabularies/vocabulary_207atomenvs_radius0_ZINC_guacamole.pkl): serialized artifact · Serialized molecular vocabulary used by the input pipeline. · `mole/training/data/vocabularies/vocabulary_207atomenvs_radius0_ZINC_guacamole.pkl`
- [Bench data](https://github.com/recursionpharma/mole_public/blob/HEAD/data/TDC_HIA_Hou_train_seed0.parquet): packaged datasets · Example labeled molecular datasets in parquet format. · `data/TDC_HIA_Hou_train_seed0.parquet`

### Packaging

Installation and environment support files.

- [Setup](https://github.com/recursionpharma/mole_public/blob/HEAD/setup.py): packaging · Package installation metadata. · `setup.py`
- [Deps](https://github.com/recursionpharma/mole_public/blob/HEAD/requirements/main.in): environment · Pinned dependency sets for different Python/runtime targets. · `requirements/main.in`
- [Docker](https://github.com/recursionpharma/mole_public/tree/HEAD/Dockerfile): container · Reproducible runtime environment. · `Dockerfile`

## Connections

- Train CLI → Base config: loads · Training entrypoint resolves experiment config.
- Train CLI → Finetune config: selects · Chooses model and finetuning settings.
- Train CLI → Data module: starts · Builds the data pipeline for training.
- Train CLI → MolE model: trains · Instantiates and optimizes the model.
- Predict CLI → MolE model: loads · Inference uses a trained checkpoint.
- Predict CLI → Preprocess: normalizes · Prepares input SMILES or molecular strings.
- Base config → Attention config: includes · Base config can delegate architecture details.
- Finetune config → MolE model: configures · Selects model variant and hyperparameters.
- Data module → Datasets: wraps · Orchestrates dataset objects and splits.
- Datasets → Preprocess: uses · Applies preprocessing and tokenization steps.
- Preprocess → Vocabulary: maps to · Converts molecules into discrete vocabulary units.
- Data module → Bench data: reads · Consumes packaged benchmark splits for local runs.
- MolE model → Base model: extends · Task model builds on shared base abstractions.
- MolE model → Encoder: encodes · Produces molecular embeddings through the encoder stack.
- Encoder → BERT blocks: uses · Encoder relies on transformer-style blocks.
- Encoder → Disentangled attn: uses · Encoder uses disentangled attention components.
- Encoder → Masks: applies · Attention over valid molecular structure is masked.
- MolE model → Metrics: evaluates · Training and validation metrics are computed on outputs.
- MolE model → Schedulers: optimizes · Learning-rate scheduling supports training runs.
- Setup → Deps: declares · Packaging depends on pinned environment sets.
- Setup → Docker: supports · Container build mirrors the package environment.
