# desirebonneau/admet-dataset-cleaner architecture

> Architecture diagram of the GitHub repository desirebonneau/admet-dataset-cleaner, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/desirebonneau/admet-dataset-cleaner
- Repository: https://github.com/desirebonneau/admet-dataset-cleaner

## Overview

Dataset Cleaner prepares molecular ADMET CSVs for downstream analysis: users configure batch inputs, then a pipeline validates and standardizes structures, flags duplicates and chemical hazards, assesses structural outliers, predicts pKa, and transfers chirality before producing cleaned data. The multithreaded runner is documented for large datasets; the batch script currently invokes the single-threaded runner. The graph covers the sampled runtime modules and README-described pKa capability; unsampled runner internals are not expanded into unsupported calls.

Main entry points: [`singlethread_main/main.py`](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/singlethread_main/main.py) (Single-thread pipeline), [`multithread_main/main.py`](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/multithread_main/main.py) (Multithread pipeline), [`utils/pubchem_utils.py`](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/pubchem_utils.py) (PubChem lookup), [`utils/outlier_utils.py`](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/outlier_utils.py) (Outlier detection)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry points"]
  node_batch["Batch runner<br/>[batch_cleaning.py]"]
  node_single["Single-thread pipeline<br/>[main.py]"]
  node_multi["Multithread pipeline<br/>[main.py]"]
end

subgraph group_pipeline["Cleaning pipeline"]
  node_config["Filter patterns<br/>[config.py]"]
  node_pka["pKa model"]
end

subgraph group_chemistry["Chemistry utilities"]
  node_smiles["SMILES handling<br/>[smiles_utils.py]"]
  node_pubchem["PubChem lookup<br/>[pubchem_utils.py]"]
  node_duplicates["Exact deduplication<br/>[duplicate_utils.py]"]
  node_filters["Chemical filters"]
  node_outliers["Outlier detection<br/>[outlier_utils.py]"]
  node_tanimoto["Fingerprints<br/>[tanimoto_utils.py]"]
  node_chirality["Chirality transfer<br/>[chirality_utils.py]"]
  node_io["File utilities<br/>[io_utils.py]"]
end

node_user(("Dataset curator"))
node_service["PubChem service<br/>external service"]

node_user -->|"configures runs"| node_batch
node_batch -->|"calls cleaner"| node_single
node_single -->|"uses"| node_smiles
node_single -->|"uses"| node_pubchem
node_single -->|"uses"| node_duplicates
node_single -->|"uses"| node_filters
node_single -->|"uses"| node_outliers
node_single -->|"uses"| node_chirality
node_single -->|"uses patterns"| node_config
node_single -.->|"predicts with"| node_pka
node_multi -->|"uses"| node_smiles
node_multi -->|"uses"| node_pubchem
node_multi -->|"uses"| node_duplicates
node_multi -->|"uses"| node_filters
node_multi -->|"uses"| node_outliers
node_multi -->|"uses"| node_chirality
node_multi -->|"uses patterns"| node_config
node_multi -.->|"predicts with"| node_pka
node_pubchem -->|"requests data"| node_service
node_outliers -->|"generates fingerprints"| node_tanimoto

click node_batch "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/batch_cleaning.py"
click node_single "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/singlethread_main/main.py"
click node_multi "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/multithread_main/main.py"
click node_config "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/config.py"
click node_pka "https://github.com/desirebonneau/admet-dataset-cleaner/tree/main/pka-model"
click node_smiles "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/smiles_utils.py"
click node_pubchem "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/pubchem_utils.py"
click node_duplicates "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/duplicate_utils.py"
click node_filters "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/chemical_filters.py"
click node_outliers "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/outlier_utils.py"
click node_tanimoto "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/tanimoto_utils.py"
click node_chirality "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/chirality_utils.py"
click node_io "https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/io_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_batch,node_single,node_multi toneBlue
class node_config,node_pka toneAmber
class node_smiles,node_pubchem,node_duplicates,node_filters,node_outliers,node_tanimoto,node_chirality,node_io toneMint
class node_user,node_service toneIndigo
```

## Components

### Entry points

- [Batch runner](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/batch_cleaning.py): `batch_cleaning.py`
- [Single-thread pipeline](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/singlethread_main/main.py): `singlethread_main/main.py`
- [Multithread pipeline](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/multithread_main/main.py): `multithread_main/main.py`

### Cleaning pipeline

- [Filter patterns](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/config.py): `config.py`
- [pKa model](https://github.com/desirebonneau/admet-dataset-cleaner/tree/main/pka-model): `pka-model`

### Chemistry utilities

- [SMILES handling](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/smiles_utils.py): `utils/smiles_utils.py`
- [PubChem lookup](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/pubchem_utils.py): `utils/pubchem_utils.py`
- [Exact deduplication](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/duplicate_utils.py): `utils/duplicate_utils.py`
- [Chemical filters](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/chemical_filters.py): `utils/chemical_filters.py`
- [Outlier detection](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/outlier_utils.py): `utils/outlier_utils.py`
- [Fingerprints](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/tanimoto_utils.py): `utils/tanimoto_utils.py`
- [Chirality transfer](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/chirality_utils.py): `utils/chirality_utils.py`
- [File utilities](https://github.com/desirebonneau/admet-dataset-cleaner/blob/main/utils/io_utils.py): `utils/io_utils.py`

### Other components

- **Dataset curator**
- **PubChem service**: external service

## Connections

- Dataset curator → Batch runner: configures runs (evidence: `README.md`)
- Batch runner → Single-thread pipeline: calls cleaner (evidence: `batch_cleaning.py`)
- Single-thread pipeline → SMILES handling: uses (evidence: `singlethread_main/main.py`)
- Single-thread pipeline → PubChem lookup: uses (evidence: `singlethread_main/main.py`)
- Single-thread pipeline → Exact deduplication: uses (evidence: `singlethread_main/main.py`)
- Single-thread pipeline → Chemical filters: uses (evidence: `singlethread_main/main.py`)
- Single-thread pipeline → Outlier detection: uses (evidence: `singlethread_main/main.py`)
- Single-thread pipeline → Chirality transfer: uses (evidence: `singlethread_main/main.py`)
- Single-thread pipeline → Filter patterns: uses patterns (evidence: `singlethread_main/main.py`)
- Single-thread pipeline → pKa model: predicts with (evidence: `README.md`)
- Multithread pipeline → SMILES handling: uses (evidence: `multithread_main/main.py`)
- Multithread pipeline → PubChem lookup: uses (evidence: `multithread_main/main.py`)
- Multithread pipeline → Exact deduplication: uses (evidence: `multithread_main/main.py`)
- Multithread pipeline → Chemical filters: uses (evidence: `multithread_main/main.py`)
- Multithread pipeline → Outlier detection: uses (evidence: `multithread_main/main.py`)
- Multithread pipeline → Chirality transfer: uses (evidence: `multithread_main/main.py`)
- Multithread pipeline → Filter patterns: uses patterns (evidence: `multithread_main/main.py`)
- Multithread pipeline → pKa model: predicts with (evidence: `README.md`)
- PubChem lookup → PubChem service: requests data (evidence: `utils/pubchem_utils.py`)
- Outlier detection → Fingerprints: generates fingerprints (evidence: `utils/outlier_utils.py`)