# microsoft/olfaction architecture

> Architecture diagram of the GitHub repository microsoft/olfaction, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/olfaction
- Repository: https://github.com/microsoft/olfaction

## Overview

The project is a research-oriented deep learning system focused on “Mapping the combinatorial coding between olfactory receptors and perception.” Its purpose is to train models (both odorant-receptor and odorant-percept models) using graph neural networks, ESM embeddings, and custom receptor binding data to predict odorant interactions and perceptual responses.

##### Main Components
- Data Layer:
- “data” directory holds a rich set of configurations (under data/configures) for multiple datasets (BACE, BBBP, ClinTox, etc.).
- The “data/datasets” folder contains raw datasets and pre-computed artifacts (like receptor OR logits, CSVs, and PT files).
- Model & Training Components:
- Files such as classification.py, classification_ESM.py, and classification_OR_feat_ESM.py represent different model training pipelines.
- These modules handle training for odorant-receptor models (using per-residue ESM embeddings) and odorant-percept models (using OR activation logits).
- Data Processing & Utilities:
- Directories like molecule_percept, receptor_binding, and individual helper scripts (data_utils.py, seq_utils.py, utils.py) encapsulate custom data processing, feature extraction, and utilities.
- Inference and Experimentation:
- The “scripts” folder (e.g., generate_OR_predictions_pseudogenes.py, run_OR_percept_ablations_HORDE.sh) facilitates batch inference and experimental ablations as used in the paper.
- Notebooks (in the notebooks directory) provide exploratory data analysis and plotting results for cross-task statistics, percept predictions, etc.
- Configuration & Deployment:
- Environment and build configuration files (environment.yml, olfaction.yml) ensure reproducibility.
- Other documentation files (README.md, CODE_OF_CONDUCT.md, LICENSE, etc.) support collaboration and open-source governance.

##### Out Relationships & Interactions
- Data Flow:
- Raw datasets/configurations are fed from the “data” layer into data utility modules (found in molecule_percept/data_utils.py and receptor_binding/data_utils.py) for preprocessing.
- Preprocessed data is then used by the training scripts (classification_ESM.py and classification_OR_feat_ESM.py) to train deep learning models.
- Model Training & Inference:
- The model training components in classification files process both ESM embeddings and OR activations.
- Inference pipelines (scripts and receptor_binding modules) utilize these trained models to compute predictions and perform ablation analyses.
- Notebook Analysis:
- Notebooks integrate outputs from the training/inference pipelines to generate plots and statistical insights, reinforcing the experimental side of the work.
- Configuration Management:
- Config JSON files (located within data/configures) drive the experimentation by setting model variations and hyperparameters for different datasets.
- Environment definitions (environment.yml) govern the dependencies needed across the processing, training, and visualization layers.

##### Architectural Patterns & Design Principles
- Modular Design: The project separates concerns into distinct modules—data processing, model training, inference, and analysis—promoting clarity and reusability.
- Config-Driven Behavior: Extensive use of JSON configuration files allows flexible experimentation across multiple datasets and network architectures.
- Reproducibility: With explicit environment setup (environment.yml) and versioned configuration directories, the design stresses reproducibility and scalability of experiments.
- Scripted Pipelines: Batch scripts (e.g., run_OR_percept_ablations_HORDE.sh) orchestrate complete workflows, making it convenient to reproduce experiments as described in the paper.

##### How
- Data Repositories (Raw Datasets, JSON Configurations).
- Preprocessing/Utility Modules.
- Model Training Engines (split into ESM-based and OR-based training).
- Inference Pipelines and Experiment Scripts.
- Analysis Notebooks and Visualization Components.
- Annotations to highlight:
- Dependency relationships (e.g., model training depends on configurations, preprocessing uses utility modules, etc.).
- The role of environment and deployment files which ensure consistency across the pipeline.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Layer
    subgraph "Data Layer"
        DL["Raw Datasets & Configurations"]:::data
    end

    %% Data Processing & Utilities
    subgraph "Data Processing & Utilities"
        U1["molecule_percept/data_utils.py"]:::utils
        U2["receptor_binding/data_utils.py"]:::utils
        U3["seq_utils.py & utils.py"]:::utils
        U4["molecule_percept subdirs (goodscents, leffingwell)"]:::utils
    end

    %% Model & Training Pipelines
    subgraph "Model & Training Pipelines"
        MT1["classification.py"]:::training
        MT2["classification_ESM.py"]:::training
        MT3["classification_OR_feat_ESM.py"]:::training
    end

    %% Inference and Experimentation
    subgraph "Inference and Experimentation"
        IE1["Scripts (batch inference & ablations)"]:::inference
        IE2["Notebooks (exploratory analysis)"]:::inference
    end

    %% Configuration & Deployment
    subgraph "Configuration & Deployment"
        CD1["environment.yml & olfaction.yml"]:::config
        CD2["Documentation (README.md, CODE_OF_CONDUCT.md, LICENSE, etc.)"]:::config
    end

    %% Data flow connections
    DL -->|"feeds"| U1
    DL -->|"feeds"| U2
    DL -->|"feeds"| U3
    DL -->|"feeds"| U4

    U1 -->|"supports"| MT1
    U2 -->|"supports"| MT2
    U3 -->|"supports"| MT3
    U4 -->|"enriches"| MT1

    CD1 -->|"configures"| MT1
    CD1 -->|"configures"| MT2
    CD1 -->|"configures"| MT3
    CD2 -->|"guides"| MT1

    MT1 -->|"triggers"| IE1
    MT2 -->|"triggers"| IE1
    MT3 -->|"triggers"| IE1

    IE1 -->|"visualizes"| IE2

    %% Click events
    click DL "https://github.com/microsoft/olfaction/tree/main/data/"
    click U1 "https://github.com/microsoft/olfaction/blob/main/molecule_percept/data_utils.py"
    click U2 "https://github.com/microsoft/olfaction/blob/main/receptor_binding/data_utils.py"
    click U3 "https://github.com/microsoft/olfaction/blob/main/seq_utils.py"
    click U4 "https://github.com/microsoft/olfaction/tree/main/molecule_percept/"
    click MT1 "https://github.com/microsoft/olfaction/blob/main/classification.py"
    click MT2 "https://github.com/microsoft/olfaction/blob/main/classification_ESM.py"
    click MT3 "https://github.com/microsoft/olfaction/blob/main/classification_OR_feat_ESM.py"
    click IE1 "https://github.com/microsoft/olfaction/tree/main/scripts/"
    click IE2 "https://github.com/microsoft/olfaction/tree/main/notebooks/"
    click CD1 "https://github.com/microsoft/olfaction/blob/main/environment.yml"
    click CD2 "https://github.com/microsoft/olfaction/blob/main/README.md"

    %% Class definitions with colours
    classDef data fill:#AEDFF7,stroke:#1F78B4,stroke-width:2px;
    classDef utils fill:#FFF2A8,stroke:#D8A20D,stroke-width:2px;
    classDef training fill:#C2E0C6,stroke:#3B7F3B,stroke-width:2px;
    classDef inference fill:#F7C8A2,stroke:#D95F02,stroke-width:2px;
    classDef config fill:#E2D1F9,stroke:#6A51A3,stroke-width:2px;
```
