# frogstar-world-b/transfer-learning-chemberta architecture

> Architecture diagram of the GitHub repository frogstar-world-b/transfer-learning-chemberta, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/frogstar-world-b/transfer-learning-chemberta
- Repository: https://github.com/frogstar-world-b/transfer-learning-chemberta

## Overview

##### Main Components
- Data Sources: Represent the “data” folder containing Excel and CSV files. These files are the raw input containing chemical information and pIC50 values.
- Preprocessing & Exploratory Data Analysis (EDA): Capture the functionality implemented in preprocessing.ipynb. This component is responsible for cleaning the data, extracting SMILES information, and preparing the dataset.
- Modeling Approaches:
a. Fine-Tuning Module: Corresponds to fine-tune.ipynb, where the ChemBERTa model is further trained using the Transformers library.
b. Transfer Learning Module: Reflected in transfer-learning.ipynb that uses pre-trained ChemBERTa embeddings along with an XGBoost regressor.
c. Extended Transfer Learning with Descriptors: Represented by transfer-learning-plus-descriptors.ipynb, which further enhances the transfer learning pipeline by incorporating molecular descriptors.
- Evaluation and Metrics: A logical part of the process where performance metrics like Mean Squared Error (MSE) and Spearman Correlation are computed to compare the approaches.

##### Denote Relationships and Interactions
- Model Pipeline: From preprocessed data, branch out into both the fine-tuning module and the transfer learning module (with an extra branch to incorporate molecular descriptors).
- Feedback/Results Loop: Connect model outputs to an evaluation component which then may feed insights back into an iteration of preprocessing or hyperparameter tuning.

##### Architectural Patterns and Design Principles
- Modularity: Even though the project is packaged as Jupyter notebooks, the design demonstrates separation of concerns by isolating data preprocessing, model training, and evaluation tasks.
- Experimentation vs. Production: The structure is set up for comparative experimentation. Capture the idea of parallel pipelines (fine-tuning vs. transfer learning) and highlight points where these pipelines differ and converge.
- Integration of Heterogeneous Technologies: Emphasize that the architecture uses multiple libraries and frameworks (PyTorch, Transformers, XGBoost, RDKit), each fulfilling specific roles (deep learning, regression, chemical informatics).

##### Relevant Technologies and Frameworks
- PyTorch and Hugging Face Transformers: Used within the fine-tuning workflow.
- XGBoost: Employed in the transfer-learning pipelines.
- RDKit: Critical for handling and processing chemical information.
- Standard Python Data Science Stack: NumPy, pandas, scikit-learn, Matplotlib, etc., which operate under the hood for EDA and evaluation.

##### Elements
- Clear Labels: Each component (Data Ingestion, Preprocessing & EDA, Fine-Tuning, Transfer Learning, Extended Transfer Learning, Evaluation) should be explicitly labeled.

- Begin on the left with “Data Sources” (show data/hDHFR.xlsx and data/hDHFR_pIC50_data.csv).
- Split the workflow into two parallel paths:
- Fine-Tuning Path: Illustrate the fine-tuning process using ChemBERTa with a block labeled “Fine-Tuning ChemBERTa” (from fine-tune.ipynb), leveraging PyTorch & Transformers.
- Transfer Learning Path:
- One branch for “Transfer Learning – ChemBERTa Embeddings” (from transfer-learning.ipynb) using XGBoost.
- Another branch for “Transfer Learning with Descriptors” (from transfer-learning-plus-descriptors.ipynb) which integrates additional molecular descriptors.
- Converge these pipelines into an “Evaluation” component where metrics (MSE, Spearman Correlation) are computed and compared.
- Optionally, include a side note that hyperparameter tuning is an iterative process which might loop back to the model training blocks.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources Subgraph
    subgraph "Data Sources"
        DS1["hDHFR.xlsx"]:::file
        DS2["hDHFR_pIC50_data.csv"]:::file
    end

    %% Preprocessing
    P["Preprocessing & EDA (preprocessing.ipynb)"]:::eda

    %% Modeling Approaches
    FT["Fine-Tuning ChemBERTa (fine-tune.ipynb)"]:::model
    TL["Transfer Learning - ChemBERTa Embeddings (transfer-learning.ipynb)"]:::model
    ETL["Transfer Learning with Descriptors (transfer-learning-plus-descriptors.ipynb)"]:::model

    %% Evaluation
    Eval["Evaluation & Metrics"]:::eval

    %% External Libraries
    PT["PyTorch & Transformers"]:::lib
    XG["XGBoost"]:::lib
    RD["RDKit"]:::lib

    %% Data Flow
    DS1 --> P
    DS2 --> P

    %% Branching from Preprocessing
    P --> FT
    P --> TL
    P --> ETL

    %% Model to Evaluation
    FT --> Eval
    TL --> Eval
    ETL --> Eval

    %% Feedback Loop
    Eval --> P

    %% External Library Usage
    FT ---|"uses"| PT
    TL ---|"uses"| XG
    ETL ---|"uses"| RD

    %% Click Events for Component Mapping
    click DS1 "https://github.com/frogstar-world-b/transfer-learning-chemberta/blob/main/data/hDHFR.xlsx"
    click DS2 "https://github.com/frogstar-world-b/transfer-learning-chemberta/blob/main/data/hDHFR_pIC50_data.csv"
    click P "https://github.com/frogstar-world-b/transfer-learning-chemberta/blob/main/preprocessing.ipynb"
    click FT "https://github.com/frogstar-world-b/transfer-learning-chemberta/blob/main/fine-tune.ipynb"
    click TL "https://github.com/frogstar-world-b/transfer-learning-chemberta/blob/main/transfer-learning.ipynb"
    click ETL "https://github.com/frogstar-world-b/transfer-learning-chemberta/blob/main/transfer-learning-plus-descriptors.ipynb"

    %% Styles for better color coding
    classDef file fill:#ffe6cc,stroke:#b36b00,stroke-width:2px;
    classDef eda fill:#ccffcc,stroke:#228b22,stroke-width:2px;
    classDef model fill:#cce6ff,stroke:#1a75ff,stroke-width:2px;
    classDef eval fill:#e6ccff,stroke:#9933ff,stroke-width:2px;
    classDef lib fill:#ffffcc,stroke:#cccc00,stroke-width:2px;
```
