# osu-nlp-group/llm4chem architecture

> Architecture diagram of the GitHub repository osu-nlp-group/llm4chem, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/osu-nlp-group/llm4chem
- Repository: https://github.com/osu-nlp-group/llm4chem

## Overview

##### Main components
- Training & Fine-Tuning Module: Represented by files such as finetune.py, trainer.py, and config.py. This component orchestrates model fine-tuning using LoRA on a designated dataset (SMolInstruct) and interacts with external base LLMs (Mistral, Llama2, etc.).
- Generation / Inference Module: Represented by generation.py and LlaSMolGeneration (as seen in the README usage example). This module handles user queries (wrapped with chemical tags) and returns model outputs.
- Evaluation Pipeline: Comes in two parts—first, generate_on_dataset.py produces predictions based on various tasks; second, extract_prediction.py (or prediction_extraction.py) and compute_metrics.py extract model outputs and compute performance metrics.
- Utility Functions: The utils directory (with modules for SMILES canonicalization, chat generation, core tagging, and general prompting) provides supporting functions critical for preprocessing, formatting, and post-processing of inputs/outputs.
- Dataset & External Resources: The SMolInstruct dataset (hosted on HuggingFace) and linked paper/model pages which are used for training/evaluation and for users to reference model outputs.

##### Relationships/interactions
- Data Flow: The dataset (SMolInstruct) is first fed into the Training Module. The Training Module uses the configuration and orchestrates model fine-tuning using the provided scripts.
- For Inference, an application or script (invoking generation.py) calls the trained model to generate responses when given properly formatted prompts (e.g., with <SMILES> or <IUPAC> tags).
- The Evaluation Pipeline works as a post-process: first generating outputs (via generate_on_dataset.py), then extracting predictions (via extract_prediction.py) and finally computing metrics (via compute_metrics.py). This forms a sequential pipeline where outputs of one step feed into the next.
- Utility functions act as shared libraries that are used across training, generation, and evaluation for tasks such as data canonicalization and formatting.
- External integrations are seen in fine-tuning commands (which require external GPU orchestration and possibly integration with wandb for logging).

##### Architectural patterns/design principles
- Modularization: The repository is clearly divided into separate scripts for training, inference, evaluation, and utility functions.
- Pipeline/Sequential Processing: There’s a linear processing flow during evaluation—from generation to extraction to metrics computation.
- Separation of Concerns: The training, inference, and utility functionalities are decoupled in different scripts and modules.
- Reusability: The utilities offer reusable code (e.g., SMILES canonicalization) which ensures that domain-specific formatting is standardized.

##### Relevant technologies
- Python (as the main programming language)
- External LLMs (e.g., Mistral, Llama2, Galactica, CodeLlama)
- LoRA for parameter-efficient fine-tuning
- Possibly accelerate (from the training command) for multi-GPU support
- wandb as a logging/monitoring tool during training

##### Specific Elements
- Clear labels for each main component (e.g., “Fine-Tuning Module (finetune.py, trainer.py)”, “Inference Module (generation.py)”, “Evaluation & Metrics (generate_on_dataset.py, extract_prediction.py, compute_metrics.py)”, “Utility Functions (utils/*)”).
- From Dataset to Training Module
- From Training Module to Model (which then is used by Inference Module)
- From Inference Module to Evaluation Pipeline
- Bidirectional flows where necessary (e.g., configuration inputs and output files).
- For utility modules (perhaps rounded corners)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Resources
    A1["SMolInstruct Dataset"]:::external
    A2["External LLMs (Mistral, Llama2, etc.)"]:::external

    %% Training & Fine-Tuning Module
    subgraph "Training & Fine-Tuning Module"
        T1["finetune.py"]:::training
        T2["trainer.py"]:::training
        T3["config.py"]:::training
        T4["model.py"]:::training
    end

    %% Trained Model Representation
    TM["Trained Model"]:::model

    %% Inference Module
    subgraph "Inference Module"
        I1["generation.py"]:::inference
    end

    %% Evaluation Pipeline
    subgraph "Evaluation Pipeline"
        E1["generate_on_dataset.py"]:::evaluation
        E2["extract_prediction.py"]:::evaluation
        E3["compute_metrics.py"]:::evaluation
    end

    %% Utility Functions
    subgraph "Utility Functions"
        U1["chat_generation.py"]:::utility
        U2["core_tagger.py"]:::utility
        U3["general_prompter.py"]:::utility
        U4["smiles_canonicalization.py"]:::utility
        U5["__input__.py"]:::utility
    end

    %% User Interaction/Results Analysis
    UF["User Query & Result Analysis"]:::user

    %% Connections: Data Flow and Dependencies
    A1 -->|"provides_data"| T1
    A1 --> T2
    A1 --> T3
    A1 --> T4
    A2 --- T1
    A2 --- T2
    A2 --- T3
    A2 --- T4

    T1 --> TM
    T2 --> TM
    T3 --> TM
    T4 --> TM

    TM --> I1
    TM --> E1

    I1 -->|"model_output"| UF

    E1 --> E2
    E2 --> E3
    E3 -->|"evaluated_results"| UF

    %% Utility Functions used across modules
    U1 --- T1
    U1 --- I1
    U1 --- E2
    U2 --- T1
    U3 --- I1
    U4 --- T1
    U5 --- T1

    %% Click Events for Component Mapping
    click T1 "https://github.com/osu-nlp-group/llm4chem/blob/main/finetune.py"
    click T2 "https://github.com/osu-nlp-group/llm4chem/blob/main/trainer.py"
    click T3 "https://github.com/osu-nlp-group/llm4chem/blob/main/config.py"
    click T4 "https://github.com/osu-nlp-group/llm4chem/blob/main/model.py"
    click I1 "https://github.com/osu-nlp-group/llm4chem/blob/main/generation.py"
    click E1 "https://github.com/osu-nlp-group/llm4chem/blob/main/generate_on_dataset.py"
    click E2 "https://github.com/osu-nlp-group/llm4chem/blob/main/extract_prediction.py"
    click E3 "https://github.com/osu-nlp-group/llm4chem/blob/main/compute_metrics.py"
    click U1 "https://github.com/osu-nlp-group/llm4chem/blob/main/utils/chat_generation.py"
    click U2 "https://github.com/osu-nlp-group/llm4chem/blob/main/utils/core_tagger.py"
    click U3 "https://github.com/osu-nlp-group/llm4chem/blob/main/utils/general_prompter.py"
    click U4 "https://github.com/osu-nlp-group/llm4chem/blob/main/utils/smiles_canonicalization.py"
    click U5 "https://github.com/osu-nlp-group/llm4chem/blob/main/utils/__input__.py"

    %% Styling Classes
    classDef external fill:#ffefba,stroke:#d6a55f,stroke-width:2px;
    classDef training fill:#c8e6c9,stroke:#388e3c,stroke-width:2px;
    classDef model fill:#fff9c4,stroke:#fbc02d,stroke-width:2px;
    classDef inference fill:#bbdefb,stroke:#1976d2,stroke-width:2px;
    classDef evaluation fill:#ffe0b2,stroke:#f57c00,stroke-width:2px;
    classDef utility fill:#d1c4e9,stroke:#7e57c2,stroke-width:2px;
    classDef user fill:#ffcdd2,stroke:#e53935,stroke-width:2px;
```
