# helloworldlty/ukbiolm architecture

> Architecture diagram of the GitHub repository helloworldlty/ukbiolm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/helloworldlty/ukbiolm
- Repository: https://github.com/helloworldlty/ukbiolm

## Overview

##### Project Type and Purpose
- This is a genomics-focused deep learning research codebase. It provides:
- A core Python library (`grelu`) for data I/O, preprocessing, model definitions, interpretation, and visualization.
- A pre-training pipeline using large variant-augmented genomic datasets.
- Fine-tuning and inference applications for cell-line improvement and variant effect prediction (UKBioBERT, UKBioFormer).
- Notebooks and scripts for data preparation, pre-training, and downstream applications.

##### File Structure
- Top-level packages:
- `grelu/`: Core reusable modules:
– `data/`, `transforms/`: dataset loading, augmentation, preprocessing.
– `model/`, `lightning/`: model architectures, heads, losses, training utilities using PyTorch Lightning.
– `io/`: readers/writers for FASTA, BED, BigWig, MEME, genome resources.
– `resources/`: static blacklists and motif libraries.
– `sequence/`, `utils.py`, `design.py`, `variant.py`: utility functions.
- `pretraining/`: Jupyter notebooks and scripts (`run_mlm.py`, `run_pretrain.sh`, etc.) to prepare datasets and run masked-LM or contrastive pre-training over UK Biobank, GTEx, ROSMAP.
- `improve_cellline/`: scripts and models for fine-tuning EPInformer on cell-line data.
- `ukbiobert_application/` and `ukbioformer_application/`: notebooks and config files for downstream inference and visualization using the pre-trained models.
- Environment YAMLs (`ukbiobert_pretrain.yml`, etc.): specify dependencies and GPU configurations.

##### Extract Key Architecture Details from README
- Environments are separated by model type (UKBioBERT vs. UKBioFormer) and hardware (H100 vs. no-H100).
- Data sources: UK Biobank, GTEx, ROSMAP (external databases).
- Pre-training → model weights → fine-tuning/inference → downstream analyses.
- External frameworks: Huggingface, PyTorch Lightning, `grelu` base library.

##### Main Components & Interactions
a. Core Library (`grelu`):
- Data ingestion & preprocessing
- Model building blocks & trunks (Enformer, Borzoi, ExplainN)
- Training utilities (losses, metrics via Lightning)
- Interpretation & visualization tools
b. Data Sources (External):
- UK Biobank, GTEx, ROSMAP raw genomic data
- Pre-computed reference sequences and variant catalogs
c. Pre-training Pipeline:
- Data pipeline (variant + reference → masked LM samples)
- Model instantiation from `grelu.model.trunks`
- Training loop via PyTorch Lightning
- Checkpoint saving
d. Fine-tuning / Application Pipelines:
- `improve_cellline`: EPInformer model training on cell-line data
- `ukbiobert_application` / `ukbioformer_application`: notebooks for inference, ensemble, variant effect checks
- Use of pre-trained checkpoints, further training/inference
e. Environments & Infrastructure:
- Conda environments for each pipeline/hardware setup
- GPU clusters (H100, A100, A40)
- Shell scripts (`run_pretrain.sh`, `download_data.sh`) for orchestration

##### Architectural Patterns & Design Principles
- Modular core library (`grelu`) that separates concerns: I/O, data transforms, model definitions, training, and interpretation.
- Pipeline orchestration via standalone scripts/notebooks.
- Use of PyTorch Lightning for standardized training loops.
- Clear separation of pre-training vs. fine-tuning workflows.
- Externalization of static resources (blacklists, motif files) under `grelu/resources`.

##### Final Checklist
- Clear labels for each component and sub‐component.
- External dependencies and resource contexts.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "External Data Sources"
        UKB["UK Biobank"]:::external
        GTEx["GTEx"]:::external
        ROSMAP["ROSMAP"]:::external
    end

    subgraph "Storage"
        RawData[(Raw Genomic Data)]:::storage
        Checkpoints[(Model Checkpoints)]:::storage
        Resources[(Static Resources)]:::storage
    end

    subgraph "Conda Environments"
        Env_UKB["ukbiobert_pretrain.yml"]:::config
        Env_UKB_H100["ukbiobert_pretrain_h100.yml"]:::config
        Env_UKF["ukbioformer.yml"]:::config
    end

    subgraph "Core Library (grelu)"
        subgraph "Data I/O & Preprocessing"
            DataIO["grelu/data/"]:::library
        end
        subgraph "Transforms & Augmentations"
            Transforms["grelu/transforms/"]:::library
        end
        subgraph "Model Definitions & Trunks"
            Blocks["blocks.py"]:::library
            Heads["heads.py"]:::library
            Layers["layers.py"]:::library
            Models["models.py"]:::library
            Position["position.py"]:::library
            subgraph "Trunks"
                Borzoi["borzoi.py"]:::library
                Enformer["enformer.py"]:::library
                ExplainN["explainn.py"]:::library
            end
        end
        subgraph "Training Utilities"
            Lightning["grelu/lightning/"]:::library
        end
        subgraph "Interpretation & Visualization"
            Interpret["grelu/interpret/"]:::library
            Visualize["visualize.py"]:::library
        end
        subgraph "I/O Readers/Writers"
            IO["grelu/io/"]:::library
        end
        subgraph "Static Resources"
            StaticRes["grelu/resources/"]:::library
        end
    end

    subgraph "Pre-training Pipeline"
        DataPrep["Data Pipeline"]:::pipeline
        PretrainScripts["pretraining/"]:::pipeline
        TrainPT["Pre-training"]:::pipeline
    end

    subgraph "Fine-tuning Pipelines"
        EPTrain["EPInformer Training"]:::pipeline
        EPCode["improve_cellline/"]:::pipeline
    end

    subgraph "Application Pipelines"
        UKBApp["UKBioBERT Application"]:::pipeline
        UKFApp["UKBioFormer Application"]:::pipeline
    end

    UKB -->|provides| RawData
    GTEx -->|provides| RawData
    ROSMAP -->|provides| RawData

    RawData -->|input| DataIO
    DataIO --> Transforms
    Transforms --> DataPrep
    DataPrep --> PretrainScripts
    PretrainScripts --> TrainPT
    TrainPT --> Checkpoints

    Checkpoints --> EPTrain
    Checkpoints --> UKBApp
    Checkpoints --> UKFApp

    EPTrain -->|fine-tuning| EPCode
    EPCode --> Checkpoints

    UKBApp -->|inference| UKBApp
    UKFApp -->|inference| UKFApp

    Env_UKB -->|runs| PretrainScripts
    Env_UKB_H100 -->|runs| PretrainScripts
    Env_UKF -->|runs| UKFApp

    PretrainScripts -->|uses| DataIO
    PretrainScripts -->|uses| Transforms
    PretrainScripts -->|uses| Models
    PretrainScripts -->|uses| Lightning

    EPCode -->|uses| Models
    EPCode -->|uses| Lightning

    UKBApp -->|uses| Models
    UKFApp -->|uses| Models
    UKBApp -->|uses| Visualize

    Resources --> DataIO
    Resources --> IO
    Resources --> Interpret
    Resources --> Visualize

    click DataIO "https://github.com/helloworldlty/ukbiolm/tree/main/grelu/data/"
    click Transforms "https://github.com/helloworldlty/ukbiolm/tree/main/grelu/transforms/"
    click Blocks "https://github.com/helloworldlty/ukbiolm/blob/main/grelu/model/blocks.py"
    click Heads "https://github.com/helloworldlty/ukbiolm/blob/main/grelu/model/heads.py"
    click Layers "https://github.com/helloworldlty/ukbiolm/blob/main/grelu/model/layers.py"
    click Models "https://github.com/helloworldlty/ukbiolm/blob/main/grelu/model/models.py"
    click Position "https://github.com/helloworldlty/ukbiolm/blob/main/grelu/model/position.py"
    click Borzoi "https://github.com/helloworldlty/ukbiolm/blob/main/grelu/model/trunks/borzoi.py"
    click Enformer "https://github.com/helloworldlty/ukbiolm/blob/main/grelu/model/trunks/enformer.py"
    click ExplainN "https://github.com/helloworldlty/ukbiolm/blob/main/grelu/model/trunks/explainn.py"
    click Lightning "https://github.com/helloworldlty/ukbiolm/tree/main/grelu/lightning/"
    click Interpret "https://github.com/helloworldlty/ukbiolm/tree/main/grelu/interpret/"
    click Visualize "https://github.com/helloworldlty/ukbiolm/blob/main/grelu/visualize.py"
    click IO "https://github.com/helloworldlty/ukbiolm/tree/main/grelu/io/"
    click StaticRes "https://github.com/helloworldlty/ukbiolm/tree/main/grelu/resources/"
    click PretrainScripts "https://github.com/helloworldlty/ukbiolm/tree/main/pretraining/"
    click EPCode "https://github.com/helloworldlty/ukbiolm/tree/main/improve_cellline/"

    classDef external fill:#cccccc,stroke:#333,stroke-width:1px
    classDef storage fill:#ffa500,stroke:#333,stroke-width:1px
    classDef pipeline fill:#8fbc8f,stroke:#333,stroke-width:1px
    classDef library fill:#f0e68c,stroke:#333,stroke-width:1px
    classDef config fill:#dda0dd,stroke:#333,stroke-width:1px
```
