# sanofi-public/mrna-lm architecture

> Architecture diagram of the GitHub repository sanofi-public/mrna-lm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sanofi-public/mrna-lm
- Repository: https://github.com/sanofi-public/mrna-lm

## Overview

##### Project type and purpose
- This is a machine-learning fine-tuning pipeline for mRNA-based language models (CodonBERT, 5UTRBERT, 3UTRBERT).
- Purpose: load pre-trained transformer models, fine-tune on specific tasks (e.g., mRNA half-life prediction), and output trained checkpoints.

##### File structure
- Top-level scripts:
- dataload.py – loads and preprocesses CSV datasets
- FullModel.py & OneModel.py – define neural architectures (multi-task vs single-task heads)
- finetune_all.py – fine-tune one model on all tasks or multiple heads
- finetune_each.py – fine-tune separate models per task
- data/ – raw CSVs for tasks
- requirements.txt – dependencies
- README.md – usage and CLI args

##### README for additional insights
- CLI-based training entry points
- Pre-trained model download URLs
- Fine-tune parameters (batch size, LR, LoRA hyperparameters, CLIP loss options)

a. Main components:
1. External storage (pre-trained model artifacts – CodonBERT, 5UTRBERT, 3UTRBERT)
2. Data source (CSV files under data/)
3. DataLoader module (dataload.py)
4. Model definitions (FullModel.py and OneModel.py)
5. Training orchestrators (finetune_all.py, finetune_each.py)
6. GPU compute resource
7. Output directory (trained checkpoints, logs)
b. Relationships and interactions:
- CLI scripts invoke DataLoader → preprocess and batch data
- DataLoader feeds batches to Model
- Model loads pre-trained weights from external storage
- Training scripts coordinate forward/backward passes on GPU
- Checkpoints & metrics written to output folder
c. Architectural patterns and design principles:
- Modular separation: Data ingestion, Model definition, Training loop
- Support for multi-task vs single-task heads via separate model classes
- Parameterization via CLI arguments for hyperparameters (loRA, CLIP, LR, etc.)
d. Key technologies:
- PyTorch (implicit from model & training scripts)
- LoRA (low-rank adaptation) for fine-tuning
- CLIP loss integration (optional)
- CSV parsing (pandas or built-in)

##### Detail without over-compartmentalizing
- Six components above
- CLI entry points and key flags (–task, –lr, –lorar, –useCLIP)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Data Layer"
        DS["CSV Datasets"]:::data
    end
    subgraph "External Pre-trained Models"
        Pre["Pre-trained Weights"]:::external
    end
    DL["DataLoader (dataload.py)"]:::module
    subgraph "Model Definitions"
        FM["FullModel.py"]:::module
        OM["OneModel.py"]:::module
    end
    subgraph "Training Orchestrators"
        FA["finetune_all.py"]:::script
        FE["finetune_each.py"]:::script
    end
    GPU["GPU Compute"]:::compute
    Out["Output Checkpoints & Logs"]:::storage
    Doc["README.md"]:::doc
    Req["requirements.txt"]:::doc

    DS -->|"raw CSV files"| DL
    DL -->|"batches of token IDs"| FM
    DL -->|"batches of token IDs"| OM
    Pre -->|"load weights"| FM
    Pre -->|"load weights"| OM
    FA -->|"invoke DataLoader & Model"| DL
    FA -->|"invoke DataLoader & Model"| FM
    FA -->|"invoke DataLoader & Model"| OM
    FE -->|"invoke DataLoader & Model"| DL
    FE -->|"invoke DataLoader & Model"| FM
    FE -->|"invoke DataLoader & Model"| OM
    FM -->|"model architecture"| GPU
    OM -->|"model architecture"| GPU
    FA -->|"start training"| GPU
    FE -->|"start training"| GPU
    GPU -->|"checkpoints & metrics"| Out
    Doc -.-> FA
    Doc -.-> FE
    Req -.-> FA
    Req -.-> FE

    click DS "https://github.com/sanofi-public/mrna-lm/tree/main/data/"
    click DL "https://github.com/sanofi-public/mrna-lm/blob/main/dataload.py"
    click FM "https://github.com/sanofi-public/mrna-lm/blob/main/FullModel.py"
    click OM "https://github.com/sanofi-public/mrna-lm/blob/main/OneModel.py"
    click FA "https://github.com/sanofi-public/mrna-lm/blob/main/finetune_all.py"
    click FE "https://github.com/sanofi-public/mrna-lm/blob/main/finetune_each.py"
    click Doc "https://github.com/sanofi-public/mrna-lm/blob/main/README.md"
    click Req "https://github.com/sanofi-public/mrna-lm/blob/main/requirements.txt"

    classDef data fill:#ffe4b5,stroke:#cc8e35,color:#333;
    classDef external fill:#e0ffff,stroke:#5f9ea0,color:#333;
    classDef module fill:#e6e6fa,stroke:#9370db,color:#333;
    classDef script fill:#fafad2,stroke:#c9ae5d,color:#333;
    classDef compute fill:#ffe4e1,stroke:#cd5c5c,color:#333;
    classDef storage fill:#f0fff0,stroke:#66cdaa,color:#333;
    classDef doc fill:#f5f5f5,stroke:#a9a9a9,color:#333;
```
