How lbcb-sci/rinalmo works
Main Components
• Data Preparation & Processing Layer:
• This consists of the “rinalmo/data” directory where various downstream tasks have their own sub-modules (expression level, ncrna classification, ribosome loading, secondary structure, splice site prediction, translation efficiency). Each sub-module contains components such as datamodule.py and dataset.py which manage data ingestion and preparation.
• The “ft_schedules” folder contains YAML configuration files for fine-tuning on various tasks (e.g., giga_mrl_ft.yaml, giga_ncRNA_class_ft.yaml, etc.). These define hyperparameters and schedules for training various downstream models.
• Model Components:
• Found in the “rinalmo/model” folder, it includes core modeling scripts like model.py, attention.py, modules.py, downstream.py, and rope.py. These components likely define the architecture for the large RNA language model and its extensions for downstream tasks.
• The “rinalmo/pretrained.py” file exposes an API to load pre-trained models.
• Utility & Configuration Layer:
• The “rinalmo/config.py” sets configuration parameters for the overall run, and the “rinalmo/utils” folder contains helper scripts for operations such as downloading resources, fine-tuning callbacks, scaling, data preparation for specific tasks, and metrics computations (e.g., splice_site_metrics).
• The “rinalmo/resources” folder stores JSON files that map resources like model files to disk or remote configurations.
• Experiment and Training Layer:
• Several training scripts at the root level (e.g., train_expression_level.py, train_ncrna_classification.py, etc.) orchestrate training and evaluation runs for various downstream tasks.
• Configuration files (both YAML in ft_schedules and parameters in pyproject.toml/environment.yml) help manage dependencies and build environments.
• External Dependencies & Setup:
• The README explains the installation process using conda (environment.yml), pip install (pyproject.toml), and downloading pre-trained/fine-tuned weights. This underlines the importance of reproducibility and well-defined environments.
Relationships and Interactions
• Data flows from raw RNA sequence inputs through the datamodules (within each downstream directory) to prepare the data. The tokens are then processed by the model which is structured within “rinalmo/model” – this provides the representations that feed into task-specific prediction heads included in the training scripts.
• The “rinalmo/pretrained.py” acts as the entry-point for inference when pre-trained weights are loaded and used.
• The utility components offer support for configuration, scaling, callbacks (for fine-tuning), and other auxiliary functions that connect the data and model layers.
• The fine-tuning and evaluation scripts tie everything together – reading configuration (from ft_schedules and command-line args), loading data, invoking the model, and producing outputs for downstream tasks.
Architectural Patterns and Design Principles
• Modularity: Each major function (data processing, model building, evaluation) is cleanly separated into its own module/folder.
• Extensibility: The use of configuration files and distinct subdirectories for each downstream task allows extending to new tasks with minimal interference.
• Reproducibility: By separating configuration (environment.yml, pyproject.toml, YAML schedules) and code, the design supports reproducible experiments.
Relevant Technologies, Frameworks, or Libraries
• PyTorch is assumed (given the use of torch, CUDA and torch.cuda.amp in the sample inference code).
• Flash-attn is explicitly mentioned as a dependency.
• The conda environment (environment.yml) implies a precise requirement for running the experiments.
• YAML for fine-tuning schedules and JSON for resource mapping.