# ml4bio/rna-fm architecture

> Architecture diagram of the GitHub repository ml4bio/rna-fm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ml4bio/rna-fm
- Repository: https://github.com/ml4bio/rna-fm

## Overview

##### Project Type and Purpose
- This is a Python-based machine-learning library and CLI tool (“RNA-FM”) providing a foundation transformer model for RNA sequences (and an mRNA variant), plus a downstream pipeline for embedding extraction, secondary-structure prediction, and extensibility into tertiary structure or inverse-folding modules.
- The README shows it is primarily a library + CLI interface rather than a web app; it integrates with external Hugging Face storage for model weights and can be used via Python API or shell commands.

##### File Structure Analysis
Top-level directories and files:
- fm/ – Core Python package containing model definitions (BERT-style Transformer), attention modules, pretrained loader, data utilities, and downstream predictor modules.
- redevelop/ – Extended codebase for CLI workflows:
– config/ – default YAML configurations (model, data, training/prediction settings)
– data/ – batch converters, datasets, samplers, transforms for FASTA and PDB inputs
– engine/ – “predictor.py” orchestrates the model inference loop
– launch/ – entrypoint scripts (e.g., launch/predict.py) that parse arguments, load config, invoke engine
– model/ – mirror of fm/ modules for CLI usage (baselines, backbones, weight init/load)
– pretrained/ – experiment and prediction YAML files under redevelop/pretrained
– utils/ – logging, checkpointing, tensorboard integration, plotting
- setup.py + environment.yml – packaging and dependency specifications (PyTorch, Hydra/OmegaConf, etc.)
- docs/ – MkDocs or Jekyll-based documentation and tutorials
- tutorials/ – Jupyter notebooks and scripts for common use cases (secondary structure, UTR prediction, clustering)

##### README Insights
- Describes two models (RNA-FM and mRNA-FM), downstream ecosystem modules (RhoFold, RiboDiffusion, RhoDesign) that are separate GitHub projects but connect conceptually.
- Shows typical CLI commands that load a YAML config, read FASTA, run “predictor.py,” and write embeddings, .ct files, .npy matrices.
- Demonstrates Python-API usage via `fm.pretrained.rna_fm_t12()`.

a. Main Components
- User Interfaces:
– CLI script (launch/predict.py)
– Python API (fm.pretrained.* calls)
- Core Library:
– fm package (model definitions, tokenizers, pretrained loader)
– redevelop package (config, data pipeline, engine, model wrappers)
- Configuration: YAML files under redevelop/config and redevelop/pretrained
- Data Layer: FASTA/PDB input → batch conversion → dataset/transforms → DataLoader
- Model Layer: Transformer backbone → downstream predictor heads (secondary structure, pairwise, UTR)
- Persistence & Logging: checkpoint manager, tensorboard, results folder (.npy, .ct, embeddings)
- External Dependencies:
– Pretrained weight storage (Hugging Face, local ./pretrained)
– Python packages (PyTorch, Hydra/OmegaConf, numpy)
– Optional online servers for web-based inference

b. Relationships and Data Flow
1. User invokes CLI or API
2. Config loader reads YAML → builds data pipeline and model
3. Data pipeline reads FASTA via batch_converter → produces token tensors
4. Model loader fetches .pth weights from pretrained folder or remote
5. Forward pass through fm Transformer → representations → downstream modules
6. Engine collects outputs → saves embeddings/structures to disk → logs metrics
7. For web deployment, same engine exposes REST endpoints (external)

c. Architectural Patterns & Principles
- Strict separation of concerns: fm (pure model code) vs. redevelop (pipeline & CLI)
- Configuration-driven design via YAML/Hydra
- Modular data loaders, transforms, and samplers that can be swapped for different tasks
- Reuse of pretrained weights via centralized loader in fm.pretrained
- Extensibility: new predictor heads plug into downstream_module APIs

d. Key Technologies & Frameworks
- Python 3.8+
- PyTorch for model definitions and inference
- Hydra OR OmegaConf for config management
- YAML for experimental settings
- Jupyter for tutorials
- MkDocs/Jekyll under docs for documentation site

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Interfaces
    subgraph "User Interfaces"
        CLI["CLI: launch/predict.py"]:::code
        API["Python API: fm.pretrained"]:::code
    end

    %% Configuration
    subgraph "Configuration"
        Config["YAML Configs (defaults)"]:::data
        PretrainedYAML["Pretrained YAMLs"]:::data
    end

    %% Data Layer
    subgraph "Data Layer"
        BatchConverter["batch_converter.py"]:::code
        Dataset["bio_seq.py"]:::code
        Samplers["samplers"]:::code
        Transforms["cla_transforms.py"]:::code
    end

    %% Model Layer
    subgraph "Model Layer"
        subgraph "Core Library (fm)"
            ModelDef["model.py"]:::code
            ESM1["esm1.py"]:::code
            Modules["modules.py"]:::code
            AxialAttn["axial_attention.py"]:::code
            MultiHead["multihead_attention.py"]:::code
            DataUtils["data.py"]:::code
            Constants["constants.py"]:::code
            Version["version.py"]:::code
        end
        subgraph "Downstream Predictors"
            DownModule["downstream_module.py"]:::code
            Baseline["baseline.py"]:::code
            PairLinear["linear_dot_product.py"]:::code
            PairConcat["pairwise_concat.py"]:::code
            WeightsInit["weights_init.py"]:::code
            WeightsLoad["weights_load.py"]:::code
        end
        subgraph "Redevelop Model Wrappers"
            Backbones["backbones.py"]:::code
            RDModule["downstream_module.py"]:::code
            RPairLinear["linear_dot_product.py"]:::code
            RPairConcat["pairwise_concat.py"]:::code
        end
    end

    %% Engine & Pipeline
    subgraph "Engine & Pipeline"
        Engine["predictor.py"]:::code
    end

    %% Persistence & Logging
    subgraph "Persistence & Logging"
        Logger["logger.py"]:::code
        Checkpoint["checkpoint.py"]:::code
        TBLogger["tensorboard_logger.py"]:::code
    end

    %% Supporting Resources
    subgraph "Supporting Resources"
        Docs["docs"]:::code
        Tutorials["tutorials"]:::code
        Setup["setup.py & environment.yml"]:::code
    end

    %% External Dependencies
    subgraph "External Dependencies"
        PyTorch["PyTorch"]:::ext
        HuggingFace["Hugging Face"]:::ext
        Hydra["Hydra/OmegaConf"]:::ext
    end

    %% Connections
    CLI -->|"loads config"| Config
    API -->|"loads config"| Config
    PretrainedYAML -->|"provides experiment settings"| Config

    Config -->|"defines data pipeline"| BatchConverter
    Config -->|"defines data pipeline"| Dataset
    Config -->|"defines data pipeline"| Samplers
    Config -->|"defines data pipeline"| Transforms

    BatchConverter -->|"raw sequences"| DataUtils
    Dataset -->|"samples"| Samplers
    Samplers -->|"batches"| Transforms
    Transforms -->|"token tensors"| DataUtils
    DataUtils -->|"token tensors"| ModelDef
    Constants -->|"constants"| ModelDef

    CLI -->|"invokes"| Engine
    API -->|"invokes"| Engine
    Engine -->|"uses wrappers"| Backbones
    Engine -->|"uses wrappers"| RDModule
    Engine -->|"loads weights"| WeightsLoad
    Engine -->|"initializes weights"| WeightsInit

    Backbones -->|"calls"| ModelDef
    RDModule -->|"calls"| DownModule
    DownModule -->|"secondary modules"| Baseline
    DownModule -->|"pairwise modules"| PairLinear
    DownModule -->|"pairwise modules"| PairConcat

    Engine -->|"outputs metrics"| Logger
    Engine -->|"saves checkpoints"| Checkpoint
    Engine -->|"logs experiments"| TBLogger

    PyTorch -->|"framework"| ModelDef
    Hydra -->|"parsing"| Config
    HuggingFace -->|"provides weights"| WeightsLoad

    %% Resources
    Setup -->|"environment"| CLI
    Setup -->|"environment"| API
    Docs -->|"documentation"| CLI
    Tutorials -->|"examples"| CLI

    %% Click Events
    click CLI "https://github.com/ml4bio/rna-fm/blob/main/redevelop/launch/predict.py"
    click API "https://github.com/ml4bio/rna-fm/blob/main/fm/pretrained.py"
    click Config "https://github.com/ml4bio/rna-fm/blob/main/redevelop/config/defaults.py"
    click PretrainedYAML "https://github.com/ml4bio/rna-fm/tree/main/redevelop/pretrained/"
    click BatchConverter "https://github.com/ml4bio/rna-fm/blob/main/redevelop/data/batch_converter.py"
    click Dataset "https://github.com/ml4bio/rna-fm/blob/main/redevelop/data/datasets/bio_seq.py"
    click Samplers "https://github.com/ml4bio/rna-fm/blob/main/redevelop/data/samplers/__init__.py"
    click Transforms "https://github.com/ml4bio/rna-fm/blob/main/redevelop/data/transforms/cla_transforms.py"
    click ModelDef "https://github.com/ml4bio/rna-fm/blob/main/fm/model.py"
    click ESM1 "https://github.com/ml4bio/rna-fm/blob/main/fm/model/esm1.py"
    click Modules "https://github.com/ml4bio/rna-fm/blob/main/fm/modules.py"
    click AxialAttn "https://github.com/ml4bio/rna-fm/blob/main/fm/axial_attention.py"
    click MultiHead "https://github.com/ml4bio/rna-fm/blob/main/fm/multihead_attention.py"
    click DataUtils "https://github.com/ml4bio/rna-fm/blob/main/fm/data.py"
    click Constants "https://github.com/ml4bio/rna-fm/blob/main/fm/constants.py"
    click Version "https://github.com/ml4bio/rna-fm/blob/main/fm/version.py"
    click DownModule "https://github.com/ml4bio/rna-fm/blob/main/fm/downstream/downstream_module.py"
    click Baseline "https://github.com/ml4bio/rna-fm/blob/main/fm/downstream/baseline.py"
    click PairLinear "https://github.com/ml4bio/rna-fm/blob/main/fm/downstream/pairwise_predictor/linear_dot_product.py"
    click PairConcat "https://github.com/ml4bio/rna-fm/blob/main/fm/downstream/pairwise_predictor/pairwise_concat.py"
    click WeightsInit "https://github.com/ml4bio/rna-fm/blob/main/fm/downstream/weights_init.py"
    click WeightsLoad "https://github.com/ml4bio/rna-fm/blob/main/fm/downstream/weights_load.py"
    click Backbones "https://github.com/ml4bio/rna-fm/blob/main/redevelop/model/backbones.py"
    click RDModule "https://github.com/ml4bio/rna-fm/blob/main/redevelop/model/downstream_module.py"
    click RPairLinear "https://github.com/ml4bio/rna-fm/blob/main/redevelop/model/pairwise_predictor/linear_dot_product.py"
    click RPairConcat "https://github.com/ml4bio/rna-fm/blob/main/redevelop/model/pairwise_predictor/pairwise_concat.py"
    click Engine "https://github.com/ml4bio/rna-fm/blob/main/redevelop/engine/predictor.py"
    click Logger "https://github.com/ml4bio/rna-fm/blob/main/redevelop/utils/logger.py"
    click Checkpoint "https://github.com/ml4bio/rna-fm/blob/main/redevelop/utils/checkpoint.py"
    click TBLogger "https://github.com/ml4bio/rna-fm/blob/main/redevelop/utils/tensorboard_logger.py"
    click Docs "https://github.com/ml4bio/rna-fm/tree/main/docs/"
    click Tutorials "https://github.com/ml4bio/rna-fm/tree/main/tutorials/"
    click Setup "https://github.com/ml4bio/rna-fm/blob/main/setup.py"
    click Setup "https://github.com/ml4bio/rna-fm/blob/main/environment.yml"

    %% Styles
    classDef code fill:#D6EAF8,stroke:#1B4F72,color:#154360
    classDef data fill:#D5F5E3,stroke:#196F3D,color:#145A32
    classDef ext fill:#FAD7A0,stroke:#B9770E,color:#B9770E
```
