How ml4bio/rna-fm works
Project Type and Purpose
• This is a Python-based machine-learning library and CLI tool (“RNA-FM”) providing a foundation transformer model for RNA sequences (and an mRNA variant), plus a downstream pipeline for embedding extraction, secondary-structure prediction, and extensibility into tertiary structure or inverse-folding modules.
• The README shows it is primarily a library + CLI interface rather than a web app; it integrates with external Hugging Face storage for model weights and can be used via Python API or shell commands.
File Structure Analysis
Top-level directories and files:
• fm/ – Core Python package containing model definitions (BERT-style Transformer), attention modules, pretrained loader, data utilities, and downstream predictor modules.
• redevelop/ – Extended codebase for CLI workflows:
– config/ – default YAML configurations (model, data, training/prediction settings)
– data/ – batch converters, datasets, samplers, transforms for FASTA and PDB inputs
– engine/ – “predictor.py” orchestrates the model inference loop
– launch/ – entrypoint scripts (e.g., launch/predict.py) that parse arguments, load config, invoke engine
– model/ – mirror of fm/ modules for CLI usage (baselines, backbones, weight init/load)
– pretrained/ – experiment and prediction YAML files under redevelop/pretrained
– utils/ – logging, checkpointing, tensorboard integration, plotting
• setup.py + environment.yml – packaging and dependency specifications (PyTorch, Hydra/OmegaConf, etc.)
• docs/ – MkDocs or Jekyll-based documentation and tutorials
• tutorials/ – Jupyter notebooks and scripts for common use cases (secondary structure, UTR prediction, clustering)
README Insights
• Describes two models (RNA-FM and mRNA-FM), downstream ecosystem modules (RhoFold, RiboDiffusion, RhoDesign) that are separate GitHub projects but connect conceptually.
• Shows typical CLI commands that load a YAML config, read FASTA, run “predictor.py,” and write embeddings, .ct files, .npy matrices.
• Demonstrates Python-API usage via fm.pretrained.rna_fm_t12().
a. Main Components
• User Interfaces:
– CLI script (launch/predict.py)
– Python API (fm.pretrained.* calls)
• Core Library:
– fm package (model definitions, tokenizers, pretrained loader)
– redevelop package (config, data pipeline, engine, model wrappers)
• Configuration: YAML files under redevelop/config and redevelop/pretrained
• Data Layer: FASTA/PDB input → batch conversion → dataset/transforms → DataLoader
• Model Layer: Transformer backbone → downstream predictor heads (secondary structure, pairwise, UTR)
• Persistence & Logging: checkpoint manager, tensorboard, results folder (.npy, .ct, embeddings)
• External Dependencies:
– Pretrained weight storage (Hugging Face, local ./pretrained)
– Python packages (PyTorch, Hydra/OmegaConf, numpy)
– Optional online servers for web-based inference
b. Relationships and Data Flow
1. User invokes CLI or API
2. Config loader reads YAML → builds data pipeline and model
3. Data pipeline reads FASTA via batch_converter → produces token tensors
4. Model loader fetches .pth weights from pretrained folder or remote
5. Forward pass through fm Transformer → representations → downstream modules
6. Engine collects outputs → saves embeddings/structures to disk → logs metrics
7. For web deployment, same engine exposes REST endpoints (external)
c. Architectural Patterns & Principles
• Strict separation of concerns: fm (pure model code) vs. redevelop (pipeline & CLI)
• Configuration-driven design via YAML/Hydra
• Modular data loaders, transforms, and samplers that can be swapped for different tasks
• Reuse of pretrained weights via centralized loader in fm.pretrained
• Extensibility: new predictor heads plug into downstream_module APIs
d. Key Technologies & Frameworks
• Python 3.8+
• PyTorch for model definitions and inference
• Hydra OR OmegaConf for config management
• YAML for experimental settings
• Jupyter for tutorials
• MkDocs/Jekyll under docs for documentation site