How biomed-ai/cellfm works
Project Type and Purpose
• This is a machine-learning research codebase implementing “CellFM” (a single-cell foundation model) using MindSpore (with optional PyTorch) for pretraining and fine-tuning on transcriptomics data.
• It provides data-processing pipelines, core model definitions, training orchestration (single and multi-device), plus tutorials demonstrating downstream tasks (cell annotation, perturbation, batch integration, gene function prediction, etc.).
File-Structure Analysis
• Top-level directories and scripts
• train.py, 1node_train.sh, scheduler_sc100m.sh, worker_sc100m.sh: orchestration of training jobs (single vs. multi-device).
• config.py: central hyperparameter and path configurations.
• data_process.py, get_gene_emb.py: data cleaning, feature extraction.
• model.py, lora.py: model architecture (CellFM) and LoRA adapter.
• loss_function.py, metrics.py, earlystop.py, retention.py: training utilities (loss, performance metrics, early stopping, checkpoint retention).
• csv/: raw data files (gene info, mappings).
• tutorials/: example notebooks and sub-packages for downstream tasks.
• utils.py: shared utility functions.
• Configuration files: none external except optional YAMLs in tutorials.
• Deployment scripts: shell scripts for single-node multi-device training.
README Insights
• Describes installation (MindSpore 2.2.10, scanpy, scib; optional gears and torch).
• Workflow: preprocess h5/h5ad → train.py (single device) or 1node_train.sh (8 devices) → tutorials.
• External dataset source: Zenodo and HuggingFace.
Main Components and Interactions
a. Data Ingestion & Preprocessing
• Sources: CSV files, H5/H5AD datasets (Zenodo).
• Modules: data_process.py, get_gene_emb.py.
b. Model Definition
• Core: model.py implements transformer-based CellFM.
• Adapters: lora.py to inject LoRA low-rank modules.
c. Training Engine
• Orchestrator: train.py loads config, dataset, model, optimizer, scheduler.
• Utilities: loss_function.py, metrics.py, earlystop.py, retention.py govern training loop.
• Multi-device: 1node_train.sh launches scheduler_sc100m.sh + worker_sc100m.sh across NPUs.
d. Evaluation & Checkpointing
• Evaluation: metrics.py plus tutorial scripts.
• Checkpoint retention: retention.py enforces best-model retention policy.
e. Tutorials & Extensions
• Each tutorial folder contains task-specific code, config YAMLs, and setups for downstream tasks.
Architectural Patterns
• Modular ML pipeline: separation of data, model, training logic, and utilities.
• Adapter pattern: LoRA code injected at runtime for parameter-efficient fine-tuning.
• Script-based orchestration: shell scripts manage distributed resource scheduling.
• Extensible tutorial ecosystem: independent sub-packages illustrating use-cases.
Final Advice
• Be exhaustive: include every directory involved in model training and extension.
• Use layering: place core pipeline at center, orchestration on one side, tutorials on the other, and external services at the periphery.