How elanapearl/interplm works
Main Components
• Data Acquisition & Preprocessing
• Ingestion of raw protein sequence data (FASTA files, UniProtKB downloads).
• Data processing modules (e.g., “data_processing” with files like shard_fasta.py and subset_fasta.py) that filter sequences and create shards.
• Embedding Generation
• “esm” module which takes processed FASTA files and converts them into embeddings (fasta_to_sae_dataset.py, embed.py, embed_uniprotkb.py).
• That these embeddings come from specific protein language models (ESM-2 variants).
• Sparse Autoencoder (SAE) Training & Inference
• “sae” module where the model training happens (train_plm_sae.py, trainer.py in the “train” directory).
• Inference components in the “sae/inference.py” showing how pretrained SAEs are loaded from HuggingFace.
• Concept Extraction & Evaluation
• “concept” module with functions to extract UniProtKB annotations, compare activations to concepts, calculate F1 metrics, and report final metrics.
• Flow from extracted data (using extract_uniprotkb_annotations.py) through concept evaluation pipelines.
• Feature Visualization & Dashboard
• Deploy the “feature_vis” and “dashboard” components (e.g., feature_activation_vis.py, start_dashboard_cache.py, dashboard/app.py).
• Interactive dashboard (built using Streamlit) that visualizes activation patterns and concept associations.
• Utilities & Configuration
• Mention configuration files (env.yml, .gitignore, config.toml in dashboard), constants, and helper utilities that support the pipeline.
• From raw data (UniProt FASTA downloads) to “data_processing” for sharding/subsetting.
• Processed sequence data flows into the “esm” module for embedding generation.
• Embeddings serve as the input for SAE training in the “train” and “sae” modules.
• The trained SAE models feed into the “concept” module for feature evaluation alongside annotation extraction.
• The outputs from both training (model checkpoints, normalized SAE) and concept evaluation (F1 scores, metrics directories) are used to populate the “dashboard” cache.
• Finally, the “dashboard” module consumes the cached metrics to provide an interactive visualization.
Architectural Patterns & Design Principles
• Modular Design:
• The project is clearly modularized into distinct directories (esm, sae, train, dashboard, etc.), representing separation of concerns.
• Pipeline Architecture:
• Sequential nature of the data processing pipeline – each module processing the outputs of the previous step.
• Reusability & Extensibility:
• How the system can easily be extended (e.g., adapting to other PLMs’ embeddings beyond ESM).
• Clear Data Flow & Dependency:
• How utilities, constants, and configuration files integrate across components.
Relevant Technologies & Frameworks
• Python as the programming language orchestrating the toolkit.
• Conda environment (env.yml), indicating dependency management.
• Streamlit for the dashboard UI.
• HuggingFace for pretrained model hosting.
• ESM-2 models for PLM embeddings.
• SAE algorithms for feature extraction.
• Be as detailed as possible:
• Sub-components within major modules (for instance, within “concept,” show separate nodes for “extract_uniprotkb_annotations.py,” “compare_activations_to_concepts.py,” “calculate_f1.py,” and “report_final_metrics.py”).
• Configuration interactions (e.g., environment variables like INTERPLM_DATA).