How openvax/mhcflurry works
Project Type and Purpose
• The README clearly states that the project implements pan-MHC I ligand prediction and provides both a Python library and command‐line interfaces.
• The purpose is twofold—predicting peptide/MHC binding affinity (using neural networks implemented in TensorFlow), and managing the underlying training data/model generation pipelines (evident from the large downloads-generation folder structure).
Main Components
a. Core Prediction Engine:
– Represent the “mhcflurry” module (with files like allele_encoding.py, class1_affinity_predictor.py, class1_neural_network.py, etc.) as the heart of the system. This is where the neural network models (using TensorFlow) are defined and invoked.
– Indicate subcomponents such as:
• Allele encoding (mhcflurry/allele_encoding.py, amino_acid.py)
• Predictor modules for different tasks (affinity, presentation, and processing predictors)
• Custom loss functions, hyperparameter management, and supporting utilities.
b. Data and Model Generation Pipelines:
– Illustrate the “downloads-generation” directory as a suite of pipelines. This includes:
• Pipelines for generating model data, curated datasets, evaluation datasets, and variations (e.g., data_curated, data_evaluation, data_predictions).
• Scripts for generating models using HPC clusters (e.g., files with “GENERATE.WITH_HPC_CLUSTER.sh” and various cluster_submit_script_header files for GPU and non-GPU runs).
c. Interfaces and Utilities:
– Command Line Interfaces: Nodes for command-line tools (mhcflurry-downloads, mhcflurry-predict, mhcflurry-predict-scan, etc.) that act as entry-points for users.
– Python Library Access: Usage from Jupyter notebooks (see the notebooks folder) and integration within Python applications.
– Docker Integration: A separate component representing the Docker container, which encapsulates the environment (including a Jupyter server) for end-user testing and demonstration.
d. Supporting Components:
– Documentation (docs folder), Continuous Integration (GitHub workflows, .github folder), and testing (test folder) should be represented to illustrate the project governance, quality assurance, and user guidance layers.
– External dependencies such as TensorFlow and any HPC integration utilities (as seen in the model generation scripts) should be shown.
Relationships and Interactions
• That data flows into the model training pipelines (from the downloads-generation pipelines) which in turn update or generate models used by the core engine.
• Communication between various pipelines (for example, how curated data and benchmarking datasets are produced and then fed into model training and evaluation processes).
• A link between the Docker container and the development/testing environments (for replicable deployment).
Architectural Patterns and Technologies
• Modular Design: The project is clearly divided into modules (core logic, data/model generation pipelines, command-line tooling), hinting at a separation of concerns.
• Pipeline/Batch Processing: The downloads-generation folder shows a strong emphasis on data pipelines that are likely run as batch jobs on HPC clusters.
• Integration with TensorFlow: Clearly indicate that model training and prediction leverage TensorFlow as the neural network backend.
• CI/CD: Represent automated testing and deployment pipelines (via GitHub workflows).