How evotec-isrd/gnn-morphmap works
Main Components
a. Data Input & External Dependencies:
• Data Folder: Contains essential input data files (e.g., drkg.tsv, GOannot.tsv, pathways.tsv, orf and crispr scores, similarity CSVs).
• External Data & Models: Pre-trained models and data objects (in trained_models) as well as external data sources (links to Google Drive and Zenodo) informing the experiments.
b. Configuration Management:
• Configurations Folder: Holds the configuration files (YAML files) which define the various prediction tasks (gene_bp, gene_cc, gene_mf, gene_pathway, etc.). This acts as the project’s “parameter store,” directing how the prediction pipelines should operate.
c. Prediction & Processing Pipelines:
• make_predictions.py: The entry-point for running predictions using the defined tasks.
• transform_predictions.py: Handles the post-processing of prediction outputs.
• compute_scores.py: Computes performance or similarity scores (for ORF and CRISPR data).
d. Evaluation & Visualization:
• Notebooks (Heatmap.ipynb, ModelAccuracy.ipynb, Scatterplots.ipynb, SLCOR.ipynb): Provide interactive explorations and visual representations of the predictions, model accuracies, and biological relationships.
e. Environment & Build Configuration:
• environment.yml, pyproject.toml, and run_morphmap_gnn.sh: These files manage the software environment, build configurations, and ease the deployment/setup process.
Relationships and Interactions
• The Data Input (data/) feeds into the prediction pipeline, with configurations (configurations/) guiding which prediction task or model type to run.
• make_predictions.py uses the configuration details to trigger predictions, generating outputs that are then transformed by transform_predictions.py.
• compute_scores.py takes these prediction outputs and computes aggregate or merged scores that are stored back into the data folder.
• The pre-trained models in trained_models are loaded (with accompanying data objects) during these processes.
• Finally, notebooks pull both the prediction results and scores to evaluate model performance and create visual plots.
• The environment management files ensure that all libraries and dependencies are consistent across these components.
Architectural Patterns and Design Principles
• Modular Design: Each module (Data, Configuration, Prediction, Transformation, Scoring, Visualization) is distinct and can be updated or replaced independently.
• Pipeline Pattern: Data flows sequentially from raw data ingestion, through prediction and processing, to final evaluation. This chain is driven by command-line scripts.
• Configuration-Driven Execution: The use of YAML configuration files indicates that behavior can be modified without code changes, supporting experiment flexibility.
• Reusability & Experimentation: Notebooks allow for interactive validation of the pipeline outputs and further exploratory analysis.
– How prediction tasks are parameterized by YAML configurations.
– The movement of raw data through the pipeline to produce prediction outputs, scores, and visual insights.
– The integration between static scripts and interactive notebooks which are used for downstream analysis.