How kad-ecoli/blastknn_benchmark works
Project type and purpose
• This is an open-source benchmarking pipeline (“blastKNN_benchmark”) for large-scale assessment of various sequence database search tools (BLAST, Diamond, MMseqs2, HHSuite, JACKHMMER, PSIBLAST, PHMMER) aimed at homology-based protein function prediction.
• There is no frontend/backend web layer; it’s a command-line, data-processing and evaluation framework built from bash wrappers, Python scripts, and C++ code, orchestrating external bioinformatics engines.
File structure
• Top-level areas:
• curated/ – preprocessed ground-truth data (ontologies, sequences, labels)
• raw/ontology – original GO OBO file
• script/ – all orchestration code, wrappers, parsers, tuners, plotters, and third-party tool subrepos (hhsuite2, mmseqs)
• Patterns:
• Download & prep scripts under script/ (download_*.sh, curate_GAF, cdhit_src)
• Runner scripts (run_blastp.sh … run_phmmer.sh) invoking each tool and saving outputs
• Tuning scripts (tune_*.sh, tune_*.py)
• Plotting scripts (plot_*.py)
• Third-party code trees in script/hhsuite2 and script/mmseqs
Key README insights
• Three stages:
a. Download data
b. Run function prediction (invoke each tool & tune parameters)
c. Plot results (compare performance)
• Ground-truth labels in curated/sequence/*.is_a
a. Main components
1. Data Layer
• Raw GO ontology (raw/ontology/go-basic.obo)
• Raw sequences & validation sets (download via scripts)
• Curated sequences & ground truth labels (curated/sequence, curated/ontology)
2. Preprocessing & Curation
• Scripts that parse OBO, build GO term tables (obo2csv), cluster sequences (cdhit_src), validate formats
3. Execution Orchestrator
• Bash wrappers (run_*.sh) driving each search engine
• Parameter-tuner scripts (tune_*.sh / tune_*.py)
4. External Search Engines
• BLAST+ (blastp, psiblast), Diamond, MMseqs2, HHSuite2, JACKHMMER, PHMMER
• Each as a separate submodule or binary invoked by scripts
5. Results Processing
• Parsers (blast2tsv, diamond2tsv, ghostx2tsv, hhsuite2tsv)
• Combine & reformat scripts (combineFPC.py, fasta2tsv.cpp)
6. Evaluation & Visualization
• assess_result.py compares predicted .is_a with ground truth
• plot_*.py generates performance curves
b. Data flows and interactions
• Orchestrator → External Engines (bidirectional: commands & results files)
• Engines → Results Processing → Evaluation
• Evaluation → Visualization → Output plots
c. Architectural patterns & principles
• Pipeline pattern: linear data flow through discrete stages
• Modular tool‐wrapper architecture: each search engine is encapsulated
• Extensibility point: add new tool by dropping in a run_<tool>.sh + parser
d. Technologies & frameworks
• Bash scripting for orchestration
• Python for data parsing, tuning, plotting
• C++ modules for performance-critical tasks & third-party tools
• CSV/TSV, FASTA formats, OBO ontology parsing