How csb5/lofreq works
Project Type and Purpose
• LoFreq is a specialized bioinformatics tool designed to perform ultra-sensitive variant calling by taking into account sequence quality.
• It is implemented in C (with some Python support for auxiliary scripts and pipelines) and is built using autotools. It relies on system dependencies like HTSlib, zlib, and third-party libraries (e.g., cdflib90 and uthash).
File Structure
• Top-level files such as Makefile.am, configure.ac, bootstrap, and various autotools files signal that the project uses the autotools build system.
• The “src/cdflib90” subdirectory contains code for statistical calculations, indicating that precision statistics are crucial for variant calling.
• "src/uthash" provides data structures used by the system (likely for in-memory hash tables).
• There is also a “src/scripts” directory (and additional Python tool directories under “src/tools” and “snakemake”) that complement the core C engine by providing pipelining, pre-/post-processing scripts, and auxiliary command-line tools (e.g., quality filtering, parallel variant calling, somatic analysis).
• The “snakemake” directory shows that a pipeline-oriented approach is supported, which ties together the components for reproducible NGS data analyses.
• The “tests” directory is rich with scripts for validation and performance checking, further clarifying how components need to be tested and verified.
Main Components
a. Build & Configuration:
• A block representing the autotools build system (bootstrap, configure, make, make install) with dependencies on the C compiler, Python 3 interpreter, zlib, and HTSlib.
• External libraries used (HTSlib, cdflib90, uthash).
b. Core Variant Calling Engine:
• Modules within “src/lofreq”:
• Input processing (BAM file handling, reference checking: e.g., bam_index, bam_md_ext, bedidx).
• Statistical modules for computing probabilities (binom, fet, multtest, etc.).
• Core variant calling routines (lofreq_call, lofreq_filter, lofreq_index, etc.).
• Supporting utilities (e.g., log, utils, viterbi) for quality scoring and data validation.
c. Auxiliary and Pipeline Components:
• Python scripts and tools in “src/scripts” and “src/tools” (including lofreq_star) that act as wrappers or specialized variant calling workflows (such as local and somatic call pipelines).
• The “snakemake” pipeline that orchestrates these components and integrates data pre-/post-processing.
• You may represent these as separate modules connected to the core engine, illustrating how they call the core executable as part of a workflow.
d. Testing & Validation:
• The “tests” directory should be represented to indicate integration and unit test components. This shows quality assurance practices integrated into the architecture.
Architectural Patterns and Relationships
• A modular design: the core engine is built as independent modules (each function implemented across different C source files) that interact with each other through well-defined function calls.
• Layered dependencies: at the lowest level are external libraries (HTSlib, cdflib90, uthash), above which sits the variant calling logic, and at the top are the pipeline and user interface scripts.
• A central “Variant Calling Engine” split into:
• Input/Parsing Modules (BAM processing, reference checks),
• Statistical Modules (binom, fet, multiple testing),
• Core Calling Algorithms (lofreq_call, lofreq_filter, viterbi),
• Utility Modules (logging, quality scoring).