How kojix2/ruby-htslib works
Project Type and Purpose
• This project is a language binding library – a “glue” layer between Ruby and the underlying C library (HTSlib).
• Its purpose is to offer Ruby developers a high-level API (in modules like HTS::Bam, HTS::Bcf, etc.) to interact with common bioinformatics file formats by internally calling native C functions via Ruby-FFI.
File Structure
• Top-level Directories & Files:
• “lib/hts”: Contains the core Ruby modules and classes for different genomic file formats. Notice subdirectories for BAM (including auxiliary files, header, flag, record, etc.), BCF, FAIDX, and Tabix; this indicates modularization according to each file format.
• “lib/hts/ffi_ext”: A dedicated directory for low-level FFI extensions, dealing with pointers, structs, and integration with the native library.
• “lib/hts/libhts”: Contains bindings to the native HTSlib functions, constants, and thread pool — essentially the low-level API enabling direct communication with HTSlib.
• “examples” and “test”: Provide scripts and tests demonstrating how the API is used (e.g., reading/writing genomic files) and ensuring correctness.
• “scripts”: Contains tools (such as c2ffi, func2rb.rb, struct2rb.rb) to generate or validate FFI bindings and to assist in keeping the Ruby-side implementations in sync with HTSlib changes.
• Other items like Dockerfile, GitHub workflows, and build configuration files indicate continuous integration and packaging processes.
Components
a. High-Level Ruby API Layer:
• Components such as HTS::Bam, HTS::Bcf, HTS::Faidx, and HTS::Tabix that provide an object-oriented interface for genomic data.
• These classes contain methods that perform tasks like opening files, iterating over records, and providing a developer-friendly interface.
b. Low-Level FFI Binding Layer:
• The LibHTS module in “lib/hts/libhts” that directly maps to HTSlib functions using Ruby-FFI.
• Any helper modules in “lib/hts/ffi_ext” that manage memory and pointer structures.
• Re-implementation of C macros into Ruby (as referenced in the README) to bypass Ruby-FFI limitations.
c. Native HTSlib Component:
• Although not a separate submodule in this repository, HTSlib (the underlying C library) is a key external dependency. Indicate that calls from LibHTS interface directly with the native library.
d. Supporting Tools and Development Infrastructure:
• The “scripts” directory providing tooling for converting C headers to Ruby ffi bindings (c2ffi, etc.).
• The “test” and “examples” directories which, while not part of production execution, illustrate how the API components interoperate and are validated.
• Build and continuous integration files (Dockerfile, GitHub workflows) that show how code is compiled, tests are run, and how integration with HTSlib is maintained.
Relationships and Interactions
• A Ruby application (or test/example script) calls a high-level API method (e.g., HTS::Bam.open).
• The high-level API delegates functionality to lower-level helper methods, which, via Ruby-FFI, invoke the corresponding C functions wrapped in the LibHTS module.
• These calls communicate with HTSlib (native C library) to perform the required operations (file reading, writing, etc.).
• Data (like BAM records or VCF fields) returned from HTSlib are processed via FFI bindings and then re-packaged into Ruby objects.
• Memory management should be highlighted, particularly where ManagedStruct and FFI::Struct are used to automatically (or sometimes manually) free memory.
Architectural Patterns and Design Principles
• Layered Architecture: Separation between the high-level Ruby API and the low-level FFI bindings.
• Modularity: Each file format (BAM, BCF, etc.) is implemented in its own set of modules, making it easier to extend or modify features independently.
• Interoperability: Emphasis on bridging Ruby and C through FFI. The reimplementation of C macros in Ruby and memory management differences (GC on Ruby vs. manual release) are critical details.
• Testing and Continuous Integration: Emphasize that tests, examples, and CI workflows ensure integration integrity.