How odelaneau/shapeit4 works
1. Repository purpose and top-level shape
- This is SHAPEIT4, a C++ command-line tool for haplotype estimation / phasing and related imputation-style graph construction.
- The core production code lives under src/, with a separate auxiliary tool under tools/bingraphsample/.
- test/ contains real reference/scaffold/unphased VCF/BCF fixtures and a recombination map, which makes the pipeline easy to validate end-to-end.
2. Primary runtime flow
- The main executable starts in src/main.cpp.
- Execution then flows into the phasing pipeline in src/phaser/, which coordinates initialization, iterative phasing logic, parameter handling, and final output.
- The phaser consumes genotype inputs, optional scaffold information, and genetic map data, then produces phased haplotypes.
3. Core orchestration layer
- src/modules/builder.* and src/modules/pbwt_solver.* are the central algorithmic modules.
- The builder likely assembles the per-region/per-individual state needed for phasing, while the PBWT solver selects informative conditioning haplotypes using the Positional Burrows-Wheeler Transform.
- This layer sits between raw input data and the HMM/phasing model, translating data structures into algorithm-ready working sets.
4. Domain objects and phasing state
- src/objects/ holds the core domain model: variants, genotype processing, and HMM parameters.
- src/objects/genotype/ contains genotype lifecycle code: build, management, pruning, masking, and sweeping.
- src/objects/hmm_parameters.* and src/objects/variant.* represent the statistical model inputs and per-variant metadata used throughout the phasing process.
- src/objects/compute_job.* suggests the work is partitioned into jobs, likely by genomic segment or chunk.
5. Data containers and in-memory representation
- src/containers/ provides the low-level in-memory structures used by the algorithm:
- bitmatrix.* for compact binary genotype/haplotype state
- genotype_set.* and haplotype_set.* for collections of sample states
- variant_map.* for mapping genomic positions/variants to internal indices
- These containers are the main boundary between file I/O and the phasing logic, and they likely drive most performance-sensitive operations.
6. I/O boundary
- src/io/ is the external data boundary.
- genotype_reader.*, gmap_reader.*, graph_writer.*, and haplotype_writer.* handle input genotype/scaffold/map data and output phased haplotypes or graph-like representations.
- The README notes HTSlib-backed VCF/BCF support; the repository also includes gzipped genetic maps in maps/ and test/, so compressed genomics formats are a first-class concern.
7. Phasing model implementation
- src/phaser/ contains the algorithm’s control logic:
- initialization
- iterative phasing/updates
- management of phasing state
- finalization/output
- parameter handling
- This layer is the runtime “brain” that coordinates the genotype graph, PBWT conditioning set selection, and HMM-based inference.
8. Statistical and segment models
- src/models/ contains haplotype segment types for single and double representation.
- These model classes are likely used to represent phased blocks or candidate haplotype paths during inference.
- Combined with src/objects/hmm_parameters.*, they form the probabilistic core of the phasing engine.
9. Utility layer
- src/utils/ provides shared infrastructure: string helpers, timing, verbosity/logging, random numbers, compressed I/O, and generic algorithms/statistics.
- This layer is intentionally non-domain-specific and supports both performance and diagnostics across the codebase.
10. Auxiliary tool: bingraphsample
- tools/bingraphsample/ is a separate build target with its own source tree and object files.
- It reuses much of the same conceptual model—genotype sets, variant maps, graph reading, haplotype writing—but is packaged as a sampling utility rather than the main phasing application.
- Architecturally, it looks like a smaller companion program for reading binary graph-like data and emitting haplotypes.
11. Inputs, outputs, and artifacts
- Input data includes unphased genotypes, reference/scaffold panels, and genetic maps.
- Output is phased haplotypes, with graph-oriented outputs also supported.
- maps/ holds genetic map archives for b37/b38 builds, while test/ provides full examples for both compressed VCF and BCF workflows.
12. Build and documentation layout
- The project uses a makefile-based build, with separate bin/ and obj/ directories for compiled artifacts.
- docs/ contains a static website/documentation bundle, separate from the executable code.
- The repository is therefore organized as a classic high-performance C++ genomics application: thin CLI entrypoint, algorithmic core, explicit data/model separation, and external-format I/O at the edges.
Main entry points: phaser_algorithm.cpp Phaser · sampler_management.cpp Sampler · builder.cpp Builder · genotype_build.cpp Genotype ops
Components 27
Core application
Main phasing engine and shared runtime code
- CLI entry · Program startup and command-line dispatch
src/main.cpp - Phaser · Coordinates initialization, iterative phasing, and final output
src/phaser/phaser_algorithm.cpp - Init · Sets up phasing inputs and runtime state
src/phaser/phaser_initialise.cpp - Manage · Handles phasing control flow and state transitions
src/phaser/phaser_management.cpp - Finalize · Completes phasing and emits final results
src/phaser/phaser_finalise.cpp - Builder · Assembles region-level working state for inference
src/modules/builder.cpp - PBWT solver · Selects informative conditioning haplotypes
src/modules/pbwt_solver.cpp - Genotype ops · Builds, manages, masks, prunes, and sweeps genotype state
src/objects/genotype/genotype_build.cpp - HMM params · Statistical parameters for the phasing model
src/objects/hmm_parameters.cpp - Variants · Variant metadata and genotype-aware processing
src/objects/variant.cpp - Compute jobs · Chunked units of phasing work across genomic regions
src/objects/compute_job.cpp - Bit matrix · Compact binary representation of genotype and haplotype state
src/containers/bitmatrix.cpp - Sample sets · Collections of genotype and haplotype samples
src/containers/genotype_set.cpp - Variant map · Maps variants and positions to internal indices
src/containers/variant_map.cpp - Input readers · Reads genotype and genetic map data
src/io/genotype_reader1.cpp - Map reader · Loads recombination maps
src/io/gmap_reader.cpp - Haplotype writer · Writes phased haplotypes
src/io/haplotype_writer.cpp - Graph writer · Emits graph-like phasing output
src/io/graph_writer.cpp - Haplotype models · Single and double haplotype segment representations
src/models/haplotype_segment_single.cpp - Utilities · Shared helpers for strings, timing, randomness, stats, and compressed I/O
src/utils/string_utils.h
Auxiliary tool
Standalone sampling utility under tools/bingraphsample
- Sampler CLI · Entry point for the binary graph sampling tool
tools/bingraphsample/src/main.cpp - Sampler · Initializes, manages, and finalizes sampling runs
tools/bingraphsample/src/sampler/sampler_management.cpp - Graph reader · Reads binary graph-like input for the sampler
tools/bingraphsample/src/io/graph_reader.cpp - Sampler I/O · Writes sampled haplotypes and handles sample-side data flow
tools/bingraphsample/src/io/haplotype_writer.cpp
Data and docs
Test fixtures, genetic maps, and documentation
- Fixtures · Reference, scaffold, unphased, and map test datasets
test/reference.vcf.gz - Genetic maps · Packaged recombination maps for supported builds
maps/genetic_maps.b37.tar.gz - Docs · Static documentation site
docs/index.html
Connections 31
- CLI entry → Phaser: starts · Main executable dispatches into phasing pipeline
- Phaser → Init: initializes · Sets up inputs and runtime state
- Phaser → Manage: coordinates · Manages iterative phasing control flow
- Phaser → Finalize: finalizes · Completes phasing and writes results
- Phaser → Input readers: loads · Consumes genotype inputs
- Phaser → Map reader: loads · Consumes recombination map data
- Phaser → Builder: prepares · Builds inference state for each region or job
- Phaser → PBWT solver: selects · Uses PBWT to choose conditioning haplotypes
- Phaser → Genotype ops: updates · Manages genotype masking, pruning, and sweep operations
- Phaser → HMM params: parameterizes · Feeds the probabilistic model
- Phaser → Haplotype models: tracks · Represents phased segments during inference
- Phaser → Haplotype writer: emits · Writes final phased haplotypes
- Phaser → Graph writer: emits · Optionally writes graph-oriented output
- Builder → Compute jobs: splits · Partitions work into compute jobs
- Builder → Bit matrix: uses · Builds compact working matrices
- Builder → Sample sets: assembles · Organizes sample-level genotype and haplotype collections
- Builder → Variant map: indexes · Maps genomic positions and variants
- PBWT solver → Sample sets: queries · Searches conditioning haplotypes in sample sets
- Genotype ops → Bit matrix: encodes · Stores genotype state compactly
- Genotype ops → Variants: annotates · Attaches variant-aware genotype processing
- Input readers → Fixtures: consumes · Reads the reference and scaffold fixtures during validation
- Map reader → Genetic maps: consumes · Loads packaged recombination maps
- Haplotype writer → Fixtures: verified by · Outputs are validated against packaged fixtures
- Sampler CLI → Sampler: starts · Sampler executable dispatches into its control layer
- Sampler → Graph reader: loads · Reads binary graph input
- Sampler → Sampler I/O: writes · Produces sampled haplotypes
- Sampler → Sample sets: reuses · Shares genotype-set style sample containers
- Sampler → Variant map: reuses · Shares variant indexing logic
- Phaser → Utilities: supports · Uses shared helpers across the pipeline
- Sampler → Utilities: supports · Uses shared helpers across the sampler
- Docs → CLI entry: describes · Documentation explains the main CLI and workflow