# odelaneau/shapeit4 architecture

> Architecture diagram of the GitHub repository odelaneau/shapeit4, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/odelaneau/shapeit4
- Repository: https://github.com/odelaneau/shapeit4

## Overview

1. **Repository purpose and top-level shape**
   - This is **SHAPEIT4**, a C++ command-line tool for **haplotype estimation / phasing** and related **imputation-style graph construction**.
   - The core production code lives under `src/`, with a separate auxiliary tool under `tools/bingraphsample/`.
   - `test/` contains real reference/scaffold/unphased VCF/BCF fixtures and a recombination map, which makes the pipeline easy to validate end-to-end.

2. **Primary runtime flow**
   - The main executable starts in `src/main.cpp`.
   - Execution then flows into the phasing pipeline in `src/phaser/`, which coordinates initialization, iterative phasing logic, parameter handling, and final output.
   - The phaser consumes genotype inputs, optional scaffold information, and genetic map data, then produces phased haplotypes.

3. **Core orchestration layer**
   - `src/modules/builder.*` and `src/modules/pbwt_solver.*` are the central algorithmic modules.
   - The builder likely assembles the per-region/per-individual state needed for phasing, while the PBWT solver selects informative conditioning haplotypes using the Positional Burrows-Wheeler Transform.
   - This layer sits between raw input data and the HMM/phasing model, translating data structures into algorithm-ready working sets.

4. **Domain objects and phasing state**
   - `src/objects/` holds the core domain model: variants, genotype processing, and HMM parameters.
   - `src/objects/genotype/` contains genotype lifecycle code: build, management, pruning, masking, and sweeping.
   - `src/objects/hmm_parameters.*` and `src/objects/variant.*` represent the statistical model inputs and per-variant metadata used throughout the phasing process.
   - `src/objects/compute_job.*` suggests the work is partitioned into jobs, likely by genomic segment or chunk.

5. **Data containers and in-memory representation**
   - `src/containers/` provides the low-level in-memory structures used by the algorithm:
     - `bitmatrix.*` for compact binary genotype/haplotype state
     - `genotype_set.*` and `haplotype_set.*` for collections of sample states
     - `variant_map.*` for mapping genomic positions/variants to internal indices
   - These containers are the main boundary between file I/O and the phasing logic, and they likely drive most performance-sensitive operations.

6. **I/O boundary**
   - `src/io/` is the external data boundary.
   - `genotype_reader.*`, `gmap_reader.*`, `graph_writer.*`, and `haplotype_writer.*` handle input genotype/scaffold/map data and output phased haplotypes or graph-like representations.
   - The README notes HTSlib-backed VCF/BCF support; the repository also includes gzipped genetic maps in `maps/` and `test/`, so compressed genomics formats are a first-class concern.

7. **Phasing model implementation**
   - `src/phaser/` contains the algorithm’s control logic:
     - initialization
     - iterative phasing/updates
     - management of phasing state
     - finalization/output
     - parameter handling
   - This layer is the runtime “brain” that coordinates the genotype graph, PBWT conditioning set selection, and HMM-based inference.

8. **Statistical and segment models**
   - `src/models/` contains haplotype segment types for single and double representation.
   - These model classes are likely used to represent phased blocks or candidate haplotype paths during inference.
   - Combined with `src/objects/hmm_parameters.*`, they form the probabilistic core of the phasing engine.

9. **Utility layer**
   - `src/utils/` provides shared infrastructure: string helpers, timing, verbosity/logging, random numbers, compressed I/O, and generic algorithms/statistics.
   - This layer is intentionally non-domain-specific and supports both performance and diagnostics across the codebase.

10. **Auxiliary tool: `bingraphsample`**
   - `tools/bingraphsample/` is a separate build target with its own source tree and object files.
   - It reuses much of the same conceptual model—genotype sets, variant maps, graph reading, haplotype writing—but is packaged as a sampling utility rather than the main phasing application.
   - Architecturally, it looks like a smaller companion program for reading binary graph-like data and emitting haplotypes.

11. **Inputs, outputs, and artifacts**
   - Input data includes unphased genotypes, reference/scaffold panels, and genetic maps.
   - Output is phased haplotypes, with graph-oriented outputs also supported.
   - `maps/` holds genetic map archives for b37/b38 builds, while `test/` provides full examples for both compressed VCF and BCF workflows.

12. **Build and documentation layout**
   - The project uses a `makefile`-based build, with separate `bin/` and `obj/` directories for compiled artifacts.
   - `docs/` contains a static website/documentation bundle, separate from the executable code.
   - The repository is therefore organized as a classic high-performance C++ genomics application: thin CLI entrypoint, algorithmic core, explicit data/model separation, and external-format I/O at the edges.

Main entry points: [`src/phaser/phaser_algorithm.cpp`](https://github.com/odelaneau/shapeit4/blob/master/src/phaser/phaser_algorithm.cpp) (Phaser), [`tools/bingraphsample/src/sampler/sampler_management.cpp`](https://github.com/odelaneau/shapeit4/blob/master/tools/bingraphsample/src/sampler/sampler_management.cpp) (Sampler), [`src/modules/builder.cpp`](https://github.com/odelaneau/shapeit4/blob/master/src/modules/builder.cpp) (Builder), [`src/objects/genotype/genotype_build.cpp`](https://github.com/odelaneau/shapeit4/blob/master/src/objects/genotype/genotype_build.cpp) (Genotype ops)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_src["Core application"]
  node_n_main["CLI entry<br/>executable<br/>[main.cpp]"]
  node_n_phaser["Phaser<br/>orchestrator"]
  node_n_phaser_init["Init<br/>phaser stage"]
  node_n_phaser_manage["Manage<br/>phaser stage"]
  node_n_phaser_final["Finalize<br/>phaser stage"]
  node_n_builder["Builder<br/>[builder.cpp]"]
  node_n_pbwt["PBWT solver<br/>conditioning<br/>[pbwt_solver.cpp]"]
  node_n_genotype_obj["Genotype ops<br/>domain logic<br/>[genotype_build.cpp]"]
  node_n_hmm["HMM params<br/>model<br/>[hmm_parameters.cpp]"]
  node_n_variant_obj["Variants<br/>domain model<br/>[variant.cpp]"]
  node_n_jobs["Compute jobs<br/>work partition<br/>[compute_job.cpp]"]
  node_n_bitmatrix["Bit matrix<br/>container<br/>[bitmatrix.cpp]"]
  node_n_sets["Sample sets<br/>container<br/>[genotype_set.cpp]"]
  node_n_vmap["Variant map<br/>index<br/>[variant_map.cpp]"]
  node_n_readers["Input readers<br/>io"]
  node_n_gmap["Map reader<br/>io<br/>[gmap_reader.cpp]"]
  node_n_writers["Haplotype writer<br/>io"]
  node_n_graph_writer["Graph writer<br/>io<br/>[graph_writer.cpp]"]
  node_n_models["Haplotype models"]
  node_n_utils["Utilities<br/>support<br/>[string_utils.h]"]
end

subgraph group_g_tools["Auxiliary tool"]
  node_n_sampler_main["Sampler CLI<br/>executable<br/>[main.cpp]"]
  node_n_sampler_core["Sampler<br/>orchestrator"]
  node_n_graph_reader["Graph reader<br/>io<br/>[graph_reader.cpp]"]
  node_n_sampler_io["Sampler I/O<br/>io"]
end

subgraph group_g_assets["Data and docs"]
  node_n_testdata["Fixtures<br/>data<br/>[reference.vcf.gz]"]
  node_n_maps["Genetic maps<br/>data"]
  node_n_docs["Docs<br/>documentation<br/>[index.html]"]
end

node_n_main -->|"starts"| node_n_phaser
node_n_phaser -->|"initializes"| node_n_phaser_init
node_n_phaser -->|"coordinates"| node_n_phaser_manage
node_n_phaser -->|"finalizes"| node_n_phaser_final
node_n_phaser -->|"loads"| node_n_readers
node_n_phaser -->|"loads"| node_n_gmap
node_n_phaser -->|"prepares"| node_n_builder
node_n_phaser -->|"selects"| node_n_pbwt
node_n_phaser -->|"updates"| node_n_genotype_obj
node_n_phaser -->|"parameterizes"| node_n_hmm
node_n_phaser -->|"tracks"| node_n_models
node_n_phaser -->|"emits"| node_n_writers
node_n_phaser -->|"emits"| node_n_graph_writer
node_n_builder -->|"splits"| node_n_jobs
node_n_builder -->|"uses"| node_n_bitmatrix
node_n_builder -->|"assembles"| node_n_sets
node_n_builder -->|"indexes"| node_n_vmap
node_n_pbwt -->|"queries"| node_n_sets
node_n_genotype_obj -->|"encodes"| node_n_bitmatrix
node_n_genotype_obj -->|"annotates"| node_n_variant_obj
node_n_readers -.->|"consumes"| node_n_testdata
node_n_gmap -.->|"consumes"| node_n_maps
node_n_writers -.->|"verified by"| node_n_testdata
node_n_sampler_main -->|"starts"| node_n_sampler_core
node_n_sampler_core -->|"loads"| node_n_graph_reader
node_n_sampler_core -->|"writes"| node_n_sampler_io
node_n_sampler_core -->|"reuses"| node_n_sets
node_n_sampler_core -->|"reuses"| node_n_vmap
node_n_phaser -->|"supports"| node_n_utils
node_n_sampler_core -->|"supports"| node_n_utils
node_n_docs -.->|"describes"| node_n_main

click node_n_main "https://github.com/odelaneau/shapeit4/blob/master/src/main.cpp"
click node_n_phaser "https://github.com/odelaneau/shapeit4/blob/master/src/phaser/phaser_algorithm.cpp"
click node_n_phaser_init "https://github.com/odelaneau/shapeit4/blob/master/src/phaser/phaser_initialise.cpp"
click node_n_phaser_manage "https://github.com/odelaneau/shapeit4/blob/master/src/phaser/phaser_management.cpp"
click node_n_phaser_final "https://github.com/odelaneau/shapeit4/blob/master/src/phaser/phaser_finalise.cpp"
click node_n_builder "https://github.com/odelaneau/shapeit4/blob/master/src/modules/builder.cpp"
click node_n_pbwt "https://github.com/odelaneau/shapeit4/blob/master/src/modules/pbwt_solver.cpp"
click node_n_genotype_obj "https://github.com/odelaneau/shapeit4/blob/master/src/objects/genotype/genotype_build.cpp"
click node_n_hmm "https://github.com/odelaneau/shapeit4/blob/master/src/objects/hmm_parameters.cpp"
click node_n_variant_obj "https://github.com/odelaneau/shapeit4/blob/master/src/objects/variant.cpp"
click node_n_jobs "https://github.com/odelaneau/shapeit4/blob/master/src/objects/compute_job.cpp"
click node_n_bitmatrix "https://github.com/odelaneau/shapeit4/blob/master/src/containers/bitmatrix.cpp"
click node_n_sets "https://github.com/odelaneau/shapeit4/blob/master/src/containers/genotype_set.cpp"
click node_n_vmap "https://github.com/odelaneau/shapeit4/blob/master/src/containers/variant_map.cpp"
click node_n_readers "https://github.com/odelaneau/shapeit4/blob/master/src/io/genotype_reader1.cpp"
click node_n_gmap "https://github.com/odelaneau/shapeit4/blob/master/src/io/gmap_reader.cpp"
click node_n_writers "https://github.com/odelaneau/shapeit4/blob/master/src/io/haplotype_writer.cpp"
click node_n_graph_writer "https://github.com/odelaneau/shapeit4/blob/master/src/io/graph_writer.cpp"
click node_n_models "https://github.com/odelaneau/shapeit4/blob/master/src/models/haplotype_segment_single.cpp"
click node_n_utils "https://github.com/odelaneau/shapeit4/blob/master/src/utils/string_utils.h"
click node_n_sampler_main "https://github.com/odelaneau/shapeit4/blob/master/tools/bingraphsample/src/main.cpp"
click node_n_sampler_core "https://github.com/odelaneau/shapeit4/blob/master/tools/bingraphsample/src/sampler/sampler_management.cpp"
click node_n_graph_reader "https://github.com/odelaneau/shapeit4/blob/master/tools/bingraphsample/src/io/graph_reader.cpp"
click node_n_sampler_io "https://github.com/odelaneau/shapeit4/blob/master/tools/bingraphsample/src/io/haplotype_writer.cpp"
click node_n_testdata "https://github.com/odelaneau/shapeit4/blob/master/test/reference.vcf.gz"
click node_n_maps "https://github.com/odelaneau/shapeit4/blob/master/maps/genetic_maps.b37.tar.gz"
click node_n_docs "https://github.com/odelaneau/shapeit4/blob/master/docs/index.html"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_main,node_n_phaser,node_n_phaser_init,node_n_phaser_manage,node_n_phaser_final,node_n_builder,node_n_pbwt,node_n_genotype_obj,node_n_hmm,node_n_variant_obj,node_n_jobs,node_n_bitmatrix,node_n_sets,node_n_vmap,node_n_readers,node_n_gmap,node_n_writers,node_n_graph_writer,node_n_models,node_n_utils toneBlue
class node_n_sampler_main,node_n_sampler_core,node_n_graph_reader,node_n_sampler_io toneAmber
class node_n_testdata,node_n_maps,node_n_docs toneMint
```

## Components

### Core application

Main phasing engine and shared runtime code

- [CLI entry](https://github.com/odelaneau/shapeit4/blob/master/src/main.cpp): executable · Program startup and command-line dispatch · `src/main.cpp`
- [Phaser](https://github.com/odelaneau/shapeit4/blob/master/src/phaser/phaser_algorithm.cpp): orchestrator · Coordinates initialization, iterative phasing, and final output · `src/phaser/phaser_algorithm.cpp`
- [Init](https://github.com/odelaneau/shapeit4/blob/master/src/phaser/phaser_initialise.cpp): phaser stage · Sets up phasing inputs and runtime state · `src/phaser/phaser_initialise.cpp`
- [Manage](https://github.com/odelaneau/shapeit4/blob/master/src/phaser/phaser_management.cpp): phaser stage · Handles phasing control flow and state transitions · `src/phaser/phaser_management.cpp`
- [Finalize](https://github.com/odelaneau/shapeit4/blob/master/src/phaser/phaser_finalise.cpp): phaser stage · Completes phasing and emits final results · `src/phaser/phaser_finalise.cpp`
- [Builder](https://github.com/odelaneau/shapeit4/blob/master/src/modules/builder.cpp): state builder · Assembles region-level working state for inference · `src/modules/builder.cpp`
- [PBWT solver](https://github.com/odelaneau/shapeit4/blob/master/src/modules/pbwt_solver.cpp): conditioning · Selects informative conditioning haplotypes · `src/modules/pbwt_solver.cpp`
- [Genotype ops](https://github.com/odelaneau/shapeit4/blob/master/src/objects/genotype/genotype_build.cpp): domain logic · Builds, manages, masks, prunes, and sweeps genotype state · `src/objects/genotype/genotype_build.cpp`
- [HMM params](https://github.com/odelaneau/shapeit4/blob/master/src/objects/hmm_parameters.cpp): model · Statistical parameters for the phasing model · `src/objects/hmm_parameters.cpp`
- [Variants](https://github.com/odelaneau/shapeit4/blob/master/src/objects/variant.cpp): domain model · Variant metadata and genotype-aware processing · `src/objects/variant.cpp`
- [Compute jobs](https://github.com/odelaneau/shapeit4/blob/master/src/objects/compute_job.cpp): work partition · Chunked units of phasing work across genomic regions · `src/objects/compute_job.cpp`
- [Bit matrix](https://github.com/odelaneau/shapeit4/blob/master/src/containers/bitmatrix.cpp): container · Compact binary representation of genotype and haplotype state · `src/containers/bitmatrix.cpp`
- [Sample sets](https://github.com/odelaneau/shapeit4/blob/master/src/containers/genotype_set.cpp): container · Collections of genotype and haplotype samples · `src/containers/genotype_set.cpp`
- [Variant map](https://github.com/odelaneau/shapeit4/blob/master/src/containers/variant_map.cpp): index · Maps variants and positions to internal indices · `src/containers/variant_map.cpp`
- [Input readers](https://github.com/odelaneau/shapeit4/blob/master/src/io/genotype_reader1.cpp): io · Reads genotype and genetic map data · `src/io/genotype_reader1.cpp`
- [Map reader](https://github.com/odelaneau/shapeit4/blob/master/src/io/gmap_reader.cpp): io · Loads recombination maps · `src/io/gmap_reader.cpp`
- [Haplotype writer](https://github.com/odelaneau/shapeit4/blob/master/src/io/haplotype_writer.cpp): io · Writes phased haplotypes · `src/io/haplotype_writer.cpp`
- [Graph writer](https://github.com/odelaneau/shapeit4/blob/master/src/io/graph_writer.cpp): io · Emits graph-like phasing output · `src/io/graph_writer.cpp`
- [Haplotype models](https://github.com/odelaneau/shapeit4/blob/master/src/models/haplotype_segment_single.cpp): model · Single and double haplotype segment representations · `src/models/haplotype_segment_single.cpp`
- [Utilities](https://github.com/odelaneau/shapeit4/blob/master/src/utils/string_utils.h): support · Shared helpers for strings, timing, randomness, stats, and compressed I/O · `src/utils/string_utils.h`

### Auxiliary tool

Standalone sampling utility under tools/bingraphsample

- [Sampler CLI](https://github.com/odelaneau/shapeit4/blob/master/tools/bingraphsample/src/main.cpp): executable · Entry point for the binary graph sampling tool · `tools/bingraphsample/src/main.cpp`
- [Sampler](https://github.com/odelaneau/shapeit4/blob/master/tools/bingraphsample/src/sampler/sampler_management.cpp): orchestrator · Initializes, manages, and finalizes sampling runs · `tools/bingraphsample/src/sampler/sampler_management.cpp`
- [Graph reader](https://github.com/odelaneau/shapeit4/blob/master/tools/bingraphsample/src/io/graph_reader.cpp): io · Reads binary graph-like input for the sampler · `tools/bingraphsample/src/io/graph_reader.cpp`
- [Sampler I/O](https://github.com/odelaneau/shapeit4/blob/master/tools/bingraphsample/src/io/haplotype_writer.cpp): io · Writes sampled haplotypes and handles sample-side data flow · `tools/bingraphsample/src/io/haplotype_writer.cpp`

### Data and docs

Test fixtures, genetic maps, and documentation

- [Fixtures](https://github.com/odelaneau/shapeit4/blob/master/test/reference.vcf.gz): data · Reference, scaffold, unphased, and map test datasets · `test/reference.vcf.gz`
- [Genetic maps](https://github.com/odelaneau/shapeit4/blob/master/maps/genetic_maps.b37.tar.gz): data · Packaged recombination maps for supported builds · `maps/genetic_maps.b37.tar.gz`
- [Docs](https://github.com/odelaneau/shapeit4/blob/master/docs/index.html): documentation · Static documentation site · `docs/index.html`

## Connections

- CLI entry → Phaser: starts · Main executable dispatches into phasing pipeline
- Phaser → Init: initializes · Sets up inputs and runtime state
- Phaser → Manage: coordinates · Manages iterative phasing control flow
- Phaser → Finalize: finalizes · Completes phasing and writes results
- Phaser → Input readers: loads · Consumes genotype inputs
- Phaser → Map reader: loads · Consumes recombination map data
- Phaser → Builder: prepares · Builds inference state for each region or job
- Phaser → PBWT solver: selects · Uses PBWT to choose conditioning haplotypes
- Phaser → Genotype ops: updates · Manages genotype masking, pruning, and sweep operations
- Phaser → HMM params: parameterizes · Feeds the probabilistic model
- Phaser → Haplotype models: tracks · Represents phased segments during inference
- Phaser → Haplotype writer: emits · Writes final phased haplotypes
- Phaser → Graph writer: emits · Optionally writes graph-oriented output
- Builder → Compute jobs: splits · Partitions work into compute jobs
- Builder → Bit matrix: uses · Builds compact working matrices
- Builder → Sample sets: assembles · Organizes sample-level genotype and haplotype collections
- Builder → Variant map: indexes · Maps genomic positions and variants
- PBWT solver → Sample sets: queries · Searches conditioning haplotypes in sample sets
- Genotype ops → Bit matrix: encodes · Stores genotype state compactly
- Genotype ops → Variants: annotates · Attaches variant-aware genotype processing
- Input readers → Fixtures: consumes · Reads the reference and scaffold fixtures during validation
- Map reader → Genetic maps: consumes · Loads packaged recombination maps
- Haplotype writer → Fixtures: verified by · Outputs are validated against packaged fixtures
- Sampler CLI → Sampler: starts · Sampler executable dispatches into its control layer
- Sampler → Graph reader: loads · Reads binary graph input
- Sampler → Sampler I/O: writes · Produces sampled haplotypes
- Sampler → Sample sets: reuses · Shares genotype-set style sample containers
- Sampler → Variant map: reuses · Shares variant indexing logic
- Phaser → Utilities: supports · Uses shared helpers across the pipeline
- Sampler → Utilities: supports · Uses shared helpers across the sampler
- Docs → CLI entry: describes · Documentation explains the main CLI and workflow
