# nmslib/hnswlib architecture

> Architecture diagram of the GitHub repository nmslib/hnswlib, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nmslib/hnswlib
- Repository: https://github.com/nmslib/hnswlib

## Overview

##### Project type and purpose
- This is a fast approximate nearest-neighbor search library (HNSW) implemented as a header-only C++ module with optional Python bindings.
- It provides both C++ and Python APIs for index construction, incremental updates, deletion, search (knn_query), serialization, and deserialization.

##### File structure
- Top‐level: build scripts (CMakeLists.txt, Makefile), packaging configs (setup.py, pyproject.toml, MANIFEST.in), CI workflow (.github/workflows).
- hnswlib/: core header‐only C++ implementation (hnswalg, spaces, helpers).
- python_bindings/: C++ binding sources (bindings.cpp), Python wrapper (LazyIndex.py), setup for pip installation.
- examples/: C++ and Python usage demos.
- tests/: C++ and Python test suites validating functionality and performance.

##### README insights
- No external dependencies beyond C++11 and Python setuptools.
- Core algorithms expose an Index class with methods for init, add_items, knn_query, delete/resize, save/load, filters.
- Emphasis on multi-threaded safety, pickling, and speed/accuracy trade-offs (parameters M, ef_construction, ef).

a. Main components
- Client Applications
– C++ example apps
– Python scripts / user code
- Build & Deployment
– CMake / Makefile for C++
– setup.py / pyproject.toml for Python packaging
– CI pipeline (.github/workflows/build.yml)
- Core Library
– hnswlib headers (graph data structures, distance spaces, search algorithm)
- Python Bindings
– bindings.cpp (C++ → Python layer)
– LazyIndex.py (higher-level Python interface)
- Tests & Examples
– C++ tests (tests/cpp)
– Python tests (tests/python)
- External Environment
– C++11 compiler (gcc/clang/MSVC)
– Python interpreter

b. Relationships and interactions
– Client code → includes hnswlib headers (C++) or imports hnswlib module (Python).
– Python setup builds the binding by invoking the C++ compiler via setup.py / pybind11 or Python C API.
– CMake builds the examples and C++ tests, linking against nothing external (header-only).
– At runtime, the Index object holds an in-memory graph; client calls methods that traverse/update the graph.
– Tests drive both APIs to verify correctness and performance.

c. Architectural patterns & design principles
– Single-module, header-only for minimal dependencies.
– Language-binding wrapper pattern: core logic in C++, thin layer for Python.
– In-memory graph data structure (HNSW) following the navigable small-world graph pattern.
– Build-time separation: C++ build vs. Python package build.

d. Key technologies & frameworks
– C++11 (template headers, inline implementations)
– CMake & Makefile for cross‐platform builds
– Python setuptools, possibly pybind11 or Python C API in bindings.cpp
– Continuous Integration via GitHub Actions

## Diagram (Mermaid)

```mermaid
graph TB
    %% Build System - C++
    subgraph "Build System"
        direction TB
        CMake["CMakeLists.txt"]:::build
        Makefile["Makefile"]:::build
    end

    %% Python Packaging
    subgraph "Python Packaging"
        direction TB
        PySetupRoot["setup.py"]:::build
        PyProject["pyproject.toml"]:::build
        Manifest["MANIFEST.in"]:::build
    end

    %% Continuous Integration
    subgraph "Continuous Integration"
        direction TB
        CI[".github/workflows/build.yml"]:::ci
    end

    %% Core Library
    subgraph "Core HNSW Graph Module"
        direction TB
        Core["hnswlib/"]:::core
    end

    %% Python Binding Layer
    subgraph "Python Binding Layer"
        direction TB
        BindSrc["bindings.cpp"]:::binding
        Init["__init__.py"]:::binding
        LazyIndex["LazyIndex.py"]:::binding
        PySetupBind["setup.py"]:::binding
    end

    %% Clients
    subgraph "Clients"
        direction TB
        CppExamples["C++ Example Apps"]:::examples
        PyExamples["Python Example Scripts"]:::examples
    end

    %% Tests
    subgraph "Tests"
        direction TB
        CppTests["C++ Test Suite"]:::tests
        PyTests["Python Test Suite"]:::tests
    end

    %% External Environment
    subgraph "External Environment" 
        direction TB
        Compiler["C++11 Compiler"]:::external
        Interpreter["Python Interpreter"]:::external
    end

    %% Build-Time Flows
    CMake -->|build C++ examples/tests| Compiler
    PySetupRoot -->|package Python wheel| Interpreter
    PySetupBind -->|compile bindings| Compiler

    CI -->|runs| CMake
    CI -->|runs| PySetupRoot

    %% Runtime Flows
    CppExamples -->|include headers| Core
    PyExamples -->|import LazyIndex| LazyIndex
    LazyIndex -->|calls bindings| BindSrc
    BindSrc -->|uses headers| Core

    CppTests -->|verify API| Core
    PyTests -->|verify API| LazyIndex

    Core -->|resides in-memory| Core

    %% Click Events
    click CMake "https://github.com/nmslib/hnswlib/blob/master/CMakeLists.txt"
    click Makefile "https://github.com/nmslib/hnswlib/tree/master/Makefile"
    click CI "https://github.com/nmslib/hnswlib/blob/master/.github/workflows/build.yml"
    click PySetupRoot "https://github.com/nmslib/hnswlib/blob/master/setup.py"
    click PyProject "https://github.com/nmslib/hnswlib/blob/master/pyproject.toml"
    click Manifest "https://github.com/nmslib/hnswlib/blob/master/MANIFEST.in"
    click Core "https://github.com/nmslib/hnswlib/tree/master/hnswlib/"
    click BindSrc "https://github.com/nmslib/hnswlib/blob/master/python_bindings/bindings.cpp"
    click Init "https://github.com/nmslib/hnswlib/blob/master/python_bindings/__init__.py"
    click LazyIndex "https://github.com/nmslib/hnswlib/blob/master/python_bindings/LazyIndex.py"
    click PySetupBind "https://github.com/nmslib/hnswlib/blob/master/python_bindings/setup.py"
    click CppExamples "https://github.com/nmslib/hnswlib/tree/master/examples/cpp/"
    click PyExamples "https://github.com/nmslib/hnswlib/tree/master/examples/python/"
    click CppTests "https://github.com/nmslib/hnswlib/tree/master/tests/cpp/"
    click PyTests "https://github.com/nmslib/hnswlib/tree/master/tests/python/"

    %% Styles
    classDef build fill:#c2a5cf,stroke:#7b3294,color:#fff;
    classDef ci fill:#f4a582,stroke:#d45087,color:#000;
    classDef core fill:#92c5de,stroke:#0571b0,color:#000;
    classDef binding fill:#b8e186,stroke:#4d9221,color:#000;
    classDef examples fill:#fdb462,stroke:#e66101,color:#000;
    classDef tests fill:#fdb462,stroke:#e66101,color:#000;
    classDef external fill:#b2b2b2,stroke:#636363,color:#000;
```
