arpatnurmamat-dotcom/vectorvault

Purpose and entry points

VectorVault is an embedded, persistent vector database: a C++17 static library with a stable C ABI, C++ wrapper, and local CLI. Applications open a single .vault file, insert/query/delete vectors with JSON metadata, and perform maintenance without a server. Public interfaces are include/vectorvault/vectorvault.h and include/vectorvault/vectorvault.hpp; the inspection/maintenance executable is tools/vv_cli.cpp.

API lifecycle and database orchestration

The database layer owns open/create, configuration validation, CRUD, synchronization, compaction, and teardown. It assembles storage, metadata, and the selected vector index behind the public API, while enforcing the documented concurrent-read / serialized-write model. Core paths: src/core/database.cpp, src/core/database.h, src/core/config.cpp.

Query execution boundary

Search requests flow through a query executor, which coordinates approximate or exact nearest-neighbor retrieval with optional metadata-filter evaluation and result handling. This is the boundary between caller-facing KNN semantics and index/storage internals. Paths: src/query/executor.cpp, src/query/executor.h, src/meta/filter.cpp.

Vector indexing and distance runtime dispatch

VectorIndex abstracts index implementations. HNSW is the primary incremental approximate index; Flat provides brute-force/exact search. HNSW persistence is separated into index I/O. Distance computation supports cosine, L2, and inner product, with platform-specific AVX2, SSE, and NEON implementations selected by runtime/platform support. Paths: src/index/vector_index.h, src/index/hnsw.cpp, src/index/flat.cpp; SIMD paths: src/index/distance.cpp, src/index/distance_avx2.cpp, src/index/distance_neon.cpp.

Metadata and filtering

Metadata is stored as JSON associated with vector IDs. A B-tree supplies ID-oriented metadata lookup and range-oriented access; the filter subsystem parses/evaluates expression predicates such as comparisons, boolean operators, and grouping against metadata. Paths: src/meta/btree.cpp, src/meta/btree.h, src/meta/filter.h.

Persistent storage engine

The single-file persistence layer is page-based (README specifies 4 KB pages). PageManager defines page allocation/layout and coordinates durable storage; BufferPool caches pages with LRU behavior; FreeList recovers reusable space after deletes/compaction. File I/O handles low-level reads/writes and WAL-related durability/recovery behavior. Paths: src/storage/page_manager.cpp, src/storage/buffer_pool.cpp, src/storage/free_list.cpp, src/storage/file_io.cpp.

End-to-end flows and infrastructure

On open, database/configuration initialization opens or creates the vault, recovers durable state as needed, initializes page-backed metadata/vector state, and restores the persisted HNSW graph. Insert writes vector/metadata records through storage, updates metadata and index structures, and records durable changes. Search executes index candidate retrieval, evaluates an optional filter, and returns nearest accepted results. Delete tombstones data; compaction reclaims space and rebuilds indexes. The only material runtime dependencies are C++/C system facilities—threading, filesystem/file descriptors, CPU-feature/platform support—with no network service or third-party dependency. Build integration is defined by CMakeLists.txt.