# phanindra798/fastvec architecture

> Architecture diagram of the GitHub repository phanindra798/fastvec, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/phanindra798/fastvec
- Repository: https://github.com/phanindra798/fastvec

## Overview

fastvec is a Go vector-similarity search engine with exact flat search and an approximate, layered HNSW index. Its main workflow reads vector data, builds or loads an index, searches queries, and measures recall and throughput against ground truth. The repository also provides dataset-subsetting support and a benchmark comparison workflow. Sampled source confirms the benchmark runner’s orchestration and core HNSW search behavior; some lower-level index and persistence wiring is represented only where documented or directly supported by the tree.

Main entry points: [`cmd/fastvec-bench/main.go`](https://github.com/phanindra798/fastvec/blob/main/cmd/fastvec-bench/main.go) (Benchmark runner), [`internal/index/hnsw.go`](https://github.com/phanindra798/fastvec/blob/main/internal/index/hnsw.go) (HNSW index), [`cmd/fastvec-subset/main.go`](https://github.com/phanindra798/fastvec/blob/main/cmd/fastvec-subset/main.go) (Subset generator), [`internal/index/flat.go`](https://github.com/phanindra798/fastvec/blob/main/internal/index/flat.go) (Exact search)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Vector data"]
  node_subsetcli["Subset generator<br/>[main.go]"]
  node_fvecs["Vector parser<br/>[fvecs.go]"]
  node_truth["Ground truth<br/>[fvecs.go]"]
end

subgraph group_index["Search index"]
  node_flat["Exact search<br/>[flat.go]"]
  node_hnsw["HNSW index<br/>[hnsw.go]"]
  node_storage["Index persistence<br/>[store.go]"]
end

subgraph group_compute["Search internals"]
  node_workspace["Search workspace<br/>[workspace.go]"]
  node_candidates["Candidate search<br/>[candidates.go]"]
  node_distance["Distance kernels<br/>[distance.go]"]
  node_topk["Top-k ranking<br/>[topk.go]"]
end

subgraph group_benchmark["Benchmarking"]
  node_benchcli["Benchmark runner<br/>[main.go]"]
  node_benchenv["Machine metadata<br/>[benchenv.go]"]
  node_results["Benchmark results"]
  node_baseline["hnswlib baseline<br/>[bench_baselines.py]"]
  node_plots["Recall/QPS plots<br/>[plots.py]"]
end

node_user(("Engineer"))
node_dataset["SIFT/GIST files"]
node_opponent["hnswlib"]

node_user -->|"runs benchmark"| node_benchcli
node_dataset -->|"vector files"| node_fvecs
node_dataset -->|"ground truth file"| node_truth
node_benchcli -->|"reads vectors"| node_fvecs
node_benchcli -->|"reads truth"| node_truth
node_benchcli -->|"loads index"| node_storage
node_benchcli -->|"builds index"| node_hnsw
node_hnsw -->|"saves index"| node_storage
node_benchcli -->|"searches queries"| node_hnsw
node_hnsw -->|"uses scratch"| node_workspace
node_hnsw -->|"searches layers"| node_candidates
node_hnsw -->|"measures vectors"| node_distance
node_hnsw -->|"ranks candidates"| node_topk
node_benchcli -->|"scores recall"| node_distance
node_benchcli -->|"uses results"| node_topk
node_benchcli -->|"captures environment"| node_benchenv
node_benchcli -->|"writes JSON"| node_results
node_user -->|"prepares subset"| node_subsetcli
node_subsetcli -->|"reads vectors"| node_fvecs
node_subsetcli -->|"computes exact neighbors"| node_flat
node_flat -->|"measures vectors"| node_distance
node_flat -->|"ranks results"| node_topk
node_user -->|"runs baseline"| node_baseline
node_baseline -->|"benchmarks"| node_opponent
node_baseline -->|"writes JSON"| node_results
node_results -->|"loads sweeps"| node_plots
node_plots -->|"renders comparison"| node_user

click node_benchcli "https://github.com/phanindra798/fastvec/blob/main/cmd/fastvec-bench/main.go"
click node_subsetcli "https://github.com/phanindra798/fastvec/blob/main/cmd/fastvec-subset/main.go"
click node_fvecs "https://github.com/phanindra798/fastvec/blob/main/internal/fvecs/fvecs.go"
click node_truth "https://github.com/phanindra798/fastvec/blob/main/internal/fvecs/fvecs.go"
click node_flat "https://github.com/phanindra798/fastvec/blob/main/internal/index/flat.go"
click node_hnsw "https://github.com/phanindra798/fastvec/blob/main/internal/index/hnsw.go"
click node_storage "https://github.com/phanindra798/fastvec/blob/main/internal/index/store.go"
click node_workspace "https://github.com/phanindra798/fastvec/blob/main/internal/index/workspace.go"
click node_candidates "https://github.com/phanindra798/fastvec/blob/main/internal/index/candidates.go"
click node_distance "https://github.com/phanindra798/fastvec/blob/main/internal/distance/distance.go"
click node_topk "https://github.com/phanindra798/fastvec/blob/main/internal/topk/topk.go"
click node_benchenv "https://github.com/phanindra798/fastvec/blob/main/internal/benchenv/benchenv.go"
click node_results "https://github.com/phanindra798/fastvec/tree/main/bench/results"
click node_baseline "https://github.com/phanindra798/fastvec/blob/main/py/bench_baselines.py"
click node_plots "https://github.com/phanindra798/fastvec/blob/main/py/plots.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_subsetcli,node_fvecs,node_truth toneBlue
class node_flat,node_hnsw,node_storage toneAmber
class node_workspace,node_candidates,node_distance,node_topk toneMint
class node_benchcli,node_benchenv,node_results,node_baseline,node_plots toneRose
class node_user,node_dataset,node_opponent toneIndigo
```

## Components

### Vector data

- [Subset generator](https://github.com/phanindra798/fastvec/blob/main/cmd/fastvec-subset/main.go): `cmd/fastvec-subset/main.go`
- [Vector parser](https://github.com/phanindra798/fastvec/blob/main/internal/fvecs/fvecs.go): `internal/fvecs/fvecs.go`
- [Ground truth](https://github.com/phanindra798/fastvec/blob/main/internal/fvecs/fvecs.go): `internal/fvecs/fvecs.go`

### Search index

- [Exact search](https://github.com/phanindra798/fastvec/blob/main/internal/index/flat.go): `internal/index/flat.go`
- [HNSW index](https://github.com/phanindra798/fastvec/blob/main/internal/index/hnsw.go): `internal/index/hnsw.go`
- [Index persistence](https://github.com/phanindra798/fastvec/blob/main/internal/index/store.go): `internal/index/store.go`

### Search internals

- [Search workspace](https://github.com/phanindra798/fastvec/blob/main/internal/index/workspace.go): `internal/index/workspace.go`
- [Candidate search](https://github.com/phanindra798/fastvec/blob/main/internal/index/candidates.go): `internal/index/candidates.go`
- [Distance kernels](https://github.com/phanindra798/fastvec/blob/main/internal/distance/distance.go): `internal/distance/distance.go`
- [Top-k ranking](https://github.com/phanindra798/fastvec/blob/main/internal/topk/topk.go): `internal/topk/topk.go`

### Benchmarking

- [Benchmark runner](https://github.com/phanindra798/fastvec/blob/main/cmd/fastvec-bench/main.go): `cmd/fastvec-bench/main.go`
- [Machine metadata](https://github.com/phanindra798/fastvec/blob/main/internal/benchenv/benchenv.go): `internal/benchenv/benchenv.go`
- [Benchmark results](https://github.com/phanindra798/fastvec/tree/main/bench/results): `bench/results`
- [hnswlib baseline](https://github.com/phanindra798/fastvec/blob/main/py/bench_baselines.py): `py/bench_baselines.py`
- [Recall/QPS plots](https://github.com/phanindra798/fastvec/blob/main/py/plots.py): `py/plots.py`

### Other components

- **Engineer**
- **SIFT/GIST files**
- **hnswlib**

## Connections

- Engineer → Benchmark runner: runs benchmark
- SIFT/GIST files → Vector parser: vector files
- SIFT/GIST files → Ground truth: ground truth file
- Benchmark runner → Vector parser: reads vectors
- Benchmark runner → Ground truth: reads truth
- Benchmark runner → Index persistence: loads index
- Benchmark runner → HNSW index: builds index
- HNSW index → Index persistence: saves index
- Benchmark runner → HNSW index: searches queries
- HNSW index → Search workspace: uses scratch
- HNSW index → Candidate search: searches layers
- HNSW index → Distance kernels: measures vectors
- HNSW index → Top-k ranking: ranks candidates
- Benchmark runner → Distance kernels: scores recall
- Benchmark runner → Top-k ranking: uses results
- Benchmark runner → Machine metadata: captures environment
- Benchmark runner → Benchmark results: writes JSON
- Engineer → Subset generator: prepares subset
- Subset generator → Vector parser: reads vectors
- Subset generator → Exact search: computes exact neighbors
- Exact search → Distance kernels: measures vectors
- Exact search → Top-k ranking: ranks results
- Engineer → hnswlib baseline: runs baseline
- hnswlib baseline → hnswlib: benchmarks
- hnswlib baseline → Benchmark results: writes JSON
- Benchmark results → Recall/QPS plots: loads sweeps
- Recall/QPS plots → Engineer: renders comparison