How pinecone-io/pg_pinecone works
pg_pinecone is a PostgreSQL extension that offers native, pgvector-compatible, and Pinecone-remote vector indexes, plus SQL-accessible full-text search. The principal workflow is an application issuing SQL to Postgres: the extension builds or selects an index engine, searches it, and returns rows; remote and document-index features synchronize data with Pinecone. The native engine’s storage and scan layers are represented separately. Some branches are documented in the README or SOURCE INDEX but were not sampled, so their links are shown only where that evidence supports them.
Main entry points: am.rs Index access method · scan.rs Native index scan · worker.rs Background worker · build.rs Native index build
Components 20
SQL and access method
- Index access method
pg_pinecone/src/am.rs - Filtered scans
pg_pinecone/src/am_filter.rs - Full-text search
pg_pinecone/src/docs_search.rs
Native vector engine
- Native index build
pg_pinecone/src/local/build.rs - Native index scan
pg_pinecone/src/local/scan.rs - Index metadata
pg_pinecone/src/local/metapage.rs - Page layout
pg_pinecone/src/local/pages.rs - RaBitQ codec
pg_pinecone/src/local/rabitq.rs - Distance metrics
pg_pinecone/src/local/metric.rs
Sync and lifecycle
- Change outbox
pg_pinecone/src/outbox.rs - Sync trigger
pg_pinecone/src/sync_trigger.rs - Background worker
pg_pinecone/src/worker.rs - Index maintenance
pg_pinecone/src/maintain.rs - Engine lifecycle
pg_pinecone/src/lifecycle.rs - Partition registry
pg_pinecone/src/registry.rs
Pinecone integration
- Remote backend
pg_pinecone/src/backend.rs - Pinecone API client
pg_pinecone-client/src/client.rs
Other components
- Application
- PostgreSQL
- Pinecone service
Connections 21
- Application → PostgreSQL: sends SQL
README.md - PostgreSQL → Index access method: plans index query
README.md - Index access method → Native index build: builds native index
pg_pinecone/src/am.rs - Index access method → Native index scan: dispatches local scan
pg_pinecone/src/am.rs - Index access method → Remote backend: uses remote mode
README.md - Index access method → Filtered scans: supports filtered scans
pg_pinecone/src/am.rs - Index access method → Index metadata: reads index metadata
pg_pinecone/src/am.rs - Native index build → Page layout: writes page layout
pg_pinecone/src/local/pages.rs - Native index build → Index metadata: writes metadata
pg_pinecone/src/local/metapage.rs - Native index scan → Page layout: reads list pages
pg_pinecone/src/local/scan.rs - Native index scan → RaBitQ codec: scores codes
pg_pinecone/src/local/scan.rs - Native index scan → Distance metrics: computes exact distances
pg_pinecone/src/local/metric.rs - PostgreSQL → Full-text search: runs search SQL
README.md - Sync trigger → Change outbox: records changes
pg_pinecone/src/sync_trigger.rs - Background worker → Change outbox: processes pending work
pg_pinecone/src/worker.rs - Background worker → Index maintenance: runs maintenance
pg_pinecone/src/worker.rs - Background worker → Engine lifecycle: runs lifecycle tasks
pg_pinecone/src/worker.rs - Remote backend → Partition registry: uses partition registry
pg_pinecone/src/backend.rs - Remote backend → Pinecone API client: calls API client
- Pinecone API client → Pinecone service: sends API requests
pg_pinecone-client/src/client.rs - Full-text search → Pinecone service: searches document index
README.md