# pinecone-io/pg_pinecone architecture

> Architecture diagram of the GitHub repository pinecone-io/pg_pinecone, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pinecone-io/pg_pinecone
- Repository: https://github.com/pinecone-io/pg_pinecone

## Overview

pg_pinecone is a PostgreSQL extension that offers native, pgvector-compatible, and Pinecone-remote vector indexes, plus SQL-accessible full-text search. The principal workflow is an application issuing SQL to Postgres: the extension builds or selects an index engine, searches it, and returns rows; remote and document-index features synchronize data with Pinecone. The native engine’s storage and scan layers are represented separately. Some branches are documented in the README or SOURCE INDEX but were not sampled, so their links are shown only where that evidence supports them.

Main entry points: [`pg_pinecone/src/am.rs`](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/am.rs) (Index access method), [`pg_pinecone/src/local/scan.rs`](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/scan.rs) (Native index scan), [`pg_pinecone/src/worker.rs`](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/worker.rs) (Background worker), [`pg_pinecone/src/local/build.rs`](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/build.rs) (Native index build)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_sql["SQL and access method"]
  node_am["Index access method<br/>[am.rs]"]
  node_filter["Filtered scans<br/>[am_filter.rs]"]
  node_doc_search["Full-text search<br/>[docs_search.rs]"]
end

subgraph group_native["Native vector engine"]
  node_local_build["Native index build<br/>[build.rs]"]
  node_local_scan["Native index scan<br/>[scan.rs]"]
  node_local_meta["Index metadata<br/>[metapage.rs]"]
  node_local_pages["Page layout<br/>[pages.rs]"]
  node_local_codec["RaBitQ codec<br/>[rabitq.rs]"]
  node_local_metric["Distance metrics<br/>[metric.rs]"]
end

subgraph group_sync["Sync and lifecycle"]
  node_outbox[("Change outbox<br/>[outbox.rs]")]
  node_trigger["Sync trigger<br/>[sync_trigger.rs]"]
  node_worker["Background worker<br/>[worker.rs]"]
  node_maintain["Index maintenance<br/>[maintain.rs]"]
  node_lifecycle["Engine lifecycle<br/>[lifecycle.rs]"]
  node_registry[("Partition registry<br/>[registry.rs]")]
end

subgraph group_pinecone["Pinecone integration"]
  node_backend["Remote backend<br/>[backend.rs]"]
  node_client["Pinecone API client<br/>[client.rs]"]
end

node_app(("Application"))
node_postgres[("PostgreSQL")]
node_pinecone_service["Pinecone service"]

node_app -->|"sends SQL"| node_postgres
node_postgres -->|"plans index query"| node_am
node_am -->|"builds native index"| node_local_build
node_am -->|"dispatches local scan"| node_local_scan
node_am -.->|"uses remote mode"| node_backend
node_am -->|"supports filtered scans"| node_filter
node_am -->|"reads index metadata"| node_local_meta
node_local_build -->|"writes page layout"| node_local_pages
node_local_build -->|"writes metadata"| node_local_meta
node_local_scan -->|"reads list pages"| node_local_pages
node_local_scan -->|"scores codes"| node_local_codec
node_local_scan -->|"computes exact distances"| node_local_metric
node_postgres -->|"runs search SQL"| node_doc_search
node_trigger -->|"records changes"| node_outbox
node_worker -->|"processes pending work"| node_outbox
node_worker -->|"runs maintenance"| node_maintain
node_worker -->|"runs lifecycle tasks"| node_lifecycle
node_backend -.->|"uses partition registry"| node_registry
node_backend -.->|"calls API client"| node_client
node_client -->|"sends API requests"| node_pinecone_service
node_doc_search -.->|"searches document index"| node_pinecone_service

click node_am "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/am.rs"
click node_filter "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/am_filter.rs"
click node_doc_search "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/docs_search.rs"
click node_local_build "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/build.rs"
click node_local_scan "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/scan.rs"
click node_local_meta "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/metapage.rs"
click node_local_pages "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/pages.rs"
click node_local_codec "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/rabitq.rs"
click node_local_metric "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/metric.rs"
click node_outbox "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/outbox.rs"
click node_trigger "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/sync_trigger.rs"
click node_worker "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/worker.rs"
click node_maintain "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/maintain.rs"
click node_lifecycle "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/lifecycle.rs"
click node_backend "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/backend.rs"
click node_client "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone-client/src/client.rs"
click node_registry "https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/registry.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_am,node_filter,node_doc_search toneBlue
class node_local_build,node_local_scan,node_local_meta,node_local_pages,node_local_codec,node_local_metric,node_postgres toneAmber
class node_outbox,node_trigger,node_worker,node_maintain,node_lifecycle,node_registry toneMint
class node_backend,node_client toneRose
class node_app,node_pinecone_service toneIndigo
```

## Components

### SQL and access method

- [Index access method](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/am.rs): `pg_pinecone/src/am.rs`
- [Filtered scans](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/am_filter.rs): `pg_pinecone/src/am_filter.rs`
- [Full-text search](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/docs_search.rs): `pg_pinecone/src/docs_search.rs`

### Native vector engine

- [Native index build](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/build.rs): `pg_pinecone/src/local/build.rs`
- [Native index scan](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/scan.rs): `pg_pinecone/src/local/scan.rs`
- [Index metadata](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/metapage.rs): `pg_pinecone/src/local/metapage.rs`
- [Page layout](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/pages.rs): `pg_pinecone/src/local/pages.rs`
- [RaBitQ codec](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/rabitq.rs): `pg_pinecone/src/local/rabitq.rs`
- [Distance metrics](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/local/metric.rs): `pg_pinecone/src/local/metric.rs`

### Sync and lifecycle

- [Change outbox](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/outbox.rs): `pg_pinecone/src/outbox.rs`
- [Sync trigger](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/sync_trigger.rs): `pg_pinecone/src/sync_trigger.rs`
- [Background worker](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/worker.rs): `pg_pinecone/src/worker.rs`
- [Index maintenance](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/maintain.rs): `pg_pinecone/src/maintain.rs`
- [Engine lifecycle](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/lifecycle.rs): `pg_pinecone/src/lifecycle.rs`
- [Partition registry](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/registry.rs): `pg_pinecone/src/registry.rs`

### Pinecone integration

- [Remote backend](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone/src/backend.rs): `pg_pinecone/src/backend.rs`
- [Pinecone API client](https://github.com/pinecone-io/pg_pinecone/blob/main/pg_pinecone-client/src/client.rs): `pg_pinecone-client/src/client.rs`

### Other components

- **Application**
- **PostgreSQL**
- **Pinecone service**

## Connections

- Application → PostgreSQL: sends SQL (evidence: `README.md`)
- PostgreSQL → Index access method: plans index query (evidence: `README.md`)
- Index access method → Native index build: builds native index (evidence: `pg_pinecone/src/am.rs`)
- Index access method → Native index scan: dispatches local scan (evidence: `pg_pinecone/src/am.rs`)
- Index access method → Remote backend: uses remote mode (evidence: `README.md`)
- Index access method → Filtered scans: supports filtered scans (evidence: `pg_pinecone/src/am.rs`)
- Index access method → Index metadata: reads index metadata (evidence: `pg_pinecone/src/am.rs`)
- Native index build → Page layout: writes page layout (evidence: `pg_pinecone/src/local/pages.rs`)
- Native index build → Index metadata: writes metadata (evidence: `pg_pinecone/src/local/metapage.rs`)
- Native index scan → Page layout: reads list pages (evidence: `pg_pinecone/src/local/scan.rs`)
- Native index scan → RaBitQ codec: scores codes (evidence: `pg_pinecone/src/local/scan.rs`)
- Native index scan → Distance metrics: computes exact distances (evidence: `pg_pinecone/src/local/metric.rs`)
- PostgreSQL → Full-text search: runs search SQL (evidence: `README.md`)
- Sync trigger → Change outbox: records changes (evidence: `pg_pinecone/src/sync_trigger.rs`)
- Background worker → Change outbox: processes pending work (evidence: `pg_pinecone/src/worker.rs`)
- Background worker → Index maintenance: runs maintenance (evidence: `pg_pinecone/src/worker.rs`)
- Background worker → Engine lifecycle: runs lifecycle tasks (evidence: `pg_pinecone/src/worker.rs`)
- Remote backend → Partition registry: uses partition registry (evidence: `pg_pinecone/src/backend.rs`)
- Remote backend → Pinecone API client: calls API client
- Pinecone API client → Pinecone service: sends API requests (evidence: `pg_pinecone-client/src/client.rs`)
- Full-text search → Pinecone service: searches document index (evidence: `README.md`)