How timescale/pgai works
pgai turns PostgreSQL into a retrieval engine: applications configure vectorizers, workers process queued source data into embeddings and write results back for search and RAG. Its Semantic Catalog adds schema description, semantic search, and natural-language-to-SQL capabilities. The README documents the worker lifecycle; source-index references and excerpts support the shown internal calls. Several implementation details, including unsampled worker and catalog modules, are represented only at subsystem level.
Main entry points: semantic_catalog.py Catalog Management · worker.py Vectorizer Worker · search.py Semantic Search · cli.py Command Line
Components 19
Vectorization
- Vectorizer Configuration
projects/pgai/pgai/vectorizer/create_vectorizer.py - Vectorizer Worker
projects/pgai/pgai/vectorizer/worker.py - Vector Processing
projects/pgai/pgai/vectorizer/vectorizer.py - Chunking
projects/pgai/pgai/vectorizer/chunking.py - Embedding Batches
projects/pgai/pgai/vectorizer/embeddings.py - Provider Adapters
projects/pgai/pgai/vectorizer/embedders/litellm.py - Worker Tracking
projects/pgai/pgai/vectorizer/worker_tracking/worker_tracking.py
Semantic Catalog
- Catalog Management
projects/pgai/pgai/semantic_catalog/semantic_catalog.py - Schema Description
projects/pgai/pgai/semantic_catalog/describe.py - Catalog Import Export
projects/pgai/pgai/semantic_catalog/file.py - Catalog Embeddings
projects/pgai/pgai/semantic_catalog/vectorizer/vectorizer.py - Semantic Search
projects/pgai/pgai/semantic_catalog/search.py - Text-to-SQL
projects/pgai/pgai/semantic_catalog/gen_sql.py - Catalog Models
projects/pgai/pgai/semantic_catalog/models.py
Runtime and Setup
- Command Line
projects/pgai/pgai/cli.py - Database Installer
projects/pgai/pgai/_install/install.py
Other components
- Application
- PostgreSQL
- Embedding Providers
Connections 26
- Application → PostgreSQL: configures
- Application → Vectorizer Configuration: defines
- Vectorizer Configuration → PostgreSQL: stores configuration
- Vectorizer Worker → PostgreSQL: reads queue
- Vectorizer Worker → Vector Processing: processes work
projects/pgai/pgai/vectorizer/worker.py - Vector Processing → Chunking: chunks text
projects/pgai/pgai/vectorizer/vectorizer.py - Vector Processing → Embedding Batches: requests embeddings
- Embedding Batches → Provider Adapters: dispatches batches
projects/pgai/pgai/vectorizer/embedders/litellm.py - Provider Adapters → Embedding Providers: calls provider
- Vectorizer Worker → PostgreSQL: writes results
- Vectorizer Worker → Worker Tracking: reports progress
projects/pgai/pgai/vectorizer/worker.py - Application → PostgreSQL: queries vectors
- Command Line → Vectorizer Worker: runs worker
- Application → Database Installer: installs database components
- Command Line → Database Installer: installs components
projects/pgai/pgai/cli.py - Command Line → Catalog Management: invokes catalog operations
projects/pgai/pgai/cli.py - Catalog Management → Schema Description: describes schema
- Schema Description → Catalog Import Export: uses catalog files
projects/pgai/pgai/semantic_catalog/describe.py - Catalog Management → Catalog Embeddings: vectorizes catalog
projects/pgai/pgai/semantic_catalog/semantic_catalog.py - Catalog Management → Semantic Search: searches catalog
projects/pgai/pgai/semantic_catalog/semantic_catalog.py - Semantic Search → PostgreSQL: queries embeddings
projects/pgai/pgai/semantic_catalog/search.py - Catalog Management → Text-to-SQL: generates SQL
projects/pgai/pgai/semantic_catalog/semantic_catalog.py - Text-to-SQL → Semantic Search: retrieves context
projects/pgai/pgai/semantic_catalog/gen_sql.py - Text-to-SQL → PostgreSQL: validates SQL
projects/pgai/pgai/semantic_catalog/gen_sql.py - Catalog Management → Catalog Models: uses models
projects/pgai/pgai/semantic_catalog/semantic_catalog.py - Semantic Search → Catalog Models: returns records
projects/pgai/pgai/semantic_catalog/search.py