How janbjorge/pgqueuer works
Project type and purpose
• PGQueuer is an open-source Python library and CLI tool that provides a high-performance job queue built on PostgreSQL. It exposes both a programmatic API (the pgqueuer package) and a command-line interface (pgq) to enqueue/dequeue jobs, schedule recurring tasks, and display a real-time dashboard.
File structure
• Top-level CLI & orchestration:
• Dockerfile, docker-compose.yml (containerized Postgres + optional services)
• Makefile (build/test tasks)
• Core library (pgqueuer/):
• db.py, drivers.py – abstract driver & Postgres driver implementations
• queries.py, qb.py – SQL query generation & execution
• executors.py, tm.py, sm.py, qm.py – thread/process executors, task manager, state machine
• heartbeat.py, listeners.py, completion.py – event/listener loops, status watchers
• buffers.py, cache.py – in-memory job buffering & caching
• cli.py, applications.py, factories.py – command-line commands & factory functions
• tracing.py, logconfig.py – optional distributed tracing & logging setup
• Support files: examples/, docs/, tools/, test/
Insights from README
• Uses PostgreSQL’s LISTEN/NOTIFY for real-time wake-ups
• Uses FOR UPDATE SKIP LOCKED for safe concurrency
• Supports recurring schedules (cron syntax) and graceful shutdown
• Offers a live dashboard (CLI) and integration points for metrics (Prometheus)