How rohannx29/pulse-job-orchestrator works
Pulse is a distributed job scheduling and orchestration system. Users manage organizations, projects, queues, schedules, and jobs through a Next.js dashboard backed by a FastAPI REST API. The API submits work into a shared execution path; Celery workers execute immediate, delayed, recurring, and dependency-aware jobs, recording attempts and failures for retry or dead-letter handling. PostgreSQL is the durable source of truth, while Redis supports Celery work and coordination. The graph also preserves dashboard operations, audit, live events, webhooks, and system health. Some subsystem links rely on README descriptions because their implementation files were not sampled.
Main entry points: tasks.py Worker tasks · main.py FastAPI application · database.py PostgreSQL · webhooks.py Webhook delivery engine
Components 29
Dashboard
- Dashboard pages
frontend/app/page.tsx - Dashboard API client
frontend/lib/api.ts - Live event hook
frontend/lib/useEvents.ts
API and tenancy
- FastAPI application
backend/app/main.py - Authentication and RBAC
backend/app/routers/auth.py - Org and project routes
backend/app/routers/orgs.py - Job and queue routes
backend/app/routers/jobs.py - Schedule routes
backend/app/routers/schedules.py - Dead-letter routes
backend/app/routers/dlq.py - Webhook management
backend/app/routers/webhooks.py
Job orchestration
- Common job dispatch
backend/app/dispatch.py - Dependency DAG engine
backend/app/dag.py - Worker tasks
backend/app/tasks.py - Priority routing
backend/app/priority.py - Queue concurrency
backend/app/concurrency.py - Celery and Beat
backend/app/celery_app.py
Operations and observability
- Audit logging
backend/app/audit.py - Live event publisher
backend/app/events.py - Health and metrics
backend/app/metrics.py - Data retention
backend/app/retention.py - Worker monitoring
backend/app/routers/workers.py - Audit explorer
backend/app/routers/audit.py - SSE event route
backend/app/routers/events.py
Storage and integrations
- Webhook delivery engine
backend/app/webhooks.py - URL safety checks
backend/app/url_safety.py - PostgreSQL
backend/app/database.py - Redis
Other components
- User / operator
- Webhook endpoints
Connections 33
- User / operator → Dashboard pages: operates
- Dashboard pages → Dashboard API client: requests data
frontend/app/page.tsx - Dashboard API client → FastAPI application: calls REST API
- Dashboard pages → Live event hook: subscribes
frontend/app/page.tsx - Live event hook → SSE event route: opens SSE
frontend/lib/useEvents.ts - SSE event route → Live event publisher: streams updates
- FastAPI application → Authentication and RBAC: routes requests
- FastAPI application → Org and project routes: routes requests
- FastAPI application → Job and queue routes: routes requests
- FastAPI application → Schedule routes: routes requests
- FastAPI application → Dead-letter routes: routes requests
- FastAPI application → Webhook management: routes requests
- Job and queue routes → Common job dispatch: submits jobs
- Schedule routes → Worker tasks: creates scheduled jobs
backend/app/tasks.py - Common job dispatch → Priority routing: maps priority
- Common job dispatch → Celery and Beat: queues work
- Celery and Beat → Worker tasks: dispatches tasks
backend/app/tasks.py - Worker tasks → Dependency DAG engine: resolves dependencies
backend/app/tasks.py - Worker tasks → Queue concurrency: limits execution
backend/app/tasks.py - Worker tasks → PostgreSQL: records execution
backend/app/tasks.py - Worker tasks → Redis: uses coordination
- Worker tasks → Data retention: runs pruning
backend/app/tasks.py - Webhook management → Webhook endpoints: tests delivery
backend/app/routers/webhooks.py - Webhook delivery engine → Webhook endpoints: sends signed events
backend/app/webhooks.py - Webhook delivery engine → URL safety checks: validates URLs
backend/app/webhooks.py - Webhook delivery engine → Worker tasks: enqueues delivery
backend/app/webhooks.py - Webhook delivery engine → PostgreSQL: records deliveries
backend/app/webhooks.py - Worker tasks → Live event publisher: publishes updates
backend/app/tasks.py - Live event publisher → Redis: publishes events
- FastAPI application → Audit logging: records actions
- Audit explorer → PostgreSQL: reads audit records
- Worker monitoring → PostgreSQL: reads worker state
backend/app/routers/workers.py - Health and metrics → PostgreSQL: reports readiness