# rohannx29/pulse-job-orchestrator architecture

> Architecture diagram of the GitHub repository rohannx29/pulse-job-orchestrator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rohannx29/pulse-job-orchestrator
- Repository: https://github.com/rohannx29/pulse-job-orchestrator

## Overview

Pulse is a distributed job scheduling and orchestration system. Users manage organizations, projects, queues, schedules, and jobs through a Next.js dashboard backed by a FastAPI REST API. The API submits work into a shared execution path; Celery workers execute immediate, delayed, recurring, and dependency-aware jobs, recording attempts and failures for retry or dead-letter handling. PostgreSQL is the durable source of truth, while Redis supports Celery work and coordination. The graph also preserves dashboard operations, audit, live events, webhooks, and system health. Some subsystem links rely on README descriptions because their implementation files were not sampled.

Main entry points: [`backend/app/tasks.py`](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/tasks.py) (Worker tasks), [`backend/app/main.py`](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/main.py) (FastAPI application), [`backend/app/database.py`](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/database.py) (PostgreSQL), [`backend/app/webhooks.py`](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/webhooks.py) (Webhook delivery engine)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_dashboard["Dashboard"]
  node_dashboard_pages["Dashboard pages<br/>[page.tsx]"]
  node_dashboard_api["Dashboard API client<br/>[api.ts]"]
  node_live_hook["Live event hook<br/>[useEvents.ts]"]
end

subgraph group_api["API and tenancy"]
  node_api_app["FastAPI application<br/>[main.py]"]
  node_auth["Authentication and RBAC<br/>[auth.py]"]
  node_tenant_routes["Org and project routes<br/>[orgs.py]"]
  node_job_routes["Job and queue routes<br/>[jobs.py]"]
  node_schedule_routes["Schedule routes<br/>[schedules.py]"]
  node_dlq_routes["Dead-letter routes<br/>[dlq.py]"]
  node_webhook_routes["Webhook management<br/>[webhooks.py]"]
end

subgraph group_execution["Job orchestration"]
  node_dispatch["Common job dispatch<br/>[dispatch.py]"]
  node_dag["Dependency DAG engine<br/>[dag.py]"]
  node_celery_tasks["Worker tasks<br/>[tasks.py]"]
  node_priority["Priority routing<br/>[priority.py]"]
  node_concurrency["Queue concurrency<br/>[concurrency.py]"]
  node_celery_config["Celery and Beat<br/>[celery_app.py]"]
end

subgraph group_operations["Operations and observability"]
  node_audit["Audit logging<br/>[audit.py]"]
  node_events["Live event publisher<br/>[events.py]"]
  node_health_metrics["Health and metrics<br/>[metrics.py]"]
  node_retention["Data retention<br/>[retention.py]"]
  node_workers_routes["Worker monitoring<br/>[workers.py]"]
  node_audit_routes["Audit explorer<br/>[audit.py]"]
  node_events_routes["SSE event route<br/>[events.py]"]
end

subgraph group_integrations["Storage and integrations"]
  node_webhook_engine["Webhook delivery engine<br/>[webhooks.py]"]
  node_url_safety["URL safety checks<br/>[url_safety.py]"]
  node_postgres[("PostgreSQL<br/>[database.py]")]
  node_redis[("Redis")]
end

node_operator(("User / operator"))
node_external_endpoints(("Webhook endpoints"))

node_operator -->|"operates"| node_dashboard_pages
node_dashboard_pages -->|"requests data"| node_dashboard_api
node_dashboard_api -->|"calls REST API"| node_api_app
node_dashboard_pages -->|"subscribes"| node_live_hook
node_live_hook -->|"opens SSE"| node_events_routes
node_events_routes -.->|"streams updates"| node_events
node_api_app -.->|"routes requests"| node_auth
node_api_app -.->|"routes requests"| node_tenant_routes
node_api_app -.->|"routes requests"| node_job_routes
node_api_app -.->|"routes requests"| node_schedule_routes
node_api_app -.->|"routes requests"| node_dlq_routes
node_api_app -.->|"routes requests"| node_webhook_routes
node_job_routes -.->|"submits jobs"| node_dispatch
node_schedule_routes -->|"creates scheduled jobs"| node_celery_tasks
node_dispatch -.->|"maps priority"| node_priority
node_dispatch -.->|"queues work"| node_celery_config
node_celery_config -.->|"dispatches tasks"| node_celery_tasks
node_celery_tasks -->|"resolves dependencies"| node_dag
node_celery_tasks -->|"limits execution"| node_concurrency
node_celery_tasks -.->|"records execution"| node_postgres
node_celery_tasks -.->|"uses coordination"| node_redis
node_celery_tasks -->|"runs pruning"| node_retention
node_webhook_routes -.->|"tests delivery"| node_external_endpoints
node_webhook_engine -->|"sends signed events"| node_external_endpoints
node_webhook_engine -->|"validates URLs"| node_url_safety
node_webhook_engine -.->|"enqueues delivery"| node_celery_tasks
node_webhook_engine -->|"records deliveries"| node_postgres
node_celery_tasks -.->|"publishes updates"| node_events
node_events -.->|"publishes events"| node_redis
node_api_app -.->|"records actions"| node_audit
node_audit_routes -.->|"reads audit records"| node_postgres
node_workers_routes -->|"reads worker state"| node_postgres
node_health_metrics -.->|"reports readiness"| node_postgres

click node_dashboard_pages "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/frontend/app/page.tsx"
click node_dashboard_api "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/frontend/lib/api.ts"
click node_live_hook "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/frontend/lib/useEvents.ts"
click node_api_app "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/main.py"
click node_auth "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/auth.py"
click node_tenant_routes "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/orgs.py"
click node_job_routes "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/jobs.py"
click node_schedule_routes "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/schedules.py"
click node_dlq_routes "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/dlq.py"
click node_webhook_routes "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/webhooks.py"
click node_dispatch "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/dispatch.py"
click node_dag "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/dag.py"
click node_celery_tasks "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/tasks.py"
click node_priority "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/priority.py"
click node_concurrency "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/concurrency.py"
click node_celery_config "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/celery_app.py"
click node_webhook_engine "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/webhooks.py"
click node_url_safety "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/url_safety.py"
click node_postgres "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/database.py"
click node_audit "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/audit.py"
click node_events "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/events.py"
click node_health_metrics "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/metrics.py"
click node_retention "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/retention.py"
click node_workers_routes "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/workers.py"
click node_audit_routes "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/audit.py"
click node_events_routes "https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/events.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard_pages,node_dashboard_api,node_live_hook,node_operator toneBlue
class node_api_app,node_auth,node_tenant_routes,node_job_routes,node_schedule_routes,node_dlq_routes,node_webhook_routes toneAmber
class node_dispatch,node_dag,node_celery_tasks,node_priority,node_concurrency,node_celery_config,node_external_endpoints toneMint
class node_audit,node_events,node_health_metrics,node_retention,node_workers_routes,node_audit_routes,node_events_routes toneRose
class node_webhook_engine,node_url_safety,node_postgres,node_redis toneIndigo
```

## Components

### Dashboard

- [Dashboard pages](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/frontend/app/page.tsx): `frontend/app/page.tsx`
- [Dashboard API client](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/frontend/lib/api.ts): `frontend/lib/api.ts`
- [Live event hook](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/frontend/lib/useEvents.ts): `frontend/lib/useEvents.ts`

### API and tenancy

- [FastAPI application](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/main.py): `backend/app/main.py`
- [Authentication and RBAC](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/auth.py): `backend/app/routers/auth.py`
- [Org and project routes](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/orgs.py): `backend/app/routers/orgs.py`
- [Job and queue routes](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/jobs.py): `backend/app/routers/jobs.py`
- [Schedule routes](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/schedules.py): `backend/app/routers/schedules.py`
- [Dead-letter routes](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/dlq.py): `backend/app/routers/dlq.py`
- [Webhook management](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/webhooks.py): `backend/app/routers/webhooks.py`

### Job orchestration

- [Common job dispatch](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/dispatch.py): `backend/app/dispatch.py`
- [Dependency DAG engine](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/dag.py): `backend/app/dag.py`
- [Worker tasks](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/tasks.py): `backend/app/tasks.py`
- [Priority routing](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/priority.py): `backend/app/priority.py`
- [Queue concurrency](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/concurrency.py): `backend/app/concurrency.py`
- [Celery and Beat](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/celery_app.py): `backend/app/celery_app.py`

### Operations and observability

- [Audit logging](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/audit.py): `backend/app/audit.py`
- [Live event publisher](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/events.py): `backend/app/events.py`
- [Health and metrics](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/metrics.py): `backend/app/metrics.py`
- [Data retention](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/retention.py): `backend/app/retention.py`
- [Worker monitoring](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/workers.py): `backend/app/routers/workers.py`
- [Audit explorer](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/audit.py): `backend/app/routers/audit.py`
- [SSE event route](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/routers/events.py): `backend/app/routers/events.py`

### Storage and integrations

- [Webhook delivery engine](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/webhooks.py): `backend/app/webhooks.py`
- [URL safety checks](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/url_safety.py): `backend/app/url_safety.py`
- [PostgreSQL](https://github.com/rohannx29/pulse-job-orchestrator/blob/main/backend/app/database.py): `backend/app/database.py`
- **Redis**

### Other components

- **User / operator**
- **Webhook endpoints**

## Connections

- User / operator → Dashboard pages: operates
- Dashboard pages → Dashboard API client: requests data (evidence: `frontend/app/page.tsx`)
- Dashboard API client → FastAPI application: calls REST API
- Dashboard pages → Live event hook: subscribes (evidence: `frontend/app/page.tsx`)
- Live event hook → SSE event route: opens SSE (evidence: `frontend/lib/useEvents.ts`)
- SSE event route → Live event publisher: streams updates
- FastAPI application → Authentication and RBAC: routes requests
- FastAPI application → Org and project routes: routes requests
- FastAPI application → Job and queue routes: routes requests
- FastAPI application → Schedule routes: routes requests
- FastAPI application → Dead-letter routes: routes requests
- FastAPI application → Webhook management: routes requests
- Job and queue routes → Common job dispatch: submits jobs
- Schedule routes → Worker tasks: creates scheduled jobs (evidence: `backend/app/tasks.py`)
- Common job dispatch → Priority routing: maps priority
- Common job dispatch → Celery and Beat: queues work
- Celery and Beat → Worker tasks: dispatches tasks (evidence: `backend/app/tasks.py`)
- Worker tasks → Dependency DAG engine: resolves dependencies (evidence: `backend/app/tasks.py`)
- Worker tasks → Queue concurrency: limits execution (evidence: `backend/app/tasks.py`)
- Worker tasks → PostgreSQL: records execution (evidence: `backend/app/tasks.py`)
- Worker tasks → Redis: uses coordination
- Worker tasks → Data retention: runs pruning (evidence: `backend/app/tasks.py`)
- Webhook management → Webhook endpoints: tests delivery (evidence: `backend/app/routers/webhooks.py`)
- Webhook delivery engine → Webhook endpoints: sends signed events (evidence: `backend/app/webhooks.py`)
- Webhook delivery engine → URL safety checks: validates URLs (evidence: `backend/app/webhooks.py`)
- Webhook delivery engine → Worker tasks: enqueues delivery (evidence: `backend/app/webhooks.py`)
- Webhook delivery engine → PostgreSQL: records deliveries (evidence: `backend/app/webhooks.py`)
- Worker tasks → Live event publisher: publishes updates (evidence: `backend/app/tasks.py`)
- Live event publisher → Redis: publishes events
- FastAPI application → Audit logging: records actions
- Audit explorer → PostgreSQL: reads audit records
- Worker monitoring → PostgreSQL: reads worker state (evidence: `backend/app/routers/workers.py`)
- Health and metrics → PostgreSQL: reports readiness
