# riverqueue/river architecture

> Architecture diagram of the GitHub repository riverqueue/river, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/riverqueue/river
- Repository: https://github.com/riverqueue/river

## Overview

##### Project Type and Purpose
- River is a job processing system that tightly integrates with Postgres to provide transactional (enqueue‐with‐your‐data) job handling.
- It is used both as an in-process client that inserts jobs transactionally into the same database used for application data and as a background job processor that executes those jobs.
- The project supports advanced features like batch insertion, scheduled/periodic jobs, cancellation, graceful shutdown, and even registration of workers (via a “worker bundle”) for different job “kinds.”

##### Components to Capture
- Client & API:
– A central Client component that is instantiated with a database pool, driver, and configuration (with optional worker registration and queue configuration).
– An insert-only mode used for job enqueueing and another mode where jobs are executed.
- Job & Worker Logic:
– A Job abstraction, which includes job arguments (defined as structs with JSON annotations) and workers that implement the Work method.
– The registration process that matches job “kind” strings to their respective Worker implementations.
- Database & Drivers:
– Postgres as the primary persistent data store.
– Two important driver packages under “riverdriver”: one for a traditional SQL interface (riverdatabasesql) and another using pgx (riverpgxv5). These encapsulate the database operations including migrations, SQL generation (using sqlc), and low-level interactions.
- Maintenance & Scheduling:
– Several “internal/maintenance” submodules that manage job scheduling, clean-up, rescuing stuck jobs, periodic job enqueuing, and queue clean-up. This set of components is key for keeping the system healthy.
- Execution & Supervisory Services:
– Other internal packages handle the job executor (internal/jobexecutor), job completer (internal/jobcompleter), and even leadership/election (internal/leadership) for managing distributed job processing.
- Utilities and Helpers:
– A rich set of utilities exists for handling logging, error management, time/scheduling, and even test helpers. They provide cross-cutting support without being core to the job processing flow.
- Command Line Interfaces & Tools:
- External Integrations:
– The README mentions a web UI for job queue inspection and even cross-language enqueueing (e.g., a client library for Python or Ruby). These integrations may be added as peripheral components.

##### Diagrammatic Representation and Interactions
– Client Layer: Represent job insertion (both transactional and insert-only) and worker registration. Show that the Client interfaces with the database driver.
– Processing & Worker Layer: Illustrate the flow of jobs moving from the job queue to execution by registered workers, including data flow from the Client to the executor and worker logic.
– From the Client to Postgres (representing transactional job enqueueing).
– From the Postgres job queue to the Worker/Executor component (indicating dispatch and work retrieval).
– From the Executor to the Maintenance components that monitor and clean up failed or delayed jobs.
– From the Maintenance modules back to the Database (for rescheduling and re-indexing).

##### Architectural Patterns and Design Principles
- Separation of concerns by showing how the system divides responsibilities:
– Job management and worker execution are separated from maintenance and utility concerns.
– Database driver abstraction allows for plugging in different implementations (riverdatabasesql vs. riverpgxv5).
- Use of dependency injection in client creation (passing configuration, workers, and database drivers).
- Transactional integrity and the use of pipelines—jobs are enqueued and committed atomically with application data.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        A["Client API"]:::client
    end

    %% Job & Worker Layer
    subgraph "Job & Worker Layer"
        B["Job Logic"]:::job
        C["Worker Logic"]:::job
    end

    %% Database Layer
    subgraph "Database Layer"
        X["Driver Interface"]:::database
        D["SQL Driver (riverdatabasesql)"]:::database
        E["PGX Driver (riverpgxv5)"]:::database
        F["Postgres Database"]:::database
    end

    %% Maintenance Layer
    subgraph "Maintenance Layer"
        G["Maintenance Services"]:::maintenance
    end

    %% Execution Layer
    subgraph "Execution Layer"
        H["Job Executor"]:::execution
        I["Job Completer"]:::execution
        J["Leadership Election"]:::execution
    end

    %% CLI & Tools Layer
    subgraph "CLI & Tools Layer"
        K["Command Line Tools"]:::cli
    end

    %% Utilities Layer
    subgraph "Utilities Layer"
        L["Internal Utilities"]:::utility
        N["Rivershared Utilities"]:::utility
    end

    %% External Interfaces
    subgraph "External Interfaces"
        M["External Integrations"]:::external
    end

    %% Connections
    A -->|"configures"| X
    A -->|"enqueue"| F
    X -->|"uses"| D
    X -->|"uses"| E
    F -->|"dispatches"| H
    H -->|"executes"| B
    H -->|"executes"| C
    F <-->|"maintenance"| G
    G -->|"monitors"| H
    A -->|"registers"| H
    K -->|"triggers"| A
    M -->|"inspects"| F
    F -->|"reports"| M
    L -->|"supports"| A
    L -->|"supports"| H
    L -->|"supports"| F
    N -->|"supports"| A
    N -->|"supports"| H
    N -->|"supports"| F

    %% Click Events
    click A "https://github.com/riverqueue/river/blob/master/client.go"
    click B "https://github.com/riverqueue/river/blob/master/job.go"
    click C "https://github.com/riverqueue/river/blob/master/worker.go"
    click X "https://github.com/riverqueue/river/blob/master/riverdriver/river_driver_interface.go"
    click D "https://github.com/riverqueue/river/tree/master/riverdriver/riverdatabasesql"
    click E "https://github.com/riverqueue/river/tree/master/riverdriver/riverpgxv5"
    click G "https://github.com/riverqueue/river/tree/master/internal/maintenance"
    click H "https://github.com/riverqueue/river/blob/master/internal/jobexecutor/job_executor.go"
    click I "https://github.com/riverqueue/river/blob/master/internal/jobcompleter/job_completer.go"
    click J "https://github.com/riverqueue/river/blob/master/internal/leadership/elector.go"
    click K "https://github.com/riverqueue/river/tree/master/cmd/river"
    click L "https://github.com/riverqueue/river/tree/master/internal/util"
    click N "https://github.com/riverqueue/river/tree/master/rivershared/util"

    %% Styles
    classDef client fill:#bbdefb,stroke:#0d47a1,stroke-width:2px;
    classDef job fill:#c8e6c9,stroke:#1b5e20,stroke-width:2px;
    classDef database fill:#dcedc8,stroke:#33691e,stroke-width:2px;
    classDef maintenance fill:#ffe0b2,stroke:#e65100,stroke-width:2px;
    classDef execution fill:#f8bbd0,stroke:#880e4f,stroke-width:2px;
    classDef cli fill:#d1c4e9,stroke:#512da8,stroke-width:2px;
    classDef utility fill:#f0f4c3,stroke:#827717,stroke-width:2px;
    classDef external fill:#b2ebf2,stroke:#006064,stroke-width:2px;
```
