# gocraft/work architecture

> Architecture diagram of the GitHub repository gocraft/work, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gocraft/work
- Repository: https://github.com/gocraft/work

## Overview

##### Project Type and Purpose
- The project is a background job processing framework for Go. Its purpose is to enqueue jobs (with optional scheduling and uniqueness), process them with a configurable WorkerPool and middleware, and manage state with Redis.
- It is similar to Sidekiq (for Ruby) and includes a Web UI for monitoring, middleware support for metrics and logging, job scheduling (cron/periodic), retries, and dead-letter handling.

##### Analysis of the File Structure
- Top-level files like client.go, enqueue.go, job.go, worker.go, and the “cmd” folder (containing executables such as workenqueue, workfakedata, workwebui) clearly define several sub-systems:
- Enqueuer/Client components that add jobs to queues.
- WorkerPool and Worker modules that process jobs. Files such as worker_pool.go, worker.go, and various job tests indicate the core processing logic.
- Support utilities/logic (e.g., heartbeater.go, requeuer.go, dead_pool_reaper.go) that ensure reliability in processing and recovery from failures.
- The “webui” directory contains assets and handlers for the Web UI. It suggests a separate visual management component exposing job status and administrative functions.
- Benchmarks (in the benches directory) demonstrate performance testing of the enqueuing mechanism.

- Client/Enqueuer:
- Represents clients that initiate jobs. This component (using client.go, enqueue.go) exposes an API for job submission.

- Redis (Persistent Job Store):
- Jobs are stored in Redis (via various scripts and keys). Represent Redis as a separate system.
- Multiple Redis key types: one for normal job queues (list-based), retry queues and scheduled jobs (z-sets), and dead job queues.

- WorkerPool & Workers:
- The WorkerPool (worker_pool.go, workwebui.go) spins up worker goroutines that dequeue jobs from Redis.
- Each Worker works independently with a process loop that uses middleware (e.g., Log, FindCustomer) and job handlers (e.g., SendEmail).

- Scheduling and Retry Mechanisms:
- Components for the periodic enqueuer (periodic_enqueuer.go), requeuer (requeuer.go), and Dead Pool Reaper (dead_pool_reaper.go) that handle scheduling future jobs, requeueing timed-out or failed jobs, and cleaning up dead ones.

- Web UI:
- The webui component (webui/webui.go and its internal assets) serves as the administrative and monitoring interface showing job states, retries, dead jobs, etc.

- Middleware & Context:

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client/API Layer
    subgraph "Client/API Layer"
        C1("Job Client"):::client
        C2("Job Enqueuer"):::client
    end

    %% Core Processing
    subgraph "Core Processing"
        R("Redis"):::redis
        subgraph "Worker Processing"
            WP("WorkerPool"):::worker
            W("Worker(s)"):::worker
        end
        subgraph "Scheduling & Retry"
            PE("Periodic Enqueuer"):::scheduler
            RE("Requeuer"):::scheduler
            DPR("Dead Pool Reaper"):::scheduler
        end
    end

    %% Web UI
    subgraph "Web UI"
        WUI("Web UI"):::webui
        WA("Web Assets"):::webui
        CMD("WebUI Executable"):::webui
    end

    %% Connections with red arrows
    C1 -->|"EnqueueJob"| R
    C2 -->|"EnqueueJob"| R
    R -->|"DequeueJob"| WP
    WP -->|"DelegateJob"| W
    PE -->|"ScheduleJob"| R
    RE -->|"RequeueJob"| R
    DPR -->|"CleanDead"| R
    R -->|"QueryJobStatus"| WUI
    WUI -->|"AdminActions"| R

    %% Click Events
    click C1 "https://github.com/gocraft/work/blob/master/client.go"
    click C2 "https://github.com/gocraft/work/blob/master/enqueue.go"
    click R "https://github.com/gocraft/work/blob/master/redis.go"
    click WP "https://github.com/gocraft/work/blob/master/worker_pool.go"
    click W "https://github.com/gocraft/work/blob/master/worker.go"
    click PE "https://github.com/gocraft/work/blob/master/periodic_enqueuer.go"
    click RE "https://github.com/gocraft/work/blob/master/requeuer.go"
    click DPR "https://github.com/gocraft/work/blob/master/dead_pool_reaper.go"
    click WUI "https://github.com/gocraft/work/blob/master/webui/webui.go"
    click WA "https://github.com/gocraft/work/tree/master/webui/internal/assets"
    click CMD "https://github.com/gocraft/work/blob/master/cmd/workwebui/main.go"

    %% Link Styles for red arrows
    linkStyle 0 stroke:red,stroke-width:2px
    linkStyle 1 stroke:red,stroke-width:2px
    linkStyle 2 stroke:red,stroke-width:2px
    linkStyle 3 stroke:red,stroke-width:2px
    linkStyle 4 stroke:red,stroke-width:2px
    linkStyle 5 stroke:red,stroke-width:2px
    linkStyle 6 stroke:red,stroke-width:2px
    linkStyle 7 stroke:red,stroke-width:2px
    linkStyle 8 stroke:red,stroke-width:2px

    %% Class Definitions for colors
    classDef client fill:#ffcccc,stroke:#ff0000,stroke-width:2px;
    classDef redis fill:#ffe6e6,stroke:#ff0000,stroke-width:2px;
    classDef worker fill:#ffdddd,stroke:#ff0000,stroke-width:2px;
    classDef scheduler fill:#fff0f0,stroke:#ff0000,stroke-width:2px;
    classDef webui fill:#ffe6e6,stroke:#ff0000,stroke-width:2px;
```
