# sujith0613/craxpert architecture

> Architecture diagram of the GitHub repository sujith0613/craxpert, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sujith0613/craxpert
- Repository: https://github.com/sujith0613/craxpert

## Overview

1. **System shape**
   - Craxpert is a single-process Go service with two main faces: a backend job simulator and a read-only browser dashboard.
   - There is no database; durability comes from an append-only write-ahead log (`jobs.log`) plus periodic checkpointing.

2. **Entry point and server bootstrap**
   - `main.go` is the orchestration layer: it starts the HTTP server, boots any restored state from disk, and likely wires together the auto job generator and background checkpoint worker.
   - The backend runs on Go’s standard `net/http` stack.

3. **Job domain model and shared in-memory state**
   - `job.go` holds the core job representation, job states, and the global job registry/map.
   - The in-memory job store is protected with `sync.RWMutex`, which is central to concurrency safety across handlers, workers, and reset/reload paths.

4. **Worker pool and execution pipeline**
   - `worker.go` contains the concurrent worker-pool logic that consumes jobs, simulates work, and handles retries.
   - Jobs transition through states such as `PENDING`, `RUNNING`, `SUCCESS`, `FAILED`, and `RETRYING`, with workers updating shared state as they execute.

5. **Persistence boundary: WAL**
   - `wal.go` is the durability layer: every job state transition is serialized to disk before or during execution, so the simulator can recover after crashes.
   - The WAL is append-only and is the authoritative recovery source on startup.

6. **Checkpointing / compaction**
   - The readme describes a background checkpoint process that periodically snapshots active jobs and compacts the WAL.
   - This likely uses a staging file such as `jobs.tmp` before replacing the live log, reducing replay time on restart.

7. **Job ingestion paths**
   - Jobs enter the system from two sources: the HTTP API and an automatic generator.
   - The API accepts manual job submission, while the auto-generator continuously creates simulated workload for the worker pool.

8. **HTTP API and control plane**
   - The server exposes endpoints for job submission and operational controls such as reset, pool resizing, cancellation, and status retrieval.
   - These endpoints are the main boundary between external clients and internal job state.

9. **Frontend dashboard**
   - `web/index.html` is a single-page, vanilla HTML/CSS/JavaScript dashboard.
   - It is not the control source of truth; it mostly polls backend status and visualizes the job queue, worker activity, and log history.

10. **Observability surfaces**
   - The dashboard exposes live queue state, raw WAL/log viewing, and connection monitoring.
   - The terminal monitor described in the README is a secondary observability surface for local runtime inspection.

11. **Utility layer**
   - `utils.go` is a small support module, likely housing random generation and helper functions used by job creation or simulation timing.
   - It is not a major subsystem, but it supports the workload generator and worker simulation.

12. **Key architectural boundary**
   - The important design split is: ephemeral concurrent execution in memory versus durable state in the WAL.
   - All meaningful state transitions flow from API or generator -> in-memory job registry/workers -> WAL append -> dashboard/status consumers, with startup recovery replaying the log back into memory.

Main entry points: [`job.go`](https://github.com/sujith0613/craxpert/blob/main/job.go) (In-memory state), [`worker.go`](https://github.com/sujith0613/craxpert/blob/main/worker.go) (Workers), [`main.go`](https://github.com/sujith0613/craxpert/blob/main/main.go) (Main), [`wal.go`](https://github.com/sujith0613/craxpert/blob/main/wal.go) (WAL)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_main["Main<br/>bootstrap<br/>[main.go]"]
  node_job["Job model<br/>domain<br/>[job.go]"]
  node_worker["Workers<br/>execution<br/>[worker.go]"]
  node_api["HTTP API<br/>control plane<br/>[main.go]"]
  node_generator(("Auto generator<br/>workload source<br/>[main.go]"))
  node_state[("In-memory state<br/>concurrent store<br/>[job.go]")]
  node_status["Status view<br/>read model<br/>[main.go]"]
  node_cancel["Cancellation<br/>control<br/>[main.go]"]
  node_resize["Pool resize<br/>control<br/>[main.go]"]
  node_utils{{"Utils<br/>helpers<br/>[utils.go]"}}
end

subgraph group_durability["Durability"]
  node_wal["WAL<br/>append-only log<br/>[wal.go]"]
  node_checkpoint["Checkpointing<br/>compaction<br/>[wal.go]"]
  node_restore["Recovery<br/>replay<br/>[wal.go]"]
end

subgraph group_ui["Browser UI"]
  node_dashboard["Dashboard<br/>browser UI<br/>[index.html]"]
  node_logview["Log view<br/>observability<br/>[index.html]"]
end

node_monitor(("Terminal monitor<br/>observability"))

node_main -->|"serves"| node_api
node_main -->|"starts"| node_worker
node_main -->|"starts"| node_generator
node_main -->|"restores"| node_restore
node_api -->|"mutates"| node_state
node_generator -->|"creates"| node_state
node_worker -->|"updates"| node_state
node_worker -->|"appends"| node_wal
node_api -->|"records"| node_wal
node_restore -->|"rebuilds"| node_state
node_checkpoint -->|"compacts"| node_wal
node_checkpoint -->|"snapshots"| node_state
node_dashboard -->|"polls"| node_status
node_dashboard -->|"shows"| node_logview
node_status -->|"reads"| node_state
node_cancel -->|"marks"| node_state
node_resize -->|"reconfigures"| node_worker
node_utils -->|"supports"| node_generator
node_utils -->|"supports"| node_worker
node_monitor -.->|"observes"| node_state
node_monitor -.->|"observes"| node_wal

click node_main "https://github.com/sujith0613/craxpert/blob/main/main.go"
click node_job "https://github.com/sujith0613/craxpert/blob/main/job.go"
click node_worker "https://github.com/sujith0613/craxpert/blob/main/worker.go"
click node_api "https://github.com/sujith0613/craxpert/blob/main/main.go"
click node_generator "https://github.com/sujith0613/craxpert/blob/main/main.go"
click node_state "https://github.com/sujith0613/craxpert/blob/main/job.go"
click node_wal "https://github.com/sujith0613/craxpert/blob/main/wal.go"
click node_checkpoint "https://github.com/sujith0613/craxpert/blob/main/wal.go"
click node_restore "https://github.com/sujith0613/craxpert/blob/main/wal.go"
click node_status "https://github.com/sujith0613/craxpert/blob/main/main.go"
click node_cancel "https://github.com/sujith0613/craxpert/blob/main/main.go"
click node_resize "https://github.com/sujith0613/craxpert/blob/main/main.go"
click node_dashboard "https://github.com/sujith0613/craxpert/blob/main/web/index.html"
click node_logview "https://github.com/sujith0613/craxpert/blob/main/web/index.html"
click node_utils "https://github.com/sujith0613/craxpert/blob/main/utils.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_job,node_worker,node_api,node_generator,node_state,node_status,node_cancel,node_resize,node_utils toneBlue
class node_wal,node_checkpoint,node_restore toneAmber
class node_dashboard,node_logview toneMint
class node_monitor toneNeutral
```

## Components

### Runtime

Process startup, HTTP serving, job execution, and shared in-memory coordination.

- [Main](https://github.com/sujith0613/craxpert/blob/main/main.go): bootstrap · Starts the server, restores state, and wires background loops. · `main.go`
- [Job model](https://github.com/sujith0613/craxpert/blob/main/job.go): domain · Core job types, states, and shared registry. · `job.go`
- [Workers](https://github.com/sujith0613/craxpert/blob/main/worker.go): execution · Concurrent worker pool that simulates job processing and retries. · `worker.go`
- [HTTP API](https://github.com/sujith0613/craxpert/blob/main/main.go): control plane · Endpoints for job submission and operational control. · `main.go`
- [Auto generator](https://github.com/sujith0613/craxpert/blob/main/main.go): workload source · Continuously creates simulated jobs to feed the workers. · `main.go`
- [In-memory state](https://github.com/sujith0613/craxpert/blob/main/job.go): concurrent store · Shared job registry protected by read/write locking. · `job.go`
- [Status view](https://github.com/sujith0613/craxpert/blob/main/main.go): read model · Current queue, worker, and job-state view exposed to clients. · `main.go`
- [Cancellation](https://github.com/sujith0613/craxpert/blob/main/main.go): control · Stops or marks jobs through operational endpoints. · `main.go`
- [Pool resize](https://github.com/sujith0613/craxpert/blob/main/main.go): control · Adjusts worker concurrency at runtime. · `main.go`
- [Utils](https://github.com/sujith0613/craxpert/blob/main/utils.go): helpers · Shared helper functions for generation and simulation support. · `utils.go`

### Durability

Append-only log and checkpointing that restore state after restart.

- [WAL](https://github.com/sujith0613/craxpert/blob/main/wal.go): append-only log · Durable record of job transitions used for crash recovery. · `wal.go`
- [Checkpointing](https://github.com/sujith0613/craxpert/blob/main/wal.go): compaction · Snapshots active jobs and compacts the write-ahead log. · `wal.go`
- [Recovery](https://github.com/sujith0613/craxpert/blob/main/wal.go): replay · Boot-time log replay that rebuilds in-memory state. · `wal.go`

### Browser UI

Read-only dashboard and operational visibility into the running simulator.

- [Dashboard](https://github.com/sujith0613/craxpert/blob/main/web/index.html): browser UI · Single-page interface that polls backend state and renders activity. · `web/index.html`
- [Log view](https://github.com/sujith0613/craxpert/blob/main/web/index.html): observability · Browser-facing display of history and WAL activity. · `web/index.html`

### Other components

- **Terminal monitor**: observability · Local runtime inspection surface mentioned by the README.

## Connections

- Main → HTTP API: serves · Bootstraps the HTTP control plane.
- Main → Workers: starts · Launches the execution pool.
- Main → Auto generator: starts · Runs the automatic workload generator.
- Main → Recovery: restores · Rebuilds state from the durable log on startup.
- HTTP API → In-memory state: mutates · Creates, cancels, and manages jobs through the shared registry.
- Auto generator → In-memory state: creates · Injects new jobs into memory for execution.
- Workers → In-memory state: updates · Transitions jobs through execution states.
- Workers → WAL: appends · Persists state transitions for recovery.
- HTTP API → WAL: records · Operational changes are written durably.
- Recovery → In-memory state: rebuilds · Replays log entries back into memory.
- Checkpointing → WAL: compacts · Rewrites the log after snapshotting active jobs.
- Checkpointing → In-memory state: snapshots · Captures active in-memory jobs for compaction.
- Dashboard → Status view: polls · Reads live status from the backend.
- Dashboard → Log view: shows · Visualizes queue and log history.
- Status view → In-memory state: reads · Exposes the current job registry without mutation.
- Cancellation → In-memory state: marks · Changes job lifecycle state via control endpoints.
- Pool resize → Workers: reconfigures · Changes worker pool size at runtime.
- Utils → Auto generator: supports · Provides helper logic for job creation and timing.
- Utils → Workers: supports · Provides helper logic for simulation behavior.
- Terminal monitor → In-memory state: observes · Secondary local visibility into runtime activity.
- Terminal monitor → WAL: observes · Can inspect durability history.