|
| 1 | +# relay |
| 2 | + |
| 3 | +A simple, reliable, distributed job scheduler for Go, backed by Redis. |
| 4 | + |
| 5 | +Enqueue background jobs from anywhere, process them across a fleet of workers, and schedule recurring work with cron. Jobs survive worker crashes and are retried automatically. |
| 6 | + |
| 7 | +## Features |
| 8 | + |
| 9 | +- At-least-once delivery with automatic crash recovery (visibility-timeout leases) |
| 10 | +- Immediate, delayed (`ProcessIn` / `ProcessAt`), and recurring **cron** jobs |
| 11 | +- Automatic retries with exponential backoff and a dead-letter archive |
| 12 | +- Optional task deduplication (`Unique`) and explicit task IDs |
| 13 | +- Weighted priority queues with pause/resume |
| 14 | +- Per-task timeouts, deadlines, and remote cancellation |
| 15 | +- Prometheus metrics out of the box |
| 16 | +- A CLI to inspect and manage queues |
| 17 | + |
| 18 | +## Install |
| 19 | + |
| 20 | +```sh |
| 21 | +go get github.com/MohamedAklamaash/relay |
| 22 | +``` |
| 23 | + |
| 24 | +Requires Go 1.24+ and a Redis server. |
| 25 | + |
| 26 | +## Quickstart |
| 27 | + |
| 28 | +Enqueue a task: |
| 29 | + |
| 30 | +```go |
| 31 | +client := relay.NewClient(relay.RedisClientOpt{Addr: "127.0.0.1:6379"}) |
| 32 | +defer client.Close() |
| 33 | + |
| 34 | +client.Enqueue(relay.NewTask("email:send", []byte(`{"to":"a@b.com"}`))) |
| 35 | +client.Enqueue(relay.NewTask("report:build", nil), relay.ProcessIn(10*time.Minute)) |
| 36 | +``` |
| 37 | + |
| 38 | +Process tasks: |
| 39 | + |
| 40 | +```go |
| 41 | +srv := relay.NewServer(relay.RedisClientOpt{Addr: "127.0.0.1:6379"}, relay.Config{ |
| 42 | + Concurrency: 10, |
| 43 | + Queues: map[string]int{"critical": 6, "default": 3, "low": 1}, |
| 44 | +}) |
| 45 | + |
| 46 | +mux := relay.NewServeMux() |
| 47 | +mux.HandleFunc("email:send", func(ctx context.Context, t *relay.Task) error { |
| 48 | + return sendEmail(t.Payload()) |
| 49 | +}) |
| 50 | + |
| 51 | +srv.Run(mux) |
| 52 | +``` |
| 53 | + |
| 54 | +Schedule recurring work: |
| 55 | + |
| 56 | +```go |
| 57 | +scheduler := relay.NewScheduler(relay.RedisClientOpt{Addr: "127.0.0.1:6379"}, nil) |
| 58 | +scheduler.Register("0 * * * *", relay.NewTask("report:build", nil)) |
| 59 | +scheduler.Run() |
| 60 | +``` |
| 61 | + |
| 62 | +## Enqueue options |
| 63 | + |
| 64 | +| Option | Description | |
| 65 | +| --- | --- | |
| 66 | +| `Queue(name)` | Target queue (default `"default"`) | |
| 67 | +| `ProcessIn(d)` / `ProcessAt(t)` | Delay execution | |
| 68 | +| `MaxRetry(n)` | Retry attempts before archiving (default 25) | |
| 69 | +| `Timeout(d)` / `Deadline(t)` | Cancel the handler context after a limit | |
| 70 | +| `Unique(ttl)` | Reject duplicate enqueues within the window | |
| 71 | +| `TaskID(id)` | Set an explicit ID; conflicting IDs are rejected | |
| 72 | +| `Retention(d)` | Keep completed task data for inspection | |
| 73 | + |
| 74 | +## Delivery semantics |
| 75 | + |
| 76 | +`relay` guarantees **at-least-once** delivery. A task may run more than once (for |
| 77 | +example when a worker crashes after finishing but before acknowledging), so your |
| 78 | +handlers must be **idempotent**. `Unique` deduplicates *enqueues*, not executions. |
| 79 | + |
| 80 | +Return `relay.SkipRetry` from a handler to archive a task immediately instead of |
| 81 | +retrying. |
| 82 | + |
| 83 | +## Metrics |
| 84 | + |
| 85 | +`relay` exposes Prometheus metrics (processed, failed, retried, archived, |
| 86 | +in-progress, and processing duration). Mount the handler on your own server: |
| 87 | + |
| 88 | +```go |
| 89 | +http.Handle("/metrics", relay.MetricsHandler()) |
| 90 | +go http.ListenAndServe(":2112", nil) |
| 91 | +``` |
| 92 | + |
| 93 | +## Cancellation |
| 94 | + |
| 95 | +Cancel a running task from anywhere; the worker's handler context is cancelled: |
| 96 | + |
| 97 | +```go |
| 98 | +insp := relay.NewInspector(relay.RedisClientOpt{Addr: "127.0.0.1:6379"}) |
| 99 | +insp.CancelTask(taskID) |
| 100 | +``` |
| 101 | + |
| 102 | +## CLI |
| 103 | + |
| 104 | +```sh |
| 105 | +go install github.com/MohamedAklamaash/relay/cmd/relay@latest |
| 106 | + |
| 107 | +relay stats |
| 108 | +relay ls default retry |
| 109 | +relay run default <task-id> |
| 110 | +relay rm default <task-id> |
| 111 | +relay cancel <task-id> |
| 112 | +relay pause low |
| 113 | +``` |
| 114 | + |
| 115 | +## Running locally |
| 116 | + |
| 117 | +```sh |
| 118 | +docker compose up -d redis |
| 119 | +go run ./examples/worker |
| 120 | +go run ./examples/producer |
| 121 | +go run ./examples/cron |
| 122 | +``` |
| 123 | + |
| 124 | +## License |
| 125 | + |
| 126 | +MIT |
0 commit comments