Skip to content

Workers can stream results to Consumer Group subscribers for results from a Queue #16

@codecakes

Description

@codecakes

Many requesting clients should be able to access results from the same queue. This is not possible using the redis-queue structure.

The nature of a queue is that each item can only be dequeued (or popped) once. Once an item is dequeued by a client, it is removed from the queue and no longer available for other clients to dequeue.

This proposal should allow multiple clients to subscribe to a consumer group just like using redis streams yet all of them can fetch the result from the enqueued job.

As a worker client, Given #15,
When a job for a queue is alloted for which that worker is assigned,
The worker should store the job_id|worker_id|timestamp into redis,
So that it can:

  1. process that job AND
  2. synchronously fetch results from redis AND
  3. stream it to a redis stream key which is a consumer group.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    📋 Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions