tastyware/streaq:v5.1.0

What's Changed

• no context manager for enqueuing by @Graeme22 in #69

Full Changelog: v5.0.1...v5.1.0

tastyware/streaq:v5.0.1

What's Changed

• fixes bug where try count wasn't displayed in web UI
• web UI now allows sorting tasks by queue time in ascending or descending order

Full Changelog: v5.0.0...v5.0.1

tastyware/streaq:v5.0.0

What's Changed

• Add missing Jinja2 dependency to extra "web" dependencies by @espdev in #63
• Do not configure logging level by default by @espdev in #64
• breaking changes for v5 by @Graeme22 in #62

Full Changelog: v4.1.1...v5.0.0

tastyware/streaq:v4.1.1

What's Changed

• Fix exposed port for Redis container in tests by @espdev in #60
• change pubsub to fix #57 by @Graeme22 in #61
  Getting results for a task with ttl=0 will now work instead of raising an error (as long as the call happens before the task finishes!)

Full Changelog: v4.1.0...v4.1.1

tastyware/streaq:v4.1.0

What's Changed

• use coredis v5 pipelines by @Graeme22 in #51
  Coredis dependency updated to new v5 release, which allows for maintaining type hints even when pipelining. Additionally, the TaskResult object now contains additional information: worker_id, the worker which ran the task, and task_try, the total number of tries. The web UI includes this information.
• Add possibility to set a custom name for tasks and cron jobs by @espdev in #54
  You can now specify a custom name when registering tasks/cron jobs. This is especially helpful when task definitions are nested or when using Worker.enqueue_unsafe():

  @worker.task(name="bar")
  async def foo(): ...

  The health check cron job run in the background is now called redis_health_check.
• Сheck main_task attribute when closing worker by @espdev in #53
• Test coverage increased in several places
• CLI now raises an error if you try to run with --web and the web module isn't installed

Full Changelog: v4.0.4...v4.1.0

tastyware/streaq:v4.0.4

What's Changed

• Enqueued task data no longer expires. Task data remains in Redis indefinitely until the task succeeds or fails, just like the stream/delayed queue entries.
• Lua scripts refactored to make code a bit cleaner
• Worker internal property names refactored
• Task.info() and Worker.info_by_id() now return None if the task has finished or doesn't exist instead of raising an exception
• Fixed rare bug where idle timeout was too short inside run_task()

Full Changelog: v4.0.3...v4.0.4

tastyware/streaq:v4.0.3

What's Changed

• Fix incorrect logic for logging related to new redis_kwargs parameter

Full Changelog: v4.0.2...v4.0.3

tastyware/streaq:v4.0.2

What's Changed

• Fix links and redirects in Web UI by @espdev in #49
  Mounting web UI as part of an existing FastAPI app with a different prefix now working correctly.
• fix #48, add redis_kwargs in 584072a by @Graeme22
  • Crontab bumped to v1.0.5 to support "z" syntax in crontabs
  • Worker now accepts redis_kwargs parameter which will be passed to Redis client on initialization

New Contributors

• @espdev made their first contribution in #49

Full Changelog: v4.0.1...v4.0.2

tastyware/streaq:v4.0.1

What's Changed

• fix bug when calling CLI with multiple workers #45

Full Changelog: v4.0.0...v4.0.1

tastyware/streaq:v4.0.0

What's Changed

• begin adding web UI by @Graeme22 in #30
  New major release! Plenty of breaking changes, but most should be welcome.

  • Simple web UI added for monitoring tasks. Try it with streaq test.worker --web! The UI is written with FastAPI, Bootstrap and HTMX and implemented in a way that allows easily mounting it onto a separate FastAPI app. (You'll need to install with the web extra: pip install -U streaq[web])

    

  • Task priorities have been completely rewritten. Previously, there was an enum of 3 hard-coded priorities, TaskPriority, which has been replaced with a Worker parameter, priorities, which allows for creating an arbitrary number of user-defined priorities and defaults to a single priority for those who don't need this functionality. Additionally, delayed tasks can be given a priority, which wasn't possible before.

    # this list should be ordered from lowest to highest
    worker = Worker(priorities=["low", "high"])
    
    async with worker:
        await sleeper.enqueue(3).start(priority="low")

  • Use of pickle.loads, the default deserializer, is a security risk as an attacker who gains access to the Redis database would be able to run arbitrary code. You can now protect against this attack vector by passing a signing_secret to the worker. The signing key ensures corrupted data from Redis will not be unpickled.

    worker = Worker(signing_secret="MY-SECRET-KEY")

  • Reliability and pessimistic execution improved across the entire codebase. A new parameter, Worker.idle_timeout, allows for detecting stale tasks that have been prefetched (and sometimes even begun execution). Tasks are more easily reclaimed when workers are shut down incorrectly and several edge cases have been eliminated. Prefetched tasks are now returned immediately to the queue on worker shutdown.

  • New function added, Worker.enqueue_many(), which allows for batching together enqueue calls into a single Redis pipeline for greater efficiency.

    async with worker:
        # importantly, we're not using `await` here
        tasks = [sleeper.enqueue(i) for i in range(10)]
        await worker.enqueue_many(tasks)

  • Many Lua scripts have been made more efficient, and several scripts have been combined to further boost efficiency.

  • TaskData has been renamed to TaskInfo and now includes additional properties dependents and dependencies.

  • TaskResult now includes additional properties fn_name and enqueue_time, and no longer includes queue_name which was redundant.

  • Task.then() typing improved

  • Worker.queue_fetch_limit renamed to Worker.prefetch. Prefetching can be disabled entirely by setting this to 0.

  • Many exceptions now contain additional stack trace info

  • Tests now use separate queue names for better isolation

  • Worker.with_scheduler has been eliminated, as listen_queue and listen_stream have been combined into a single, more efficient loop, meaning all workers can schedule tasks. Polling is now eliminated entirely from the library.

  • Several internal worker functions have been combined into more efficient versions

  • Worker shutdown no longer kills tasks prematurely in some circumstances

  • Task abortion now works immediately for delayed tasks that are still enqueued

Full Changelog: v3.0.0...v4.0.0