Laravel Horizon repeating completed jobs when using Redis with igbinary serializer

[adnan-smlatic]

Horizon Version

5.30

Laravel Version

11.34.2

PHP Version

8.3

Redis Driver

PhpRedis

Redis Version

7.2.7

Database Driver & Version

No response

Description

I experienced a very strange behaviour when using Laravel Horizon on Redis with igbinary serialization.

It would complete the jobs normally, but it seems like it doesn't clear them from Redis properly, so after a minute or so it starts doing the same jobs in an infinite loop.

  2025-02-03 12:44:27 App\Mail\OrderConfirmation ..................... RUNNING
  2025-02-03 12:44:28 App\Mail\OrderConfirmation ............... 663.47ms DONE
  2025-02-03 12:44:28 App\Mail\OrderAdminConfirmation ................ RUNNING
  2025-02-03 12:44:28 App\Mail\OrderAdminConfirmation .......... 241.97ms DONE
  2025-02-03 12:45:59 App\Mail\OrderConfirmation ..................... RUNNING
  2025-02-03 12:45:59 App\Mail\OrderConfirmation ................. 8.91ms FAIL
  2025-02-03 12:45:59 App\Mail\OrderAdminConfirmation ................ RUNNING
  2025-02-03 12:45:59 App\Mail\OrderAdminConfirmation ............ 2.31ms FAIL

The serialized data looks very weird in Redis, with some of it being serialized and some not. Probably the reason it can't identify them properly?

Eg. snippet
{"completed_at":"\u0000\u0000\u0000\u0002\u0011\u000e1738583159.951","status":"\u0000\u0000\u0000\u0002\u0011\u0006failed","failed_at":"\u0000\u0000\u0000\u0002\u0011\u000f1738583159.9526","payload":"\u0000\u0000\u0000\u0002\u0012\u0004ß{\"backoff\":null,\"uuid\":\"ab7687bd-aed4-4e73-ad28-5a80f8e541a4\",\"timeout\":null,\"tags\":[\"App\\\\Models\\\\Order:2498\"],\"id\":\"ab7687bd-aed4-4e73-ad28-5a80f8e541a4\",\"job\":\"Illuminate\\\\Queue\\\\CallQueuedHandler@call\",\"pushedAt\":\"1738583067.1348\",\"maxExceptions\":null,\"maxTries\":null,\"data\":{\"command\":\"O:34:\\\"Illuminate\\\\Mail\\\\SendQueuedMailable\\

In my database.php, I had set the serializer to "2", which is igbinary. Commenting it out immediately solved the issue.

    'redis' => [

        'client' => env('REDIS_CLIENT', 'phpredis'),

        'options' => [
            'cluster' => env('REDIS_CLUSTER', 'redis'),
            // 'serializer' =>  2,
            'prefix' => env('REDIS_PREFIX', Str::slug(env('APP_NAME', 'laravel'), '_').'_database_'),
        ],

I am not sure if I was supposed to set the Horizon to explicitly user igbinary somewhere? Or if I made a mistake somewhere during my config.

This was an absolute headache to debug and find the reason.

Steps To Reproduce

Install the above versions of Laravel, Horizon and Redis. Make sure redis and igbinary is installed

Check:
php -m | grep igbinary
igbinary

php -m | grep redis
redis

Set the env

QUEUE_CONNECTION=redis
CACHE_DRIVER=redis
REDIS_CLIENT=phpredis

In database config set the redis serializer to 2

'redis' => [

    'client' => env('REDIS_CLIENT', 'phpredis'),

    'options' => [
        'serializer' =>  2,

run php artisan horizon

Dispatch any job

[github-actions]

Thank you for reporting this issue!

As Laravel is an open source project, we rely on the community to help us diagnose and fix issues as it is not possible to research and fix every issue reported to us via GitHub.

If possible, please make a pull request fixing the issue you have described, along with corresponding tests. All pull requests are promptly reviewed by the Laravel team.

Thank you!

[PhilippeThouvenot]

According to the documentation : https://laravel.com/docs/12.x/queues#connections-vs-queues
You can't use serializer :
The serializer and compression Redis options are not supported by the redis queue driver.

You need to set an another connection without serializer for queue/horizon.