Skip to content

Support custom serialization methods for message queue packets #755

Open
@miguelgrinberg

Description

@miguelgrinberg
Owner

Currently, pickle is used, without an easy option to change it.

Activity

cfal

cfal commented on Jul 27, 2021

@cfal

this would be useful for our usecase, in a hacky way :)

we currently use RedisManager and for many messages, we need to send the same object to different rooms. the objects can be quite large, and currently in RedisManager._publish the same object gets pickled once every time we call emit (since PubSubManager only supports a single room).

if this was added, we could pre-pickle the data before passing it to emit, and then make pickle.dumps be a no-op. a slightly cleaner solution might be to allow multiple rooms to be passed into RedisManager.emit, but that might be confusing for people that don't realize multiple messages still have to be sent..

miguelgrinberg

miguelgrinberg commented on Jul 27, 2021

@miguelgrinberg
OwnerAuthor

@cfal for any custom needs, you can subclass RedisManager and replace the serialization logic with anything you like. I don't think what I'm planning here will help your case, which is , in my view, not a common use case for this package.

cfal

cfal commented on Jul 27, 2021

@cfal

thanks! we thought about doing this, but there's no great way to do that at the moment, because the serialization logic is inside the _publish method: https://github.com/miguelgrinberg/python-socketio/blob/main/src/socketio/redis_manager.py#L74-L87

it seems like we'd have to copy over the redis logic in _publish as well, which might be ok, but make upgrades a bit scarier..

mbwmbw1337

mbwmbw1337 commented on Oct 28, 2021

@mbwmbw1337

Hi All -

Any updates on this? Anywhere I might be able to help with my limited programming skills? Would love to be able to implement msgpack-python.

miguelgrinberg

miguelgrinberg commented on Oct 28, 2021

@miguelgrinberg
OwnerAuthor

@mbwmbw1337 I'm not actively working on this right now. I intend to add this at some point, but there are other things I want to do first.

mbwmbw1337

mbwmbw1337 commented on Oct 28, 2021

@mbwmbw1337

@mbwmbw1337 I'm not actively working on this right now. I intend to add this at some point, but there are other things I want to do first.

Thank you, sir. Keep up the excellent work. Loving what you've done w/ this so far and someone is maintaining a implementation of socket-io!

Regards,

Michael

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @miguelgrinberg@mbwmbw1337@cfal

      Issue actions

        Support custom serialization methods for message queue packets · Issue #755 · miguelgrinberg/python-socketio