Skip to content

Double encoding of egress events #406

@Glutexo

Description

@Glutexo

The events produced on the platform.inventory.host-egress topic are currently double-encoded to JSON.

  1. First encoding happens in the app.queue.egress.build_host_event function, where Marshmallow Schema is used to validate and encode the event dictionary. This produces a JSON string.
  2. Second encoding happens in the app.queue.egress.KafkaEventProducer.write_event method by plain json.dumps.

The result is a JSON string (not a dictionary) containing properly-escaped JSON dictionary literal. "{\"type\":\"created\"}" instead of {"type":"created"}. That means that if a client decodes the message, it doesn’t get the dictionary, but a string, which must be decoded again.

Possible solutions:

  • Don’t use Marshmallow to validate messages that our code produce. These messages should be tested and thus trusted.
  • Pass the Marshmallow object instead of an encoded string. Make the producer wrapper expect such objects and dump them directly to the queue.

Passing down the Marshmallow object is an easy and quite ok solution.

I am disappoint that there are no tests for the message format. A bug like this would be probably caught by a test.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions