Skip to content

fix kafka decoding error blocking consumption#390

Merged
mirceaulinic merged 1 commit intonapalm-automation:developfrom
matthieugouel:mgouel/kafka-utf-8-decode-error
Feb 9, 2026
Merged

fix kafka decoding error blocking consumption#390
mirceaulinic merged 1 commit intonapalm-automation:developfrom
matthieugouel:mgouel/kafka-utf-8-decode-error

Conversation

@matthieugouel
Copy link
Copy Markdown
Contributor

For some reason we received a non utf-8 message from Kafka topic, and the application is getting stuck with this error:

Process Process-27:
Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3/dist-packages/napalm_logs/listener_proc.py", line 102, in start
    log_message, log_source = self.listener.receive()
                              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/napalm_logs/listener/kafka.py", line 83, in receive
    log_source = log_source.decode()
                 ^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa2 in position 1: invalid start byte

This fix try catch on the initial key decoding and return and follow the same error handling pattern as the message json decoding below.

For some reason we received a non utf-8 message from Kafka topic, and the application is getting stuck with this error:

```
Process Process-27:
Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3/dist-packages/napalm_logs/listener_proc.py", line 102, in start
    log_message, log_source = self.listener.receive()
                              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/napalm_logs/listener/kafka.py", line 83, in receive
    log_source = log_source.decode()
                 ^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa2 in position 1: invalid start byte
```

This fix try catch on the initial key decoding and return and follow the same error handling pattern as the message json decoding below.
Copy link
Copy Markdown
Member

@mirceaulinic mirceaulinic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Odd. The tests aren't passing, but many things are broken here, this repo needs some love and I'll make some time soon for a refresh & release. Thanks for your contribution @matthieugouel!

@mirceaulinic mirceaulinic merged commit d2f3b4f into napalm-automation:develop Feb 9, 2026
0 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants