Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cryptic Load Reader error with binary.Read "invalid type" is likely due to a single error buffer type in the code #3430

Open
ShadowJonathan opened this issue Nov 23, 2023 · 2 comments · May be fixed by #3431
Open

Cryptic Load Reader error with binary.Read "invalid type" is likely due to a single error buffer type in the code #3430

ShadowJonathan opened this issue Nov 23, 2023 · 2 comments · May be fixed by #3431

Comments

@ShadowJonathan
Copy link

This has taken me ages to track down, but you probably have a binary.Read bug right here:

cadvisor/utils/cpuload/netlink/netlink.go

Line 210 in 2129e1c

var errno int32

I'm getting spammed (and see issues on getting spammed with) error binary.Read: invalid type int32, i've been looking in the entire codebase for this single type, since i know that it has caused this problem, and this is the only instance in the netlink path.

For reference, i'm tracking down why enabling --enable_load_reader=true and then deploying cadvisor as a daemonset creates these log messages.

I've seen #3242 also refer to this, in their case it was simply disabling the load reader that worked for them, but I'd like to have the metrics that're associated with it.
@ShadowJonathan ShadowJonathan linked a pull request Nov 24, 2023 that will close this issue
Open
@ShadowJonathan
Copy link
Author

The reason this codepath got hit in the first place is #3137

@namelivia
Copy link

namelivia commented May 10, 2024 •
edited
Loading

I am having the same issue, I am setting --enable_load_reader=true across different hosts, in most of them running Ubuntu I have no issues and I've been able to get the CPU load. But in two Debian hosts I get error binary.Read: invalid type int32 no data.
I've checked the structure of /sys/fs/cgroup in both types of hosts and coincidentally the Ubuntu ones have the cgroup v1 directory structure, but the Debian ones that are failing have the cgroup v2 structure.

The piece of code you linked corresponds to Netlink error handling, so I suspect this is hiding another error, maybe if the original error is fixed this entire piece of code can be avoided.

By the way, my intuition tells me this TODO is related, it is signed by @rjnagal

@namelivia namelivia mentioned this issue May 10, 2024
Open
@chengjoey chengjoey mentioned this issue Aug 9, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix errno reference ShadowJonathan/cadvisor
2 participants
@namelivia @ShadowJonathan