Skip to content

Reduce iotcored logging while offline #959

@aws-kevinrickard

Description

@aws-kevinrickard

iotcored can fill the storage with syslog entries very quickly, it filled my 120 gb disk overnight (I had just deleted syslogs and expanded the volume cause it filled yesterday as well)

example logs showing it repeats at the nanosecond level:

2025-10-07T00:00:04.797015+00:00 ip-172-31-8-43 iotcored[16178]: #033[0;34mD[ggl-sdk] socket_epoll.c:69: Calling epoll callback on thread 16178.#033[0m
2025-10-07T00:00:04.797019+00:00 ip-172-31-8-43 iotcored[16178]: #033[1;31mE[ggl-socket-server] socket_server.c:40: Failed to accept on socket 3: 24.#033[0m
2025-10-07T00:00:04.797060+00:00 ip-172-31-8-43 iotcored[16178]: #033[0;34mD[ggl-sdk] socket_epoll.c:69: Calling epoll callback on thread 16178.#033[0m
2025-10-07T00:00:04.797065+00:00 ip-172-31-8-43 iotcored[16178]: #033[1;31mE[ggl-socket-server] socket_server.c:40: Failed to accept on socket 3: 24.#033[0m
2025-10-07T00:00:04.797069+00:00 ip-172-31-8-43 iotcored[16178]: #033[0;34mD[ggl-sdk] socket_epoll.c:69: Calling epoll callback on thread 16178.#033[0m
2025-10-07T00:00:04.797073+00:00 ip-172-31-8-43 iotcored[16178]: #033[1;31mE[ggl-socket-server] socket_server.c:40: Failed to accept on socket 3: 24.#033[0m

workaround might be diabling journal forwarding to syslog, currently trying this out:

sudo mkdir -p /etc/systemd/journald.conf.d
sudo tee /etc/systemd/journald.conf.d/99-limit.conf <<'EOF'
SystemMaxUse=500M
MaxFileSec=7day
ForwardToSyslog=no
EOF
sudo systemctl restart systemd-journald

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions