journald feature: log as many as mainlog #9597
Conversation
It's our new default logger on Linux.
icinga-probot
bot
added
area/log
…per endpoint using a flag whether already done. That's reset on successful connection. In a randomly picked customer mainlog there are 1241932 lines starting with "[". 70222 of them say "Reconnecting to endpoint".
|
At best read the individual commit messages. |
|
With some agents gone, the log is still flooded with messages like these: |
|
Once or recurring? |
|
With every attempt. |
…r endpoint+error by remembering the last prominently logged one. That's forgotten on successful connection. In a randomly picked customer mainlog there are 1241932 lines starting with "[". 68357 of them say "Cannot connect to host".
… only once per checkable using a flag whether already done. That's reset on valid perfdata. In a randomly picked customer mainlog there are 1241932 lines starting with "[". 56817 of them say "Ignoring invalid perfdata for checkable".
This reverts commit 3c55635.
by moving some messages to notice.
How does this PR fix this? It maybe makes the log message less prominent, but if what's given in that issue is valid perfdata, it should not generate a log message saying the opposite in the first place. |
|
In general, I wouldn't pursue changing the default logging in this PR. I fear this will just result in addressing the worst offenders that show in a somewhat standard configuration and once we release, there will be complaints about all the cases we missed, resulting in overall frustration for users and developers. Instead, I think we should start by cleaning up and improving log messages (that involved more than just changing the severity) step by step until we are confident that the log file is great and is suitable for also logging into syslog/journald. This simply involves more work than will realistically be done in a single PR and is a process over some time that also means opening issues whenever you notice unhelpful log messages and addressing these. I'm aware that this probably implies that we won't switch the default logging for 2.14, but the longer I think about, the more utopian this sounds. That release should feature lots of improved logging nonetheless, this will allow finding further places for improvement from logs we receive that were written by newer version. |
|
Then I'll outsource the log less garbage part to another PR once you consider it OK as-is or request any changes. (Hello GHA!) This PR is for the utopia. |
It's our new default logger on Linux.
fixes #8857
refs #9282
TODO