Skip to content

trapping f format errors so they log to file and don't crash running code #1366

@kiml

Description

@kiml

Thanks for loguru.

I know the "issue" with f-format strings causing problems with some data is noted in the documentation and I've seen several github issues such as #1008 discussing aspects of this.

I want to use f-strings in my logging because it's more elegant and significantly reduces line length and ugly 'redundant' code like data=data, value=value etc, however I naturally don't want my program to crash based on data values like dictionaries causing issues.

So I try to balance the problem by using f strings for most data but adding {{data}} ... with the attendant data=data for the known problem cases. This seems pragmatic to me.

However I have a large code base and it's too hard to exhaustively check every value of every log for possible problems. When the above problem occurs, exceptions are ONLY output to the console, not the log files and my program doesn't crash but one of threads in it does. Trying to redirect stderr/stdout kind of defeats half the point of a good logging system anyway.

I accept that the troublesome logging statements are technically faulty and one could argue it's not loguru's place to address that but it's actually an undesired interaction between two pieces of unrelated code (mine and loguru) where neither code on its own is wrong.

I have three issues:

  • It is not acceptable for logging to silently crash one of my threads whilst the rest of the program merrily carries on in a broken way (I cannot check liveness of all threads all the time, especially as I don't own all threads and logging should never contribute program failure).
  • I cannot easily monitor the console of my app long term - logging issues may be latent in "production" code. I need to be able to look through log-FILES and find the issue.
  • It's nearly impossible to find the faulty code lines.

I made the following exploratory change to the loguru code (Logger._log) (
try:
log_record["message"] = message.format(*args, **kwargs)
except Exception as e:
log_record["message"] = f"*** LOGURU FORMAT ERROR: {message} :: {args} :: {kwargs}"

This has the twofold benefit that
a) Logging issues do not crash application threads.
b) Issues are clearly logged and I can find the errant lines and address

I know the use of the broad Exception is not ideal, it could certainly be narrowed down to the handful of possible exceptions but also being just one line it seemed relatively harmless. Anyhow my point is the principle. I am led to understand that the cost of try-except is also near-enough free in cases where the exception is not triggered, so I infer performance would not be adversely affected.

Certainly this change allowed my program to continue in the face or errors whilst also highlighting those lines that were causing an issue. For me it's a win-win. Would you consider adding this handling into the code base so others can benefit.

regards
Kim

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementImprovement to an already existing feature

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions