json format before send to sink

[RedheatWei]

loguru config

if logger_config_data.get("kafka") and logger_config_data["kafka"]["enable"]:
    logger.debug("kafka handler enabled")
    kafka_handler = KafkaLoggingHandler(kafka_brokers=logger_config_data["kafka"]["broker"],
                            topic=logger_config_data["kafka"]["topic"])
    logger.add(
        kafka_handler.commit,
        format=Logger.kafka_format(app_name=APP_NAME, env=config_info.get_env()),
        level=logger_config_data["kafka"]["level"].upper(),
        backtrace=True,
        diagnose=True,
        serialize=True
    )

format

    @staticmethod
    def get_format_dict(app_name, env):
        return {
            "time": "{time}",
            "component": app_name,
            "env": env,
            "level": "{level.name}",
            "location": "{name}:{line}",
            "msg": "{message}",
        }

    @staticmethod
    def console_format(app_name, env):
        return " ".join(Logger.get_format_dict(app_name, env).values())

    @staticmethod
    def kafka_format(app_name, env):
        return json.dumps(Logger.get_format_dict(app_name, env))

so kafka_format return this
{"time": "{time}", "component": "xxx", "env": "local", "level": "{level.name}", "location": "{name}:{line}", "msg": "{message}"}
error

--- Logging error in Loguru Handler #2 ---
  File "xxxxxxxx/.venv/lib/python3.13/site-packages/loguru/_handler.py", line 180, in emit
    formatted = precomputed_format.format_map(formatter_record)
KeyError: '"time"'
--- End of logging error ---

kafka_handler just a normal object and not a logging handler object. so why for this ?
thanks

[Delgan]

The problem comes from the { and } braces that surround your JSON format. These confuse Loguru, which interprets them as fields to be formatted. Remember that internally, Loguru will call something roughly equivalent to log_format.format(**record_fields).

As a workaround, you could escape the outer braces, but I think it's just easier to just build the message directly in a custom formatter:

def loguru_formatter(record):
    data = {
        "time": str(record["time"]),
        "component": APP_NAME,
        "env": config_info.get_env(),
        "level": record["level"].name,
        "location": f"{record['name']}:{record['line']}",
        "msg": record["message"],
    }
    serialzied = json.dumps(data)
    record["extra"]["json"] = serialzied
    # We must return a format template, not the already formatted message.
    return "{extra[json]}\n"


logger.add(
    kafka_handler.commit,
    format=loguru_formatter,
    level=logger_config_data["kafka"]["level"].upper(),
    backtrace=True,
    diagnose=True,
    serialize=True,
)

Additionally, note that Loguru is compatible with standard library handlers out of the box. You can therefore configure the KafkaLoggingHandler with your desired format and simply add the handler to Loguru.

kafka_handler = KafkaLoggingHandler(kafka_brokers=logger_config_data["kafka"]["broker"],
                        topic=logger_config_data["kafka"]["topic"])
logger.add(kafka_handler, format="{message}")