[FEATURE] logging support

[Faylixe]

Is your feature request related to a problem

When it comes to logging, there is no documentation except that we should use echo related function to log message. But sometime you would like to stick to traditional logging model (when using third party library for instance) and play with a verbose flag to control the logging level eventually. This is especially true when you are transitioning from an older model based on argparseto Typer and you do not want to break compatibility too much.

This is typically our case, as we are currently switching to Typer for our source separation library Spleeter. We would like to stick to logging module, as we also control the logging of TensorFlow. In this case this is even more true as the library could be used directly as Python library AND as CLI command.

The solution you would like

I came up with this very naïve solution which seems to work:

import logging

from typer import echo

class TyperLoggerHandler(logging.Handler):
    """ A custom logger handler that use Typer echo. """

    def emit(self, record: logging.LogRecord) -> None:
        echo(self.format(record))

formatter = logging.Formatter('%(levelname)s:%(name)s:%(message)s')
handler = TyperLoggerHandler()
handler.setFormatter(formatter)
logger: logging.Logger = logging.getLogger('spleeter')
logger.addHandler(handler)
logger.setLevel(logging.INFO)

# ... log now as usual

It would be great to maybe integrate such behaviour directly to Typer, with maybe a more complex handler implementation with message formatting using styles.

If this sounds appropriate I would gladly contribute and submit a pull request

[angarg]

Would a callback on your main app help here?

app = typer.Typer()

@app.callback()
def main(verbose: bool = typer.Option(False, "--verbose", "-v")):
    lvl = logging.INFO
    fmt = "%(message)s"
    if verbose:
        lvl = logging.DEBUG
    logging.basicConfig(level=lvl, format=fmt)

@app.command()
def dosomething():
    logging.debug("debug")
    logging.info("info")

This might let you use the logging module everywhere in your commands.

[codethief]

Totally unrelated question but:

as we also control the logging of TensorFlow

How exactly do you do that? I've been struggling with this for quite some time and haven't found a solution yet – Tensorflow seems to completely ignore my logging setup.

[Faylixe]

Totally unrelated question but:

as we also control the logging of TensorFlow

How exactly do you do that? I've been struggling with this for quite some time and haven't found a solution yet – Tensorflow seems to completely ignore my logging setup.

Oh yeah well there is no fixed solution, it totally depends on Tensorflow version as well as they change this feature all the time, my better advice would be to search SO and GH for related issues and try everything mentioned until it works.

You can check how we are doing it with Spleeter here. The key to disable / configure logging for TF is mostly to do it before starting importing any TF modules by doing the following:

• Disable CPP level warnings by setting TF_CPP_MIN_LOG_LEVEL environment variable.
• Call tensorflow.compat.v1.logging.set_verbosity (you should import tensorflow.compat.v1.logging before before any other TF related import in your runtime flow).
• Finally retrieving TF logger and remove every handlers and replace with our.

Hope it will help :)

[Faylixe]

Would a callback on your main app help here?

app = typer.Typer()

@app.callback()
def main(verbose: bool = typer.Option(False, "--verbose", "-v")):
    lvl = logging.INFO
    fmt = "%(message)s"
    if verbose:
        lvl = logging.DEBUG
    logging.basicConfig(level=lvl, format=fmt)

@app.command()
def dosomething():
    logging.debug("debug")
    logging.info("info")

This might let you use the logging module everywhere in your commands.

Unless it has changed but if you are not using echo function any output to STDOUT using either print function and logging does not work. Plus you may want to configure third party logger which won't use the basic configuration.

[ssbarnea]

Controlling verbosity of an application is not as easy as having a boolean flag. In fact it involves two additive (can be specified multiple times) options, -v to increase default verbosity and -q to reduce it.

I do have one very small cli tool that does this using click, but that app does not use Typer. I am not quite pleased how I achieved it but it seems to work fine for the moment.

It would quite interesting to see an example of such an implementation using typer.

My dilemma is around how I can initialize the logging as soon as possible because main() callback runs far too late for logging, when I reach it a lot of code that produces logging was already executed (i have dynamic commands).

[ricardogsilva]

@Faylixe code seems to work great. I enhanced it with some colors:

import logging

import typer

app = typer.Typer()
logger = logging.getLogger(__name__)


class TyperLoggerHandler(logging.Handler):

    def emit(self, record: logging.LogRecord) -> None:
        fg = None
        bg = None
        if record.levelno == logging.DEBUG:
            fg = typer.colors.BLACK
        elif record.levelno == logging.INFO:
            fg = typer.colors.BRIGHT_BLUE
        elif record.levelno == logging.WARNING:
            fg = typer.colors.BRIGHT_MAGENTA
        elif record.levelno == logging.CRITICAL:
            fg = typer.colors.BRIGHT_RED
        elif record.levelno == logging.ERROR:
            fg = typer.colors.BRIGHT_WHITE
            bg = typer.colors.RED
        typer.secho(self.format(record), bg=bg, fg=fg)


@app.command()
def main():
    logger.debug("Hi, World!")
    logger.error("Yes, hi indeed!")


if __name__ == "__main__":
    typer_handler = TyperLoggerHandler()
    logging.basicConfig(level=logging.DEBUG, handlers=(typer_handler,))
    app()

It would be nice to have this handler become part of typer. Perhaps also have a typer.logging_basic_config() function that does the rest of the boilerplate?

[robinbowes]

I just found Typer, and am going to use it with my next CLI tool instead of Click. I too would really like to see better integration with logger, also for 3rd-party libs.

[tiangolo]

Thanks for all the discussion everyone!

Going back to the original question, do you see any benefit in using Typer's (Click's) echo() function instead of the default logging functionality?

I would think it would be better to just use the standard logging module without dealing with Typer/Click for that. I don't think Click's echo() additional logic would be useful when dealing with normal logging.

Now, I wonder, do you feel like the docs "discourage" using something else other than echo()? Do you feel it looks like the docs discourage using standard logging? If you do, then that's a problem I have to fix in the docs.

On the other hand, if you wanted fancier logs, I would instead recommend Rich: https://rich.readthedocs.io/en/stable/logging.html

I want to document how to "integrate" Rich with Typer (just how to use it).

[ssbarnea]

@tiangolo I think you should be aware of a problem related to rich-logging, the fact that it does not do soft-line wrapping, basically you cannot resize terminal (or web browser) and expect for the log lines to (re)wrap well.

That is why, almost two years ago, I had to write the enrich wrapper to add soft-wrapping for rich logger. That library is currently used by two major Ansible tools (ansible-lint and molecule), but I would very happy not to have to use the wrapper.

If you are curious about the subject, read pycontribs/enrich#1 and maybe you are more success than on persuading rich's author to add that feature to it as my original PR was refused.

[pmyjavec]

Is there a way to disable changing output configuration altogether and just relying on standard libraries?

We have an application that uses the standard logging library, now we see three log lines for every one. I'm assuming it's related to this library as we didn't see it before deployment. I've not had the time to dive deeper yet.

Any ideas on how we might be able to simply rely on existing logging and not need to use typer.echo()

Thanks.