-
Notifications
You must be signed in to change notification settings - Fork 25
Description
Description
At the moment, when using the latest wheel via the Hugging Face DCL in Google us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-2.transformers.4-41.ubuntu2204.py311, the logs are not dumped within Google Cloud Logging, and that's due to the default uvicorn logging configuration.
Note
EDIT: Those are indeed dumped, just that some logs take a while to appear within Google Cloud. But the logging messages on a per-request basis are missing, meaning that only failures are notified, so we could still try to add some logging messages for each incoming request.
Additionally, besides the default uvicorn logs on start up, GET/POST requests, etc. we should also add more logging messages within the inference interface to see what's going on when the inference is running to better debug the errors if any.