Skip to content

Fix uvicorn logging when in Vertex AI #75

@alvarobartt

Description

@alvarobartt

Description

At the moment, when using the latest wheel via the Hugging Face DCL in Google us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-2.transformers.4-41.ubuntu2204.py311, the logs are not dumped within Google Cloud Logging, and that's due to the default uvicorn logging configuration.

Note

EDIT: Those are indeed dumped, just that some logs take a while to appear within Google Cloud. But the logging messages on a per-request basis are missing, meaning that only failures are notified, so we could still try to add some logging messages for each incoming request.

Additionally, besides the default uvicorn logs on start up, GET/POST requests, etc. we should also add more logging messages within the inference interface to see what's going on when the inference is running to better debug the errors if any.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions