Provide users choice of timezone to FileTaskHandler, for log timestamp formatting #44376
Description
Description
As a user, i want the logs presented in the Airflow UI to match the choice of timezone, set in the top right corner of the UI. So that timestamps from the logs are consistent with other times presented in the UI, such as task start and end time.
Normally, with logs stored in files, the timestamps are stuck in the file, as it was materialized during write. [*]
With modern remote logging solutions, that is no longer the case.
In our case, we have overridden the task logger to save all logs in a remote logging solution; Loki.
Loki internally stores all its logs in UTC time format, and leaves choice of time zone to the presentation layer.
In the airflow webserver, we set logging_config_class to a class inheriting (FileTaskHandler, ExternalLoggingMixin) to fetch the logs from Loki, format them and present to the UI.
The problem is that the users choice of timezone is not exposed to the FileTaskHandler.read function, so we don't know which timezone to select.
What i'm asking for is either:
- A: Forward the users choice of time zone to
FileTaskHandler.read, to allow that function to format the log in the desired time zone. - B: Rehaul the interface of sending logs between the backend and the frontend UI, so the backend sends logs as
(utc-time, entry)-tuples instead of raw text, then leave the formatting of time to the UI layer. This second option has more benefits in that the user can choose to hide the timestamps entirely, if desired.
[*] (Even with timestamps written to files, you can solve this, if the logfile has a structured format and something converts the timestamps during read of that file)
Use case/motivation
No response
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct