Serializing and deserializing across changes to the timezone database

[annevk]

In whatwg/html#6284 @anba brought up this concern:

I just wonder if calendar and time zones should get re-validated when deserialised, in case the supported calendars or time zones changed in some way. For example when serialising to disk and later deserialising from disk with a newer browser version, it's possible that some calendars or time zones are no longer supported or are canonicalised differently (only relevant for calendar identifiers).

I was initially leaning towards then maybe not supporting timezone serialization, but @hsivonen brought up some good points:

I think we should not prohibit serializing values that have attached time zones, because it would be bad to give Web developers an incentive to prefer UTC offset over cities in values that get serialized. UTC offset does not accomplish the same as a city even within one year in places that have daylight saving time, and more generally, user-perceived time zones over time are attached to political rules in a particular place and not to a particular offset.

I think we should explicitly cover the possibility of time zone canonicalization changing over time (Kiev to Kyiv, Calcutta to Kolkata, etc.) and the possibility of a time zone going away. In the case of canonicalization change, we can assume there to be data that enables re-canonicalization under the new rules.

I'm not sure what we should spec about time zones going away, but it seem unlikely enough that we shouldn't prohibit serialization due to that possibility.

In the scenario where we cannot recover we can probably reuse the existing "DataCloneError" DOMException.

Two questions come to mind:

1. What does TC39 think about this?
2. Who will specify the infrastructure to validate a serialized timezone and "correct" it when deserialization is requested?

cc @Ms2ger @ptomato @nicolo-ribaudo

[nicolo-ribaudo]

About @hsivonen's point, Temporal preserves the user-provided timezone identifier rather than canonicalizing it (you can verify it with Temporal.ZonedDateTime.from("2026-01-01T13:20[Europe/Kiev]").timeZoneId and Temporal.ZonedDateTime.from("2026-01-01T13:20[Europe/Kyiv]").timeZoneId).

There is a separate proposal that allows developers to convert timezone identifiers to their canonicalized version (https://github.com/tc39/proposal-canonical-tz), but the default behavior is to respect the data that the developer provided.

I think it's reasonable to throw a DataCloneError when trying to deserialize a timezone that does not exist anymore, but serialization should not take care of re-canonicalizing.

2. Who will specify the infrastructure to validate a serialized timezone and "correct" it when deserialization is requested?

Temporal provides infrastructure to canonicalize calendar IDs: https://tc39.es/proposal-temporal/#sec-temporal-canonicalizecalendar

[annevk]

But they are still matched against some kind of database, right? For instance, Europe/TEST throws.

So an entry could disappear but have an equivalent entry that could be used instead. What algorithm would we use in the deserialization steps to find it? (I don't think this can be canonicalization, since it needs to handle historical identifiers.)

And if that algorithm does not succeed, then we got a real failure.

[justingrant]

TL;DR - I agree with @hsivonen that we should serialize & deserialize Temporal.ZonedDateTime objects via structuredClone. Failures will be vanishingly rare... I'd expect many other errors (like out-of-memory errors) to be orders of magnitude more common. More details are below to explain why I have this opinion, and to clarify the problem space a bit.

Apologies for long response below!

About @hsivonen's point, Temporal preserves the user-provided timezone identifier rather than canonicalizing it (you can verify it with Temporal.ZonedDateTime.from("2026-01-01T13:20[Europe/Kiev]").timeZoneId and Temporal.ZonedDateTime.from("2026-01-01T13:20[Europe/Kyiv]").timeZoneId).

Yep, this is correct.

There is a separate proposal that allows developers to convert timezone identifiers to their canonicalized version (https://github.com/tc39/proposal-canonical-tz), but the default behavior is to respect the data that the developer provided.

Sorry, this is backwards. Previous behavior in JS (in Intl.DateTimeFormat which was the only pre-Temporal part of JS that user-specified time zones were accepted) was that all time zone input IDs would be canonicalized. This behavior meant that whenever a rename happened (rare, but an ID is renamed every few years on average, most recently Europe/Kiev => Europe/Kyiv in 2022) then the behavior of existing code would change. This would break things like automated tests that expected a particular time zone to be returned. And it would generate annoyed complaints from developers because renames are sometimes politically sensitive (i.e. Kiev/Kyiv) so respecting the developer's original choice is a safe bet.

The new Temporal.ZonedDateTime's behavior, on the other hand (after https://github.com/tc39/proposal-canonical-tz was merged into Temporal in 2023) will retain any time zone ID provided by the user instead of changing it. As noted above, if you provide Europe/Kiev to Temporal.ZonedDateTime.from then you'll get Europe/Kiev back. Note that Temporal.ZDT will make one change, however: it will normalize the letter case, e.g. from "EUroPE/KIeV" => "Europe/Kiev". It does this so that implementations can store time zone IDs as 10-bit indexes into the list of TZDB IDs instead of storing the whole ID string.

Oh, also https://github.com/tc39/proposal-canonical-tz has been archived since 2023 when it reached Stage 3 and was absorbed into Temporal. So https://github.com/tc39/proposal-canonical-tz is useful for historical context but Temporal is the right place to look for current behavior.

So an entry could disappear but have an equivalent entry that could be used instead. What algorithm would we use in the deserialization steps to find it? (I don't think this can be canonicalization, since it needs to handle historical identifiers.)

It depends what you mean by "disappear". If you mean renames in TZDB (like Europe/Kiev => Europe/Kyiv) or other changes to which group of aliases is considered canonical that's not a disappearance. Europe/Kiev continues to be valid and will be valid forever. Per above, it should be deserialized as Europe/Kiev.

If you mean cases like (copying from @anba's whatwg/html#6284 (comment)) US/Pacific-New, Canada/East-Saskatchewan, SystemV/AST4ADT, those legacy IDs were removed many years ago:

• Canada/East-Saskatchewan in 2017, because it violated the modern TZDB's string length limit (and was a legacy ID anyways that had been superseded by America/Regina)
• US/Pacific-New and SystemV/* also in 2020

It's not accidental that AFAIK there have been no more removals of legacy IDs since 2020; there was so much negative feedback about the 2020 removals that TZDB now no longer removes IDs, even legacy ones that have been superseded years (if not decades) ago. This is OK because the entire TZDB is only ~600 IDs, with only a handful added each year. Keeping old IDs around forever is practical.

All that said, what can happen is for JS implementations to, in violation of the ECMAScript spec, decide to support legacy IDs in two categories:

• IDs removed from TZDB long ago, as discussed above, to avoid breaking anyone
• IDs that were never supported by TZDB, but apparently were supported by ICU at one point. "PST" is one example.

The key word here is "legacy". These are very obscure cases. The main way that we (meaning people building the web platform) usually learn about problems with IDs like these are:

• Exhaustive test suites that break if a long-deprecated ID is removed
• Complaints from users with misconfigured Linux machines, because Linux makes it easy to enter your preferred time zone ID in a CLI, unlike 99.9%+ of users who choose their time zone from a UI time zone picker that only shows modern, canonical IDs. (APIs that power time zone pickers, like Intl.supportedValuesOf('timeZone') only list entries corresponding to modern, canonical TZDB zones, because otherwise you'd end up with a confusing list for end users.)

So I'm not saying that encountering a used-to-work-but-not-anymore ID will never happen across billions of devices, but these failures will be exceedingly rare for end users. Not least because not only will users need to use a legacy ID but then JS engines must choose to stop supporting those legacy IDs. The Venn diagram overlap of those two things is extremely small.

And if that algorithm does not succeed, then we got a real failure.

If someone serializes an obscure, long-deprecated legacy ID on their (almost certainly Linux) machine into IndexedDB, and then updates Chrome or Firefox to a later version that finally drops support for those deprecated IDs, then IMO it's appropriate to throw an exception upon deserialization because the current JS engine has no idea how to turn that persisted value into a valid Temporal.ZonedDateTime. Ideally, the DataCloneError returned would provide the original serialized string representation so that programs can explain to users what happened.

Alternatively, you could throw on serialization when the user chooses an ID that's not in TZDB. This would require browsers to expose a way to differentiate between "this ID is spec-compliant and is in the IANA Time Zone Database" vs. "this ID is a weird not-spec-compliant ID that the browser added on their own for compatibility with legacy ICU and Intl.DateTimeFormat". This would still not be foolproof, because a user could always downgrade a browser to an older version that contained an older TZDB version which might not recognize all the IDs in the later version of the browser.

Note that this problem of unknown IDs is not unique to HTML deserialization. JS engines already face the same problem in a much more common case: when computer A with a later version of TZDB sends a serialized RFC 9557 string (the standard behind Temporal.ZonedDateTime serialization) to computer B that has an older version that's never heard of the newly-added ID that A sent it. If TZDB on A has a new ID that B's TZDB has never heard of, then Temporal.ZonedDateTime throws an exception, because there's no way for B to know that this is a valid ID and no way for it to build a valid ZDT instance with an unknown time zone ID. This is an inherent problem with serialization using a changing database; IMO the best solution is to throw and let the developer figure out how to handle the problem.

Feel free to follow up with questions or concerns... this is a complex area with, in some cases, no easy answers.

This is already a long comment, so I'll talk about calendars in my next comment.

[justingrant]

Calendars are is a simpler case, because (unlike with time zones where a 3rd party owns the list of IDs) JS engines control the list of calendars and the behavior. There's even an entire TC39 proposal dedicated to standardizing things that vary across calendars: https://github.com/tc39/proposal-intl-era-monthcode

@sffc (champion of that proposal) - How did you decide to handle use of each of the deprecated calendars? Canonicalize or leave as-is?

For structured clone, I'd recommend that we follow what Temporal.X.from methods behave in the case where a user inputs a deprecated calendar ID, unless there's some overriding reason not to.

[ptomato]

What Justin said! I was preparing to write a comment but have little to add now 😄

Note that ECMA-402 already recommends keeping old IANA IDs for at least 2 years, and after that demoting them to aliases rather than dropping them: https://tc39.es/ecma402/#sec-use-of-iana-time-zone-database

So I don't think this is a problem.

Note also https://tc39.es/proposal-temporal/docs/timezone.html#ambiguity-caused-by-permanent-changes-to-a-time-zone-definition. The structuredClone PR effectively implements the behaviour of offset: 'use'.

[sffc]

A key design decision of Temporal is to retain the time zone identifier and calendar identifier, so it shouldn't be thrown away when using structuredClone.

Neither CLDR nor IANA ever removes IDs. They can mark them as legacy, and Temporal already handles them.

@sffc (champion of that proposal) - How did you decide to handle use of each of the deprecated calendars? Canonicalize or leave as-is?

Unsupported calendars are unsupported and throw errors (like islamic-rgsa). Aliases (like islamicc) are canonicalized, according to https://tc39.es/proposal-temporal/#sec-temporal-canonicalizecalendar