ICU4X objects that try_new with a provider should store and expose the resolved locale

[hsivonen]

ECMA-402 requires various objects to be able to expose the resolved options. Common across different types is the resolved locale.

For ECMA-402 compat, we should make various ICU4X objects call take_metadata_and_payload instead of take_payload when loading their primary provider-backed payload and store the DataLocale from the metadata. We should then have a convention across ICU4X for retrieving that Locale from the ICU4X object.

The finer points of DataLocale vs. Locale are unclear to me, so I'm not sure if the convention should be fn resolved_locale(&self) -> &DataLocale allowing the application to call .into_locale() or fn resolved_locale(&self) -> Locale.

[sffc]

What is the "primary provider-backed payload"? It's not well defined in all cases.

The whole concept of the "resolved locale" is fraught and it's not up to ICU4X to perpetuate it. #58 has suggestions for how to solve the 402 problem in user land.

[hsivonen]

What is the "primary provider-backed payload"? It's not well defined in all cases.

For the collator, it would be the payload for CollationMetadataV1Marker.

The whole concept of the "resolved locale" is fraught and it's not up to ICU4X to perpetuate it. #58 has suggestions for how to solve the 402 problem in user land.

My understanding of provider internals is insufficient for understanding what exactly #58 proposes for the ECMA-402 resolved locale info, especially in the case where the glue code opted to implement "best fit" by delegating to ICU4X's lookup mechanism.

Should an application do what Boa does, and try to load data payloads ahead of actual ICU4X object instantiation with the assumption that the preflight with match the actual instantiation or should the application load whatever is considered the primary payload lazily after the fact if the code calling ECMA-402 APIs requests the resolved options?

Either way, which key should the glue code query for if the primary payload for a given ICU4X object isn't well defined in all cases?

[sffc]

Either way, which key should the glue code query for if the primary payload for a given ICU4X object isn't well defined in all cases?

I think it would be fine if ICU4X suggested which key to use for the resolved locale in cases required by 402

Should an application do what Boa does, and try to load data payloads ahead of actual ICU4X object instantiation with the assumption that the preflight with match the actual instantiation or should the application load whatever is considered the primary payload lazily after the fact if the code calling ECMA-402 APIs requests the resolved options?

Neither. The data provider should be instrumented to get the resolved locale out of the DataResponseMetadata.

[hsivonen]

Either way, which key should the glue code query for if the primary payload for a given ICU4X object isn't well defined in all cases?

I think it would be fine if ICU4X suggested which key to use for the resolved locale in cases required by 402

Does it make sense to merely suggest it as opposed to providing a concrete crate for it with documentation that the crate is only provided for 402 compat? Or providing a Cargo option to enable such code in each component directly? I wouldn't mind if the option was named to discourage use along the lines of enable_conceptually_questionable_resolved_locale_for_ecma_402_compat_only.

Should an application do what Boa does, and try to load data payloads ahead of actual ICU4X object instantiation with the assumption that the preflight with match the actual instantiation or should the application load whatever is considered the primary payload lazily after the fact if the code calling ECMA-402 APIs requests the resolved options?

Neither. The data provider should be instrumented to get the resolved locale out of the DataResponseMetadata.

The example seems to preclude the use of the baked-mode constructors that don't take a provider argument, which is unfortunate. I'm not sure, but my initial reaction is that I'd rather do a duplicative lookup if the JS app looks at the resolved options than defeat the baked code path for object construction.

[robertbastian]

Would it be incorrect to always return the requested locale as the resolved locale?

[hsivonen]

Would it be incorrect to always return the requested locale as the resolved locale?

That question can be understood in at least three senses: 1) what the caller wants to know if they care to actually inspect the resolved locale, 2) what's Web-compatible, 3) what fits within the spec's notion of implementation-defined.

In sense 1, incorrect. (Most notably, if the requested locale has a non-language component and the resolved locale does not retain that component, this shows that the implementation's data does not explicitly alter the main flavor of the language in a way that the component would change. Is this actionable information for the caller? Perhaps not.)

In sense 2, probably not Web-compatible considering that things that deviate from what major browsers do tends not to be Web-compatible but maybe Web-compatible in the sense that the information isn't really that actionable anyway.

In sense 3, maybe not strictly incorrect if you read all implementation-defined behavior as not required to even make sense and the observer not getting an infinite number of observations.

[robertbastian]

Returning the requested locale as the resolved locale gives the same result as preresolving locales at datagen time. If we don't have es-419 data and we fall back to es, I don't think the information whether the data is actually es-419 is actionable.

The bigger problem is falling back to und. Maybe we can have a flag in locid_transform that changes fallback behaviour to return an error when und is reached. This would need to be key-dependent, as we know for collation, for example, fallback to und is fine.

I agree that a solution that is compatible with compiled data would be preferrable.

[hsivonen]

I didn't look at the source, but I think Intl.Segmenter, which is a rather special case in its relationship to locale data, in Chrome fakes its resolved locale roughly by 1) considering languages that CLDR knows about in some sense as supported and 2) retaining the region if the region is CLDR-known to be associated with the language (sv-FI retains the region, but fi-SE does not).

[robertbastian]

Do you know any concrete uses of this information, which would break if we deviate? I don't want to let Chrome dictate how standards should be interpreted.

[hsivonen]

From a very quick look at GitHub search, I see one use case beyond test cases and debug logging:

Determining the host locale by executing Intl.DateTimeFormat().resolvedOptions().locale per StackOverflow and various other teaching materials.

So perhaps just echoing back the requested locale could work and not break the Web.

Of course, this only looks at the case where .locale is appended directly to .resolvedOptions() instead of there being an intermediate variable.

[sffc]

Yeah, echoing back the requested locale probably works. I think the most useful piece of information you can get is which one out of a list of locales you got. For example, if the locales requested were ["ff", "fr", "ar"], it is potentially useful to know that "fr" was chosen out of that list, so that you can for example render other components in that language.

[hsivonen]

Yeah, echoing back the requested locale probably works.

Should ECMA-402 change to require this?

[sffc]

@zbraniecki Thoughts on the above?