Language negotiation support

[mnot]

Describe the issue

The beta W3C site has links to Japanese and Chinese versions of the site in a banner on every page, but ignores language preferences sent by the browser in the Accept-Language request header field.

To reproduce

1. Set your browser to prefer japanese and/or chinese over other languages.
2. Navigate to the beta site.

Expected behavior

The site appears in the user's preferred langauge.

Screenshots

n/a

Additional context

My browser is sending Accept-Encoding: ja,zh;q=0.8,en-US;q=0.5,en;q=0.3.

I'd argue that the banner not be shown at all, once negotiation is working properly.

[frivoal]

I agree with the general request here.

However:

I'd argue that the banner not be shown at all, once negotiation is working properly.

I disagree on this one. Content negotiation is not always reliable because not always set up properly, people use shared computers… Having a manual way to switch is important. That said, it may not need to be as preeminent as if content negotiation was indeed working.

[fantasai]

It's also not designed in a way that scales very well. It would be good to:

• Use language negotiation by default
• Allow for overrides
• Fall back to English if the desired language is not available for that particular page
• Make it possible to scale to more languages

[proimage]

There's also no indication for people who don't read Japanese or Chinese that those are part of a language switcher. Right now it could easily be mistaken for accidental code barf. 😉 It could stand to have something like a 🌍 (globe) icon to give a bit more context.

[gosko]

The plan for localized sites and translated content is documented in the Internationalization approach on the redesign project blog.

A note on content negotiation

Please note this approach matches the current usage more accurately and offers a localized country site for users. It does, however, mean we cannot easily use content negotiation to serve pages since we have multiple sites rather than one site whose content is entirely translated in all available languages.

Given only part of the site will be translated, rather than the entire site, having separate localized sites seems a more sensible option.

It is also better for SEO to have unique URLs for each language.

However, we wanted to note we are aware W3C is very keen on content negotiation. We do not think this would work well in practice considering the structure of the W3C website in the different languages.

Personally, I'd like to be able to dismiss the links to localized sites to reclaim that vertical space (maybe rolled up into a language icon in the main nav bar when dismissed)

We also discussed possibly highlighting the availability of translated content in the user's preferred language(s) using javascript. I think we are very unlikely to support classical language negotiation on the redesigned site.

@mnot do you know of many popular sites that make good use of use language negotiation today? My experience has been that it's rarely used elsewhere (for example not used by sites like Google, Apple, Amazon, nor the Canadian Govt which publishes everything in English and French). But maybe it is used elsewhere and I have been missing it due to my English-centric accept headers. This blog post seems to confirm current practice is all over the place.

[mnot]

My understanding is that there's hand-me-down false information circulating about conneg not working from the mid-00's that is still followed as practice; sites don't do it because they don't see other sites doing it, and so it's a sort of cobbled together 'best practice' that really isn't best.

[gosko]

Another practical issue for us is that the CDN we're using does not support conneg.

If there are sites that have managed to get language negotiation to work seamlessly, I'd love to see how they work.

We have tried a number of approaches for different areas of the W3C site over the past few decades and none of them seem obviously better to me than the current plan for the redesigned site.

[r12a]

If there are sites that have managed to get language negotiation to work seamlessly, I'd love to see how they work.
We have tried a number of approaches for different areas of the W3C site over the past few decades and none of them seem obviously better to me than the current plan for the redesigned site.

Fwiw, the W3C i18n WG has been successfully using 'sticky' content negotiation (backed up explicit links on each page) for (at a guess) about 15 years. Try visiting one of our articles, such as the following, which describes how we do it. https://www.w3.org/International/questions/qa-site-conneg

That page only has one translation (Swedish) but other pages have up to 14.

Here's our summary:

Use server-based, language-related content negotiation to point the user to the page that matches their browser preferences, but also add links to each page so that the user can change languages easily if they prefer.

Consider how to indicate to the user where the in-page language links are, and if the page is available in a long list of languages, consider whether or not to use something like a select control (and if so, how to make it obvious what its function is).

If the user switches to a different language, offer them the opportunity to remember that choice and serve up subsequent pages in that language, overriding their browser settings.

There are more articles about content negotiation at https://www.w3.org/International/articlelist#navigating

And yes, the i18n WG did recommend early on that the beta site use content negotiation. My guess is that the CDN is the main culprit here. (Note that github.io doesn't support content negotiation either.)

[gosko]

Yes, the i18n site is one of the approaches I had in mind. Another is the WAI site which has lots of translated content as well, for example Introduction to Web Accessibility. I noticed they were planning to use conneg but in the end they went with something similar to what Studio24 has proposed for the new W3C site – maybe their options were limited due to serving their content from GitHub Pages. We are constrained by what's possible with the CMS as well.