How should sites (Google and/or you) treat Accept-Language header?

Question

For a long time I've been not happy with the behavior of Google in the cases below, and after accidentally noting that 80+ other people feel the same way (and 20+ people upvote the idea of Google bug report) I thought it's useful to tap into SO's wisdom on HTTP protocols and the web culture.

This behavior appears every time I go to a different country and access google.com from my laptop or install a new browser. In many countries, including Vietnam, google automatically redirects to a localized version (e.g. google.com.vn), the one I can't actually read so I scan it in a desperate attempt to find Google.com in English link.

The first time it happened I was puzzled and spent some time finding out what could be wrong with my request headers; as I expected, my Accept-Language is always en-US or something like en-US,ru;q=0.5. Further tests with different IPs confirmed that they do IP geolocation, even when you are logged in. Later I learned there is a language query ?hl=..., but still.

So, my question would be: do you think this is reasonable and expected behavior, webculture-wise? Do other sites do it too? If you were to design a multilanguage site what would you do with a person who comes from .vn IP but with en-US language?

I imagine there are different approaches to IP geolocation:

• contra: I think this is impolite: I tell you explicitly what I want, what's good about giving me something else?

• pro: It could be that a person has just installed browser with the default language pack, without any idea about Accept-* headers, and still would prefer to see localized version of a website.

• contra: And yet, non-techie people can easily change accept-language by installing a language pack. In fact, if you go to download Firefox, chances are you'll download the one in the language you want, with the accept-language header that respects your wishes.

Note that I read the HTTP 1.1 specification from 1999 but they seem to omit the question of how much the Accept-Language should be really taken into consideration.

I believe this question belongs to SO as a web-app design question. And I'm still thinking about filing a formal bug report (if I am correct that this behavior is far from web standards).

Answer 1

I agree Google is misbehaving. I suspect this is a result of a structural over-optimization. If you put servers close to users, and your goal is insanely fast results, you're better off serving a single cache of search results in a single language, off of each local server cluster.

That means when your IP comes in, it's not so much IP Geolocation as, which server cluster answered your request first. And that cluster nearest to you in Vietnam is optimized to serve pages in Vietnamese.

I'm sure this is resolvable but I wouldn't be surprised if the design of that optimization makes it difficult for them to fix.

Answer 2

I'f from Latvia, I prefer English. Firefox sends en-US by default, IE sends lv-LV (even though it is English version) because of Windows regional settings. So accept-header is unreliable.

IMHO best way is to do both geolocation and accept-language check,

if region and language matches => auto redirect if there is no match => serve your best guess and give user a choice (maybe top right corner of the screen is the best place for language switch as it is UI standard more or less) + remember that choice in a cookie

Answer 3

I share your frustration. I've been disappointed by a great number of sites. Most do IP-geolocation but obviously have no idea about the browser culture settings. Needless to say, the user location is an unreliable information source about which language and culture the user prefers. Especially in the times of globalization where peoples and cultures are highly mixed.

Note that I read the HTTP 1.1 specification from 1999 but they seem to omit the question of how much the Accept-Language should be really taken into consideration.

This setting is actively supported by modern browsers. There is no good reason not to take it into consideration. If the header is not present in the request, do geolocation, otherwise respect the user culture setting.

Answer 4

I'm guessing that a significant percentage of people in non-English speaking countries use browsers that report Accept-language of English (perhaps because they use pirated English versions, who knows).

Most of them, aside from a vocal minority on a English-language programming web site :), would expect to see web sites in their own language, and the common folk don't care about HTTP specifications. This would make the Accept-language header essentially useless, a perfect example of how the real world != programmer fantasy world.

Answer 5

I think there are really two separate issues at play here. Language and country. They most definitely are not the same thing. The HTTP spec explicitly defines the Accept-Language request-header field as language specifier, not a country specifier.

For example: en-US refers to a specific language (the version of English spoken mostly in the United States), en-GB refers to another language (the version of English spoken mostly in the United Kingdom). This really has nothing to do with where you currently are in the world, it just defines what language is most acceptable to you when viewing a website. And that should be respected because even novice users should have a browser in their language which will by default send the correct request-header field.

But that's not what sites traditionally use it as. Like another poster, I've noticed that sites equate language with country and tailor their site to that country's unique political restrictions/freedoms/the company's offerings there, not necessarily for the language. The example given was a company that doesn't sell a particular product in a particular country but does in others.

In the case of what products to display for different countries, geolocating the IP address would be better than making decisions based off the language but it still wouldn't be perfect. What if I'm in another country temporarily but want to order something and have it delivered back home? What if my IP address doesn't reflect where I physically am, it's not perfect?

Yes, things work correctly for most people most of the time. But why on Earth would you make it hard for people to get the site in the language that they ask for?? It's just bad UI design.

Answer 6

Although i understand and even share your frustration, there are some legit reasons to do that.

Many companies have sites tailored to each specific country. For example, because they don't sell all their products to every country, or because a law in a specific country put some specific obligation on their site (a line of product banned, a legal notice added, or whater, ...).

Therefore, they will serve you the version tailored to the country you're in because that's supposed to be the most useful (it doesn't matter that company X is selling product Y in country Z if i'm not currently in country Z).

Answer 7

I'm betting that there are more people use software that doesn't properly utilize this HTTP header value than there are people who live in a country and don't speak/read the native tongue. That would explain Google's behavior.

Answer 8

Google is smart. I imagine they do what they do because for the majority of cases if you're in Vietnam you most likely want .vn - we're technical people so we know all about accept-language and such, but the vast majority of people don't.

That being said, I think one thing they could to at least be polite to the accept-language header, is to show some kind of notification (kind of like the banners that show in this website) saying something like "Looking for Google in English? Click here" if the header indicated they want english - while that may not work for Google (as they don't do it), I would certainly consider that if I was making a multilingual website.