Skip to content

Instantly share code, notes, and snippets.

@amake
Last active June 18, 2024 02:22
Show Gist options
  • Save amake/0ac7724681ac1c178c6f95a5b09f03ce to your computer and use it in GitHub Desktop.
Save amake/0ac7724681ac1c178c6f95a5b09f03ce to your computer and use it in GitHub Desktop.
Correct localization on Android 7

Correct localization on Android 7

Prior to Android 7, the system had a single preferred locale, and fallback behavior was quite rudimentary. Starting with Android 7, the user can now specify a priority list of locales, and fallback behavior is improved.

However, in many cases it is still surprisingly difficult to make full use of locale fallback, and there are some hidden gotchas when trying to fully support both Android 7 and earlier versions.

What is fallback?

Assume a user has set the following list of preferred locales:

  1. fr-FR
  2. de-DE
  3. en-US

Assume also that your app provides the following locales:

  • en (default)
  • de
  • es
  • es-MX
  • sr-Latn

When running your app on Android 7, this user should see the display in de because fr-FR (their first choice) is not provided, and de (their second choice) is preferred over en (their third choice).

On Android 6 or earlier, the app would be displayed in en: the user could only specify fr-FR as the preferred locale, and when not available the system would fall back to the app default locale.

What’s the problem?

On Android you can experience an issue I call language resource contamination (or just resource contamination for short). This is where resources for languages you don’t support “leak” into your app, making the system think you support them. When this happens, the fallback mechanism described above fails to work correctly.

I have identified two different kinds of resource contamination:

  • Compile-time contamination
  • Runtime contamination

Compile-time contamination

If you include the AppCompat v7 library (included by default in new projects created by Android Studio 3.1) or any of the Google Play Services libraries in your app, fallback will almost certainly be broken.

The reason is that these libraries contain resources for a huge number of locales; when building your APK these resources will be merged in, and it will appear to the system that your app supports all of these locales.

Under the initial example scenario, the system will believe that the app supports fr-FR when it actually does not supply any fr-FR versions of its “own” strings. Thus the display locale will become fr-FR, but each individual string will fall back to the default variant, and the app will appear to be shown in en.

The solution to this is to filter your app’s resources, allowing only the locales that you actually support. You can do this in build.gradle:

android {
  defaultConfig {
    // No need to specify the default locale here
    resConfigs 'de', 'es', 'es-rMX', 'b+sr+Latn'
  }
}

This will ensure that only resources for locales you actually support are included in the APK, allowing the OS to correctly determine the fallback locale at runtime. As a bonus, your APK will be a little bit smaller.

Prior to version 3.1.0 of the Android Gradle Plugin it was possible to specify resConfigs 'auto', where the appropriate locales would be automatically detected. However this is now deprecated, and you are recommended to manually list your supported locales.

See also:

Runtime contamination

Starting with Android 7, the WebView component is no longer the Android System WebView package, but is actually the Chrome app itself. This has some surprising consequences.

When first instantiating a WebView, the Chrome app will be loaded into your app’s current Activity. This apparently causes the current Activity’s preferred locale list to be overwritten with Chrome’s. Chrome supports a large number of locales, most likely including the user’s top preference. This then creates the same situation as described earlier, where the Activity’s locale may be set to e.g. fr-FR but all strings are displayed in en. This situation persists even after removing the WebView.

This issue was reported as issue 218310 but was closed as “WorkingAsIntended” but without a clear, supported fix offered.

One workaround is to fix up the current Activity’s locale immediately after loading the WebView:

WebView webView = (WebView) rootView.findViewById(R.id.webView);
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
  Resources resources = getResources();
  Configuration config = resources.getConfiguration();
  LocaleList currentLocales = config.getLocales();
  if (!isSupportedLocale(currentLocales.get(0))) {
    LocaleList supportedLocales = filterUnsupportedLocales(currentLocales);
    if (!supportedLocales.isEmpty()) {
      config.setLocales(supportedLocales);
      // updateConfiguration() is deprecated in SDK 25, but the alternative
      // requires restarting the activity, which we don't want to do here.
      resources.updateConfiguration(config, resources.getDisplayMetrics());
    }
  }
}

However, not only is your current Activity affected, but it appears to the system that the entire application also supports all of Chrome’s locales. After a configuration change (such as rotating the screen) the user’s top preference will again be used, and settings like Locale.getDefault() will be overwritten.

Once your app’s locales are polluted, the only way to maintain correct locales through configuration changes is to wrap your Activity’s base context with overriding values as follows.

@Override
protected void attachBaseContext(Context base) {
  if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
    LocaleList currentLocales = base.getResources().getConfiguration().getLocales();
    if (!isSupportedLocale(currentLocales.get(0))) {
      LocaleList supportedLocales = filterUnsupportedLocales(currentLocales);
      if (!supportedLocales.isEmpty()) {
        Configuration config = new Configuration();
        config.setLocales(supportedLocales);
        base = base.createConfigurationContext(config);
      }
    }
  }
  super.attachBaseContext(base);
}

Not defined above are isSupportedLocale() and filterUnsupportedLocales(); you must implement these yourself. Unfortunately it appears that there is no official, public API to determine at runtime what locales an app actually supports:

You can bite the bullet and call getNonSystemLocales() by reflection, or you can do the following.

Above we already added a list of supported locales in order to filter the app’s resources. We can simply make this list available at runtime as a field in BuildConfig.

In build.gradle:

ext {
  // Include the default language and base languages for runtime use
  supportedLocales = ['en', 'de', 'es', 'es-rMX', 'b+sr+Latn']
  // Convert Android locale qualifier values to standard identifiers
  // - Remove the 'r' before the region
  // - Remove the 'b+' prefix on BPC 47 tags
  // - Change '+' to '-' in BPC 47 tags
  resToLoc = { res -> res.replaceAll(/-r/, '-').replaceAll(/^b\+/, '').replaceAll(/\+/, '-') }
}

android {
  defaultConfig {
    resConfigs supportedLocales
    buildConfigField "String[]", "LOCALES", '{"' + supportedLocales.collect { resToLoc(it) }.join('","') + '"}'
  }
}

In the app:

@RequiresApi(api = Build.VERSION_CODES.N)
public LocaleList filterUnsupportedLocales(LocaleList locales) {
  List<Locale> filtered = new ArrayList<>(locales.size());
  for (int i = 0; i < locales.size(); i++) {
    Locale loc = locales.get(i);
    if (isSupportedLocale(loc)) {
      filtered.add(loc);
    }
  }
  return new LocaleList(filtered.toArray(new Locale[filtered.size()]));
}

@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
public boolean isSupportedLocale(Locale locale) {
  for (int i = 0; i < BuildConfig.LOCALES.length; i++) {
    String loc = BuildConfig.LOCALES[i];
    if (loc.equals(locale.getLanguage()) || loc.equals(locale.toLanguageTag())) {
      return true;
    }
  }
  return false;
}

See Also:

New locales vs old locales: Chinese

In Android 7 many new locales are supported by default. It can be tricky to support both old locales and new locales correctly in some cases; here I will discuss one I happen to be aware of: Chinese.

Basic background:

  • Chinese is written in two different scripts: Simplified and Traditional
  • Each Chinese-speaking region generally uses just one script

While ideally one would localize for each region, we will assume here that we have just one resource set for each script.

Prior to Android 7, the following Chinese locales were available:

  • zh-CN (Simplified)
  • zh-TW (Traditional)
  • In some cases:
    • zh-SG (Simplified)
    • zh-HK (Traditional)
    • zh-MO (Traditional)

A common resource layout scheme to support the above locales while minimizing resource duplication would be:

  • values-zh: Traditional
    • values-zh-rCN: Simplified
    • values-zh-rSG: Simplified

In other words Traditional resources are put at the root, and zh-TW, zh-HK, and zh-MO are covered by fallback.

In Android 7, the older language-region locales are gone, replaced by the following:

  • zh-Hans-CN
  • zh-Hans-MO
  • zh-Hans-HK
  • zh-Hans-SG
  • zh-Hant-TW
  • zh-Hant-HK
  • zh-Hant-MO

Note:

  • The script and region are specified separately
  • There are now default locales specifying Simplified script in traditionally Traditional regions: zh-Hans-MO and zh-Hans-HK.

Problems using the old scheme in Android 7:

  • zh-Hans-* falls back to zh before any children of zh, and thus would appear as Traditional instead of Simplified
    • zh-Hans does not appear to be recognized at all
    • This indicates a preference for zh to be Simplified, not Traditional. However this is not clear from the SDK itself, which has only zh-CN, zh-HK, and zh-TW resources.
  • zh-Hant-* falls back to zh-Hant and then the default, and thus would appear as en

Just zh and zh-Hant are sufficient for covering the Android 7 locales, but we need to maintain support for Android 6 and earlier. Thus the minimal correct resource layout is now:

  • values-zh: Simplified
    • values-zh-rTW: Traditional
    • values-zh-rHK: Traditional
    • values-zh-rMO: Traditional
    • values-b+zh+Hans+HK: Simplified
    • values-b+zh+Hans+MO: Simplified

With this we get the desired behavior:

On Android 6 and earlier:

  • zh-CN and zh-SG fall back to zh (Simplified)
  • zh-TW, zh-HK, and zh-MO have specific resources (Traditional)

On Android 7:

  • zh-Hans-CN and zh-Hans-SG fall back to zh (Simplified)
  • zh-Hant-TW, zh-Hant-HK, and zh-Hant-MO fall back to their language-region locales (Traditional)
  • zh-Hans-HK and zh-Hans-MO have specific resources (Simplified)

Conclusion

Locale fallback was a long-awaited feature, but it is extremely hard to use correctly.

It’s hard to see the Chrome resource contamination issue as anything other than a bug; I hope that Google can either fix the issue or provide appropriate guidance.

The merging of dependency resources also seems like it must be a widespread pain point. Filtering out locales not provided by the app itself would seem to be a reasonable default behavior; perhaps this can be considered in the future.

@kenyee
Copy link

kenyee commented May 9, 2018

FYI, for layouts, you need them in layout-b+zh+Hant instead of only in layouts-zh or you'll find that Traditional Taiwanese uses the default english layouts with the wrong field layouts :-)

@ozmium
Copy link

ozmium commented Jun 24, 2018

This problem also happens if you have 2 English string resources in an Android app:

  • res/values/strings.xml (default English)
  • res/values-en-rUS/strings.xml (English variant for USA users)

On Android 6 and below, it works fine, as expected. But on Android 7, the app will display en-US strings for all English locales! So when the locale is English-Australia or English-Great Britain, the app will incorrectly display English-USA strings.

The only workaround I found was to add an additional English variant, like this:

  • res/values-en/strings.xml (default English copy, to workaround Android 7 localization bug)

Somehow, this seems to fix it for all English locales, such as en-GB, en-AU, and en (no country).

@aljosamrak
Copy link

You don't have to define so many different resources to cover all the cases, because Android handles simplified Chinese and traditional Chines as two different languages.

For example:

  • App supports default and zh (simplified)
  • The user selects zh-TW (traditional)

By following the logic and explanation in the documentation zh-TW should fallback to zh and the texts should be in simplified Chinese which is wrong. In practice, this is not true. In practice, the default local is selected.

There a few more examples, but this is the simplest.

If you define just these 2, it should work everywhere: values-b+zh, values-b+zh+hant+TW.
I tested it on Android 6.0.1, Android 8.0.0 and Android 10

This is the expected output:
zh → zh-hans simplified
zh-CN → zh-hans simplified
zh-TW → zh-hant traditional
zh-SG → zh-hans simplified
zh-HK → zh-hant traditional
zh-MO → zh-hant traditional
zh-Hans-CN → zh-hans simplified
zh-Hans-MO → zh-hans simplified
zh-Hans-HK → zh-hans simplified
zh-Hans-SG → zh-hans simplified
zh-Hant-TW → zh-hant traditional
zh-Hant-HK → zh-hant traditional
zh-Hant-MO → zh-hant traditional

Official documentation: https://developer.android.com/guide/topics/resources/multilingual-support

@FrancisDx
Copy link

It seems to be fixed on Android V
https://issuetracker.google.com/issues/109833940

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment