Correct localization on Android 7

Prior to Android 7, the system had a single preferred locale, and fallback behavior was quite rudimentary. Starting with Android 7, the user can now specify a priority list of locales, and fallback behavior is improved.

However, in many cases it is still surprisingly difficult to make full use of locale fallback, and there are some hidden gotchas when trying to fully support both Android 7 and earlier versions.

What is fallback?

Assume a user has set the following list of preferred locales:

fr-FR
de-DE
en-US

Assume also that your app provides the following locales:

en (default)
de
es
es-MX
sr-Latn

When running your app on Android 7, this user should see the display in de because fr-FR (their first choice) is not provided, and de (their second choice) is preferred over en (their third choice).

On Android 6 or earlier, the app would be displayed in en: the user could only specify fr-FR as the preferred locale, and when not available the system would fall back to the app default locale.

What’s the problem?

On Android you can experience an issue I call language resource contamination (or just resource contamination for short). This is where resources for languages you don’t support “leak” into your app, making the system think you support them. When this happens, the fallback mechanism described above fails to work correctly.

I have identified two different kinds of resource contamination:

Compile-time contamination
Runtime contamination

Compile-time contamination

If you include the AppCompat v7 library (included by default in new projects created by Android Studio 3.1) or any of the Google Play Services libraries in your app, fallback will almost certainly be broken.

The reason is that these libraries contain resources for a huge number of locales; when building your APK these resources will be merged in, and it will appear to the system that your app supports all of these locales.

Under the initial example scenario, the system will believe that the app supports fr-FR when it actually does not supply any fr-FR versions of its “own” strings. Thus the display locale will become fr-FR, but each individual string will fall back to the default variant, and the app will appear to be shown in en.

The solution to this is to filter your app’s resources, allowing only the locales that you actually support. You can do this in build.gradle:

android {
  defaultConfig {
    // No need to specify the default locale here
    resConfigs 'de', 'es', 'es-rMX', 'b+sr+Latn'
  }
}

This will ensure that only resources for locales you actually support are included in the APK, allowing the OS to correctly determine the fallback locale at runtime. As a bonus, your APK will be a little bit smaller.

Prior to version 3.1.0 of the Android Gradle Plugin it was possible to specify resConfigs 'auto', where the appropriate locales would be automatically detected. However this is now deprecated, and you are recommended to manually list your supported locales.

Runtime contamination

Starting with Android 7, the WebView component is no longer the Android System WebView package, but is actually the Chrome app itself. This has some surprising consequences.

When first instantiating a WebView, the Chrome app will be loaded into your app’s current Activity. This apparently causes the current Activity’s preferred locale list to be overwritten with Chrome’s. Chrome supports a large number of locales, most likely including the user’s top preference. This then creates the same situation as described earlier, where the Activity’s locale may be set to e.g. fr-FR but all strings are displayed in en. This situation persists even after removing the WebView.

This issue was reported as issue 218310 but was closed as “WorkingAsIntended” but without a clear, supported fix offered.

One workaround is to fix up the current Activity’s locale immediately after loading the WebView:

WebView webView = (WebView) rootView.findViewById(R.id.webView);
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
  Resources resources = getResources();
  Configuration config = resources.getConfiguration();
  LocaleList currentLocales = config.getLocales();
  if (!isSupportedLocale(currentLocales.get(0))) {
    LocaleList supportedLocales = filterUnsupportedLocales(currentLocales);
    if (!supportedLocales.isEmpty()) {
      config.setLocales(supportedLocales);
      // updateConfiguration() is deprecated in SDK 25, but the alternative
      // requires restarting the activity, which we don't want to do here.
      resources.updateConfiguration(config, resources.getDisplayMetrics());
    }
  }
}

However, not only is your current Activity affected, but it appears to the system that the entire application also supports all of Chrome’s locales. After a configuration change (such as rotating the screen) the user’s top preference will again be used, and settings like Locale.getDefault() will be overwritten.

Once your app’s locales are polluted, the only way to maintain correct locales through configuration changes is to wrap your Activity’s base context with overriding values as follows.

@Override
protected void attachBaseContext(Context base) {
  if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
    LocaleList currentLocales = base.getResources().getConfiguration().getLocales();
    if (!isSupportedLocale(currentLocales.get(0))) {
      LocaleList supportedLocales = filterUnsupportedLocales(currentLocales);
      if (!supportedLocales.isEmpty()) {
        Configuration config = new Configuration();
        config.setLocales(supportedLocales);
        base = base.createConfigurationContext(config);
      }
    }
  }
  super.attachBaseContext(base);
}

Not defined above are isSupportedLocale() and filterUnsupportedLocales(); you must implement these yourself. Unfortunately it appears that there is no official, public API to determine at runtime what locales an app actually supports:

Locale.getAvailableLocales() returns all locales available to the Java runtime (hundreds)
LocaleList.getDefault() will include the top user-set locale, even if your app doesn’t support it
LocaleList.getAdjustedDefault() has the same problem as getDefault()
AssetManager.getLocales() returns all locales the instance has data for, which appears to be the locales provided with the OS (150+)
AssetManager.getNonSystemLocales() returns precisely what we are looking for! Except it’s marked @hide so you can only call it by reflection.

You can bite the bullet and call getNonSystemLocales() by reflection, or you can do the following.

Above we already added a list of supported locales in order to filter the app’s resources. We can simply make this list available at runtime as a field in BuildConfig.

In build.gradle:

ext {
  // Include the default language and base languages for runtime use
  supportedLocales = ['en', 'de', 'es', 'es-rMX', 'b+sr+Latn']
  // Convert Android locale qualifier values to standard identifiers
  // - Remove the 'r' before the region
  // - Remove the 'b+' prefix on BPC 47 tags
  // - Change '+' to '-' in BPC 47 tags
  resToLoc = { res -> res.replaceAll(/-r/, '-').replaceAll(/^b\+/, '').replaceAll(/\+/, '-') }
}

android {
  defaultConfig {
    resConfigs supportedLocales
    buildConfigField "String[]", "LOCALES", '{"' + supportedLocales.collect { resToLoc(it) }.join('","') + '"}'
  }
}

In the app:

@RequiresApi(api = Build.VERSION_CODES.N)
public LocaleList filterUnsupportedLocales(LocaleList locales) {
  List<Locale> filtered = new ArrayList<>(locales.size());
  for (int i = 0; i < locales.size(); i++) {
    Locale loc = locales.get(i);
    if (isSupportedLocale(loc)) {
      filtered.add(loc);
    }
  }
  return new LocaleList(filtered.toArray(new Locale[filtered.size()]));
}

@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
public boolean isSupportedLocale(Locale locale) {
  for (int i = 0; i < BuildConfig.LOCALES.length; i++) {
    String loc = BuildConfig.LOCALES[i];
    if (loc.equals(locale.getLanguage()) || loc.equals(locale.toLanguageTag())) {
      return true;
    }
  }
  return false;
}

New locales vs old locales: Chinese

In Android 7 many new locales are supported by default. It can be tricky to support both old locales and new locales correctly in some cases; here I will discuss one I happen to be aware of: Chinese.

Basic background:

Chinese is written in two different scripts: Simplified and Traditional
Each Chinese-speaking region generally uses just one script

While ideally one would localize for each region, we will assume here that we have just one resource set for each script.

Prior to Android 7, the following Chinese locales were available:

zh-CN (Simplified)
zh-TW (Traditional)
In some cases:
- zh-SG (Simplified)
- zh-HK (Traditional)
- zh-MO (Traditional)

A common resource layout scheme to support the above locales while minimizing resource duplication would be:

values-zh: Traditional
- values-zh-rCN: Simplified
- values-zh-rSG: Simplified

In other words Traditional resources are put at the root, and zh-TW, zh-HK, and zh-MO are covered by fallback.

In Android 7, the older language-region locales are gone, replaced by the following:

zh-Hans-CN
zh-Hans-MO
zh-Hans-HK
zh-Hans-SG
zh-Hant-TW
zh-Hant-HK
zh-Hant-MO

Note:

The script and region are specified separately
There are now default locales specifying Simplified script in traditionally Traditional regions: zh-Hans-MO and zh-Hans-HK.

Problems using the old scheme in Android 7:

zh-Hans-* falls back to zh before any children of zh, and thus would appear as Traditional instead of Simplified
- zh-Hans does not appear to be recognized at all
- This indicates a preference for zh to be Simplified, not Traditional. However this is not clear from the SDK itself, which has only zh-CN, zh-HK, and zh-TW resources.
zh-Hant-* falls back to zh-Hant and then the default, and thus would appear as en

Just zh and zh-Hant are sufficient for covering the Android 7 locales, but we need to maintain support for Android 6 and earlier. Thus the minimal correct resource layout is now:

values-zh: Simplified
- values-zh-rTW: Traditional
- values-zh-rHK: Traditional
- values-zh-rMO: Traditional
- values-b+zh+Hans+HK: Simplified
- values-b+zh+Hans+MO: Simplified

With this we get the desired behavior:

On Android 6 and earlier:

zh-CN and zh-SG fall back to zh (Simplified)
zh-TW, zh-HK, and zh-MO have specific resources (Traditional)

On Android 7:

zh-Hans-CN and zh-Hans-SG fall back to zh (Simplified)
zh-Hant-TW, zh-Hant-HK, and zh-Hant-MO fall back to their language-region locales (Traditional)
zh-Hans-HK and zh-Hans-MO have specific resources (Simplified)

Conclusion

Locale fallback was a long-awaited feature, but it is extremely hard to use correctly.

It’s hard to see the Chrome resource contamination issue as anything other than a bug; I hope that Google can either fix the issue or provide appropriate guidance.

The merging of dependency resources also seems like it must be a widespread pain point. Filtering out locales not provided by the app itself would seem to be a reasonable default behavior; perhaps this can be considered in the future.

amake/android-7-localization.org