Kibana i18n Proposal - 20161220

KibanaProposalArchive-6.md

Kibana Globalization Proposal

Purpose

• Provide translation readiness on the Kibana web-facing UI
• An integrated framework for federation of multiple globalization frameworks
• Delivered in a phased approach

Phase 1

Purpose

• Implement the globalization engine (I18n class) which provides a means to supply translations in a standard format that it not dependent on a localization framework
• Translate the Kibana welcome message which proves that the I18n class provides translations as registered for Kibana

Decide Language for Translation Algorithm

• The algorithm which decides the language for loading translations is as follows:
  1. First, do a direct comparison with the highest priority locale in the HTTP header “accept-language” against the registered translation languages. If comparison found then this is the language returned.
  2. Next, do a partial comparison of this locale whereby you try and get a language code comparison against the registered translation languages. For example, "fr" or "fr-FR" is used for all regions of French if comparison applies. If comparison found then this is the language returned.
  3. Next, repeat step 2 and if need be step 3 for the next highest priority locale in the “accept-language” list.
  4. Continue, until match found in step 2/step 3 or end of the “accept-language” list. If no match found then return empty language.

I18n Class

• Manages the language translations for Kibana

• Responsible for loading translated content per language

• The translations file are JSON files with a flat keyspace, where the keys are unique. This uniqueness between translation plugins could be achieved by prefixing the keys with the plugin name. The key signifies the translation ID which would be referenced in translatable files (like JS, HTML etc.).

• The key value is the translation string

• Example translation JSON file

en.json

    {
        "UI-WELCOME_MESSAGE": "Loading Kibana",        
        "UI-WELCOME_ERROR": "Kibana did not load properly. Check the server output for more information."
    }

• Core Kibana plugins like ‘kibana’ and ‘status_page’ could come with their own English translations bundled in
• API:
  • Return all translations for registered locales
    • getAllTranslations()
    • A Promise object where keys are the locale and values are Objects of translation keys and translations
  • Return all translations registered for the default locale:
    • getTranslationsForDefaultLocale()
    • A Promise for an object where keys are translation keys and values are translations
  • Return translations for a suitable locale from a user side locale list of BCP 47 language tags:
    • getTranslations(...languageTags)
    • A Promise for an object where keys are translation keys and values are translations
    • This object will contain all registered translations for the highest priority locale which is registered with the i18n module
    • This object can be empty if no locale in the language tags can be matched against the registered locales
  • Register translations:
    • registerTranslations(<absolute_path_to_translation_file>)
    • The path to the translation file is registered with i18n class

UiI18n Class

• Handles the interaction between the UI/server and localization (I18n class)
• Maps the accept-language header to BCP 47 tags
• Fetches the language as requested where the locale is decided by the translation algorithm
• It also substitutes any missing translations with the default locale translation

Tool for Verifying All Translations have Translatable Strings

• Grunt run task tool that tests all translatable strings have a translation (i.e. all translation IDs have a corresponding translation string)
• This could be run by CI to verify globalization end-to-end capability
• A possible solution:
  • For non-angular constructs: Enforce a pattern to be used. For example a i18n(<key>) function in the Jade template. A tool can then be used to find such pattern and extract the keys to file
  • The keys in the key file(s) would then be checked against the language translation files registered

Deliverable

• Translate the start-up message (“Kibana is loading ...”) and error message in the Jade template (Kibana Jade template)
• Translation message will be loaded using the language as per the deciding algorithm. For phase 1 the default 'en' langauge will be used.
• I18n unit tests
• Tool can be run by CI to verify translation IDs have a corresponding translation string
• Kibana core plugin registers its translation file during the initialization phase. The translation file contains strings for the welcome message and the start-up error message.

Phase 2

Purpose

• Provide the angular template for localization engineers to use
• Translate a view with an angular contruct and HTML which proves that UI localization frameworks integrates with i18n class to translate the Kibana views
• Create a boilerplate plugin which localization engineers can use to translate Kibana for different languages

Translations Available Client Side

• Make the translations available on the client side by:
  • Embedding the translations in the initial HTML payload:
    • Reduces complexity and removes a round trip
    • Evaluation required to test if it performs well and does not create a large payload
  • If the evaluation of HTML payload is non performant, then the following approaches should be investigated in this order:
    1. Use current bundle mechanism:
       • (Kibana Jade template) to call API and generate the bundle which will be loaded during start-up
       • The JavaScript bundle produced will be of the following format: i18n_<language>.bundle.js
       • Kibana loads all resource bundles on the client side after starting the single-page application
    2. Client side directly calls REST API and loads the JSON payload:
       • GET /i18n/translations ==> returns the English (or German or whatever) translations negotiated with the browser HTTP header “accept-language” priority list compared against the languages supported
       • GET /i18n/translationsForDefaultLocale ==> returns the default locale translations
  • All approaches will decide the language to be served up by using the language deciding algorithm (from Phase 1)

Translation Identifiers Added to Kibana UI

• Ids are added to the relevant view with Angular and HTML content
• English (en) translation file is generated for the Ids defined
• Approach for translating UI content:
  • Angular UI portion:
    • Use angular-translate for simplicity of the UI template resources, taking advantage of angular idioms.

BEFORE (HTML)

    <div class="sidebar-list">
    …

    <div class="sidebar-list-header">
        <h5>Selected Fields</h5>
    </div>

AFTER (HTML)

    <div class="sidebar-list">
    …

    <div class="sidebar-list-header">
        <h5 translate="FIELDS_SELECTEDFIELDS"></h5>
    </div>

Tool for Verifying All Translations have Translatable Strings (Update)

• Verify translation IDs in Angular templates
• Possible solutions are:
  • Use grunt-angular-translate . Searches all view and JS scripts to find angular-translate calls and extracts keys to file
  • Update current verification tool to search for angular-translate pattern in code and extract the keys
• The keys in the key file(s) would then be checked against the language translation files registered

Translation Plugin Boilerplate

• A boilerplate of a Kibana plugin which contains the minimal of actual code to enable registration of translations with the i18n class
• The translation plugin calls i18n class to register translations at the plugin initialization phase
• The plugin contains a translation JSON file per language
• An example translation plugin structure : Matt's template

Deliverable

• Angular View where globalization is enabled showing pattern to use
• Verification tool updated to check translations in angular-translate constructs
• [Elastic] Translatation and verification of xplugins (React)
• A bolierplate that localization engineers can use to produce language translations and integrate them in Kibana in an easy manner

Phase 3

Purpose

• Add the translation identifiers and English translation strings for all of the Kibana UI

Translation Identifiers Added to Kibana UI

• Ids are added to all the relevant UI content (HTML, JS etc.)
• English (en) translation file(s) are generated for the Ids defined

Deliverables

• English translation bundles are available so that localization engineers can start generating translation plugins for different languages

Phase 4

Purpose

• Implement a tool to generate the translation plugin

Translation Plugin Generator

• Tool which generates a translation plugin
• Localization engineer should only need to add translation file(s) within the plugin directory and add plugin to Kibana
• Deliverable:
  • Translation plugin is generated which can be installed as plugin in Kibana with he translation files registered during the plugin initialization phase

Deliverables

• Tool which localization engineers can use to generate a translation plugin

Blockers

Open Questions/Issues

• Translation of user data
• Server side set locale (ignore the client's requested locale)
• Provide mechanism in UI for user to manually switch their language from list of supported languages
• What about bi-directional language support? This affects language, charts and UI content support.
• UI views should be able to handle different languages (This means when switching from one language to another (for example, English to German) that the look and feel is maintained) :
  • Different languages can have variable content lengths. This means having adequate spacing to handle the strings in each language.
  • Need to consider different fonts. Some web fonts won't support all possible languages.

History

This section has links to prior versions of this issue text.

• 2016 December 14
• 2016 August 3
• 2016 August 2
• 2016 July 26
• 2016 July 20
• 2016 March 11

Authors

• Martin Hickey: @hickeyma
• Scott Russell: @DTownSMR
• Shikha Srivastava: @shikhasriva
• Steven R. Loomis: @srl295