Ansible, Jinja2 and Types

Strings, strings everywhere

Jinja2 is a templating engine and as such its primary use case is to render templates into text; which is usually HTML output saved into text file.

Consider the following Ansible play that uses Jinja2 templates to evaluate expressions:

- hosts: localhost
  gather_facts: no
  vars:
    foo: "{{ 1 + 2 }}"
  tasks:
    - debug:
        msg: "{{ foo + 3 }}"

The expectation is that the debug task would print 6. However this happens instead:

"Unexpected templating type error occurred on ({{ foo + 3 }}): can only concatenate str (not \"int\") to str"

The above error message implies that foo is a string. That is because Jinja2 renders strings because of its initial use case. So even though {{ 1 + 2 }} was evaluated to an integer of value 3 it is stored as string. To fix that, we need to explicitly cast foo to int if we want to use it as such:

- hosts: localhost
  gather_facts: no
  vars:
    foo: "{{ 1 + 2 }}"
  tasks:
    - debug:
        msg: "{{ foo|int + 3 }}"

safe_eval to the rescue

To remedy some of the conversions to strings, Ansible has a mechanism called safe_eval which in specific cases converts "stringified" values back to their original type. Consider the specific example:

- hosts: localhost
  gather_facts: no
  vars:
    a_list:
      - 1
      - 2
  tasks:
    - debug:
        msg: "{{ item }}"
      loop: "{{ a_list }}"

Without safe_eval the above playbook would fail because loop expects a list but Jinja2 evaluates "{{ a_list }}" to string '[1, 2]'. safe_eval converts this back to a list which allows the loop to loop over the list.

It is called safe_eval because it tries to ensure that the data are safe, for example it prevents arbitrary functions to be evaluated which could be dangerous because safe_eval is executed outside of constrained Jinja2 environment.

Sometimes safe_eval does not do the right thing which could result into unexpected conversions. Consider the following Ansible play:

- hosts: localhost
  gather_facts: no
  vars:
    foo: '{1,2,3}'
  tasks:
    - debug:
        msg: "{{ foo }}"

Here safe_eval converts foo to a set ({1,2,3} is a set in Python) which may not be the intent of play's author. The recommended work around for this is to explicitly convert to desired type:

- hosts: localhost
  gather_facts: no
  vars:
    foo: '{1,2,3}'
  tasks:
    - debug:
        msg: "{{ foo | string }}"

In other example, JSON data are misinterpreted as a Python dictionary. To work around that, use to_json filter:

  tasks:
    - debug:
        msg: "{{ json_data_var | to_json }}"

Using to_json and a few other filters (see STRING_TYPE_FILTERS configuration option) actually bypasses safe_eval.

Other examples that might end up with unexpected casting

None would become an empty string (see Tempar._finalize function)
default(1) renders "1" (string) instead of 1 (int)
mode: 0644 would result in mode being 420 (integer); the solution is to mark the mode as a string: mode: "0644"

Conclusion

Generally speaking one cannot rely on type at the time the variable is defined because Jinja2 would change it to string with some exceptions that safe_eval is able to handle. Types need to be forced via filters on consumption of variables.

Enter Native Python Types

The shortcomings of safe_eval led to introducing an alternative solution to types in Jinja2. Since version 2.10 Jinja2 offers Native Python Types functionality that introduces a possibility that rendering a template produces a native Python type.

This functionality has been integrated into Ansible since version 2.7. It is off by default as of Ansible 2.10 but can be enabled through config option or setting ANSIBLE_JINJA2_NATIVE environment variable. Enabling this feature will bypass safe_eval.

With the native types functionality enabled, the first example works as expected without any explicit casting:

- hosts: localhost
  gather_facts: no
  vars:
    foo: "{{ 1 + 2 }}"
  tasks:
    - debug:
        msg: "{{ foo + 3 }}"

TASK [debug] ***********************************
ok: [localhost] => {
    "msg": "6"
}

Due to an issue when native jinja prevented the template module/lookup from returning JSON, starting with Ansible 2.11 (not released at the time of writing) the native jinja is always disabled for the template module. For the template lookup the feature is disabled by default as well but offers an option to enable it, like so:

lookup('template.j2', jinja2_native=True)

Notes

While Native Python Types were introduced in Jinja2 2.10, it is recommended to use 2.11 version as it includes important bugfixes for the functionality.
Since Native Python Types treats expressions differently and your playbooks might make assumptions based on how Jinja2 has been historically working, properly test your playbooks after enabling the feature to prevent undesirable behavior.

mkrizek/ansible-jinja-types.md