ansible - example of using filters to change each item in a list

00_description

The problem:

I wanted to use the jinja 'map' filter to modify each item in a string, in this simple 
example, adding '.conf' to each item.

The 'format' filter in jinja takes arguments (value, *args, **kwargs).  Unfortunately, 
it uses 'value' as the pattern. When called inside map 'value' is the current item in 
the list, or in other words *args as far as format is concerned. So it's the wrong way 
around.

The following example creates a simple filter that has the format args in an order we 
can use when calling 'map'.  Note this is extremely simplestic, and only allows for 
adding a prefix or suffix to an item. i.e. prefix%s or %ssuffix.  You can't do more 
advanced ones like '%s - %s' with this example as that requires passing the current 
item as an array and (at least in python 2x), variable length args have to come after 
named args.

_Note: A better approach might be to use the ansible regex_replace filter within map, 
but when I tried that, it returned \u0001.conf, so I'm guessing that there's some 
unicode foo (or lack thereof) happening. i.e. I tried
  mylist2: "{{ mylist | map('regex_replace', '^(.*)$', '\\1.conf' ) | list }}" 

01_map_format.py

# plugins/filter/map_format.py (check path in ansible.cfg)

# This code is essentially a direct copy of the jinja 'format' filter with
# some minor mods to the args and their use, see do_format in
# https://github.com/mitsuhiko/jinja2/blob/master/jinja2/filters.py
# for the original

from jinja2.utils import soft_unicode

def map_format(value, pattern):
    """
    Apply python string formatting on an object:
    .. sourcecode:: jinja
        {{ "%s - %s"|format("Hello?", "Foo!") }}
            -> Hello? - Foo!
    """
    return soft_unicode(pattern) % (value)

class FilterModule(object):
    ''' jinja2 filters '''

    def filters(self):
        return {
            'map_format': map_format,
        }

02_play.yml

---
- hosts: all 
  remote_user: vagrant
  sudo: true
  vars:
    mylist: 
      - Alice
      - Bob 
      - Carol
    mylist2: "{{ mylist | map('map_format', '%s.conf' ) | list }}" 
  tasks:
    - debug: var=mylist2

03_output

...
TASK: [debug var=mylist2] ***************************************************** 
ok: [vagrant] => {
    "var": {
        "mylist2": [
            "Alice.conf", 
            "Bob.conf", 
            "Carol.conf"
        ]
    }
}
...

When there is a need to add both prefix and suffix (and making everything a list), look at:

  set_fact:
    extended_etcd_endpoints_list: "{{ groups['etcd'] | map('extract', hostvars, ['ansible_default_ipv4','address']) | map('regex_replace', '^(.*)$','https://\\1:2379') | list  }}"

What is does: takes the list of all machines in the group etcd, extracts the ipv4 address, adds a prefix of 'https://' and a suffix of ':2379'.
At the end, everything is transformed to a list.

When there is a need to add both prefix and suffix (and making everything a list), look at:

  set_fact:
    extended_etcd_endpoints_list: "{{ groups['etcd'] | map('extract', hostvars, ['ansible_default_ipv4','address']) | map('regex_replace', '^(.*)$','https://\\1:2379') | list  }}"

What is does: takes the list of all machines in the group etcd, extracts the ipv4 address, adds a prefix of 'https://' and a suffix of ':2379'.
At the end, everything is transformed to a list.

I wish I could upvote this somehow!!!! Thank you!!!!!!! :+10000:

from jinja2.utils import soft_unicode

def _format(value, pattern):
    return soft_unicode(pattern) % (value)

def prefix(value, prefix):
    return soft_unicode(prefix + value)

def suffix(value, suffix):
    return soft_unicode(value + suffix)

class FilterModule(object):
    ''' jinja2 filters '''

    def filters(self):
        return {
            'format': _format,
            'prefix': prefix,
            'suffix': suffix
        }

# mylist -> ["a", "b", "c"]
mylist1: "{{ mylist | map('prefix', 'mod_') | list }}" # -> ["mod_a", "mod_b", "mod_c"]
mylist2: "{{ mylist | map('suffix', '.conf' ) | list }}" # -> ["a.conf", "b.conf", "c.conf"]
mylist3: "{{ mylist | map('format', 'mod_%s.conf' ) | list }}" # -> ["mod_a.conf", "mod_b.conf", "mod_c.conf"]

For more complex cases, It would be better to use regex_replace.