Dynamic templates In Index Template

dynamic_templates_in_index_template.md

For practicality setup env variable to point to OpenSearch Node HTTP endpoint:

export OSS=localhost:9200

Start a fresh OpenSearch. In this case it is OpenSearch version 2.19.3 and it is the Min distribution (without most of the plugins that are found in standard distro, especially without security plugin which makes it easy to setup without security configuration).

curl -s -X GET "http://${OSS}/" | jq .version.number

# Output:
"2.19.3"

Create index template that contains dynamic templates:

• In this case any field starting with a priority prefix will be converted and indexed as a short type.
• Any string field will be indexed as a text field (ie. ideal for full text search) and a second field with .raw suffix will created as a keyword type (ie. ideal for exact match).

curl -X PUT \
     -H 'Content-Type: application/json' \
     http://${OSS}/_index_template/my_template -d \
'{
  "index_patterns": [
    "logs-*"
  ],
  "template": {
    "mappings": {
      "dynamic_templates": [
        {
          "priority_field": {
            "path_match": "priority*",
            "match_mapping_type": "string",
            "mapping": {
              "type": "short"
            }
          }
        },
        {
          "strings_as_keyword": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "text",
              "fields": {
                "raw": {
                  "type": "keyword"
                }
              }
            }
          }
        }
      ],
      "properties": {
      }
    }
  }
}'

List index templates (there is only my_template template):

curl -X GET "http://${OSS}/_cat/templates?v"

# Output:
name        index_patterns order version composed_of
my_template [logs-*]       0             []

List indices (assuming this is a cluster without any internal plugins, like security, ISM, ... so there should be no indices):

curl -X GET "http://${OSS}/_cat/indices?v"

Index a new document:

curl -X POST \
     -H 'Content-Type: application/json' \
     http://${OSS}/logs-index/_doc -d \
'{
  "name": "John  Doe",
  "priority": "-3.5"
}'

A new index has been created and we can check its mapping:

curl -X GET \
     -H 'Content-Type: application/json' \
     "http://${OSS}/logs-index/_mappings?pretty"

Now, we can correctly search for all documents having priority value lower than 2. (Hint: This would not work correctly without the dynamic template)

curl -X GET \
     -H 'Content-Type: application/json' \
     http://${OSS}/logs-index/_search -d \
'{
  "query": {
    "range": {
      "priority": {
        "lt": 2
      }
    }
  }
}'

The following search is matching only documents having exact match on the name value (ie. including double space between given name and family name)

curl -X GET \
     -H 'Content-Type: application/json' \
     http://${OSS}/logs-index/_search -d \
'{
  "query": {
    "match": {
      "name.raw": "John  Doe"
    }
  }
}'

Important

There are two dynamic templates defined as part of the index template. Think about what would happen if they were defined in reverse order. That means "strings_as_keyword" first and "priority_field" second.

Hint: Conversion of the "priority" field to a short value would not happen and the search for documents having priority value lower than 2 would not work correctly.