Skip to content

Instantly share code, notes, and snippets.

@edudant

edudant/reindex-order.md

Created October 16, 2025 12:20
Show Gist options
  • Save edudant/cfb1bba77c66a70a03b916b0df117bab to your computer and use it in GitHub Desktop.
Save edudant/cfb1bba77c66a70a03b916b0df117bab to your computer and use it in GitHub Desktop.
Download ZIP
Raw
reindex-order.md

Migrace Elasticsearch indexu order

Script pro změnu nastavení indexu a/nebo mapování.

0) Proměnné prostředí

export BASE_URL="https://elastic-tsm-kip-ppt-a-moje.o2.cz"
export SRC_INDEX="o2-kiporder_20250904_142206"
export DEST_INDEX="o2-kiporder_20251016_000001"
export ALIAS="o2-kiporder"

# přihlašování
export AUTH="" # -uUSER:PASS

1) Získám stávající mapování indexu order

curl -sS $AUTH \
  -H 'Content-Type: application/json' \
  "$BASE_URL/$SRC_INDEX/_mapping" | jq .

Tip: jen properties bez nastavení lze získat např. ?filter_path=**.properties.

2) Vytvořím kopii indexu s novým názvem (prázdný index se shodným mappingem & settings)

Vlož svůj JSON se settings/mappings:

curl -sS -X PUT $AUTH \
  -H 'Content-Type: application/json' \
  "$BASE_URL/$DEST_INDEX" \
  --data-binary @- <<'JSON'
{
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_keyword": {
          "match_mapping_type": "string",
          "mapping": {
            "normalizer": "ascii-lower",
            "type": "keyword"
          }
        }
      }
    ],
    "properties": {
      "_class": {
        "type": "keyword",
        "index": false,
        "doc_values": false
      },
      "_timestamp": {
        "type": "date",
        "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
      },
      "accountId": {
        "type": "keyword"
      },
      "chars": {
        "type": "flattened"
      },
      "charsTyped": {
        "type": "object"
      },
      "custom": {
        "type": "object"
      },
      "customerId": {
        "type": "keyword"
      },
      "dataTags": {
        "type": "keyword",
        "normalizer": "ascii-lower"
      },
      "deliveryDate": {
        "type": "date",
        "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
      },
      "deliveryDateRequested": {
        "type": "date",
        "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
      },
      "description": {
        "type": "keyword",
        "normalizer": "ascii-lower"
      },
      "entityCatalogSpecification": {
        "properties": {
          "_class": {
            "type": "keyword",
            "index": false,
            "doc_values": false
          },
          "catalogType": {
            "type": "keyword"
          },
          "code": {
            "type": "keyword"
          },
          "entityCatalogSpecificationType": {
            "properties": {
              "_class": {
                "type": "keyword",
                "index": false,
                "doc_values": false
              },
              "code": {
                "type": "keyword"
              }
            }
          },
          "id": {
            "type": "keyword"
          }
        }
      },
      "entityCatalogSpecificationId": {
        "type": "keyword"
      },
      "entityInstanceConfigurationId": {
        "type": "keyword"
      },
      "filteredNested": {
        "type": "object"
      },
      "id": {
        "type": "keyword"
      },
      "joinField": {
        "type": "join",
        "eager_global_ordinals": true,
        "relations": {
          "Order": "OrderLine"
        }
      },
      "key": {
        "type": "keyword",
        "normalizer": "ascii-lower"
      },
      "keyNormalized": {
        "type": "keyword",
        "normalizer": "ascii-lower"
      },
      "lineType": {
        "type": "keyword"
      },
      "name": {
        "type": "keyword",
        "normalizer": "ascii-lower"
      },
      "operation": {
        "type": "keyword"
      },
      "orderDate": {
        "type": "date",
        "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
      },
      "orderId": {
        "type": "keyword"
      },
      "orderStatus": {
        "properties": {
          "_class": {
            "type": "keyword",
            "index": false,
            "doc_values": false
          },
          "code": {
            "type": "keyword",
            "normalizer": "ascii-lower"
          },
          "id": {
            "type": "keyword"
          },
          "name": {
            "type": "keyword",
            "normalizer": "ascii-lower"
          }
        }
      },
      "orderTypeCode": {
        "type": "keyword",
        "normalizer": "ascii-lower"
      },
      "orderTypeId": {
        "type": "keyword"
      },
      "orderTypeName": {
        "type": "keyword",
        "normalizer": "ascii-lower"
      },
      "parentOrderLineId": {
        "type": "keyword"
      },
      "personRoleIds": {
        "type": "keyword"
      },
      "process": {
        "properties": {
          "_class": {
            "type": "keyword",
            "index": false,
            "doc_values": false
          },
          "code": {
            "type": "keyword"
          },
          "processDefinitionVersionTag": {
            "type": "keyword"
          },
          "version": {
            "type": "integer"
          }
        }
      },
      "scriptFields": {
        "type": "object"
      },
      "subject": {
        "type": "keyword",
        "normalizer": "ascii-lower"
      },
      "tasks": {
        "type": "nested",
        "properties": {
          "_class": {
            "type": "keyword",
            "index": false,
            "doc_values": false
          },
          "_timestamp": {
            "type": "date",
            "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
          },
          "active": {
            "type": "boolean"
          },
          "activityId": {
            "type": "keyword"
          },
          "canceled": {
            "type": "boolean"
          },
          "chars": {
            "type": "flattened",
            "ignore_above": 8191
          },
          "charsTyped": {
            "type": "object"
          },
          "commitedStartTimeFrom": {
            "type": "date",
            "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
          },
          "commitedStartTimeTo": {
            "type": "date",
            "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
          },
          "custom": {
            "type": "object"
          },
          "description": {
            "type": "keyword"
          },
          "dueDate": {
            "type": "date",
            "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
          },
          "endTime": {
            "type": "date",
            "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
          },
          "entityType": {
            "type": "keyword"
          },
          "executionId": {
            "type": "keyword"
          },
          "followupDate": {
            "type": "date",
            "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
          },
          "history": {
            "type": "nested",
            "properties": {
              "_class": {
                "type": "keyword",
                "index": false,
                "doc_values": false
              },
              "changes": {
                "type": "keyword"
              },
              "duration": {
                "type": "long"
              },
              "endTime": {
                "type": "date",
                "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
              },
              "startTime": {
                "type": "date",
                "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
              },
              "status": {
                "type": "keyword"
              },
              "userGroupCode": {
                "type": "keyword"
              },
              "userId": {
                "type": "keyword"
              }
            }
          },
          "id": {
            "type": "keyword"
          },
          "incident": {
            "properties": {
              "_class": {
                "type": "keyword",
                "index": false,
                "doc_values": false
              },
              "date": {
                "type": "date",
                "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
              },
              "id": {
                "type": "keyword"
              },
              "message": {
                "type": "keyword",
                "normalizer": "ascii-lower"
              },
              "resolved": {
                "type": "boolean"
              }
            }
          },
          "module": {
            "type": "keyword"
          },
          "name": {
            "type": "keyword"
          },
          "plannedEndTime": {
            "type": "date",
            "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
          },
          "plannedStartTime": {
            "type": "date",
            "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
          },
          "processDefinitionId": {
            "type": "keyword"
          },
          "processInstanceId": {
            "type": "keyword"
          },
          "skills": {
            "type": "keyword"
          },
          "startTime": {
            "type": "date",
            "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
          },
          "status": {
            "type": "keyword"
          },
          "taskDefinitionKey": {
            "type": "keyword"
          },
          "taskSpecification": {
            "type": "keyword"
          },
          "taskType": {
            "type": "keyword"
          },
          "tsmTaskDefinitionCode": {
            "type": "keyword"
          },
          "userGroupCode": {
            "type": "keyword"
          },
          "userGroupId": {
            "type": "keyword"
          },
          "userId": {
            "type": "keyword"
          },
          "wfmUserId": {
            "type": "keyword"
          }
        }
      },
      "tsmModule": {
        "type": "keyword"
      },
      "whenEdited": {
        "type": "date",
        "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
      },
      "whenInserted": {
        "type": "date",
        "format": "uuuu-MM-dd'T'HH:mm:ss.SSSZ"
      },
      "whoEdited": {
        "type": "keyword"
      },
      "whoInserted": {
        "type": "keyword"
      }
    }
  },
  "settings": {
    "index": {
      "routing": {
        "allocation": {
          "include": {
            "_tier_preference": "data_content"
          }
        }
      },
      "number_of_shards": "4",
      "max_inner_result_window": "100000",
      "max_result_window": "100000",
      "analysis": {
        "filter": {
          "czech_keywords": {
            "keywords": [],
            "type": "keyword_marker"
          },
          "czech_stemmer": {
            "type": "stemmer",
            "language": "czech"
          },
          "czech_stop": {
            "type": "stop",
            "stopwords": [
              "_czech_"
            ]
          }
        },
        "normalizer": {
          "ascii-lower": {
            "filter": [
              "lowercase",
              "asciifolding"
            ],
            "type": "custom",
            "char_filter": []
          }
        },
        "analyzer": {
          "lowerCaseCzechAnalyzer": {
            "filter": [
              "lowercase",
              "czech_stop",
              "czech_keywords",
              "czech_stemmer"
            ],
            "type": "custom",
            "tokenizer": "standard"
          }
        }
      },
      "load_fixed_bitset_filters_eagerly": "true",
      "number_of_replicas": "1"
    }
  }
}
JSON

3) Překopírování dat do nového indexu (bulk reindex, poslední 1 hodina)

curl -sS -X POST $AUTH \
  -H 'Content-Type: application/json' \
  "$BASE_URL/_reindex?wait_for_completion=false" \
  --data-binary @- <<JSON
{
  "source": {
    "index": "$SRC_INDEX",
    "query": {
      "range": { "whenInserted": { "gte": "now-1h" } }
    }
  },
  "dest": { "index": "$DEST_INDEX" }
}
JSON

4) Ověření stavu kopírování

Tip: v browseru - https://elastic-tsm-kip-ppt-a-moje.o2.cz/_cat/tasks?detailed=true&pretty

curl -sS $AUTH "$BASE_URL/_tasks?detailed=true&group_by=parents" | jq .

Alternativa: ?detailed=true&group_by=parents

Případně lze uložit výstup do taskId:

export TASK_ID="$(curl -sS -X POST $AUTH -H 'Content-Type: application/json' \
  "$BASE_URL/_reindex?wait_for_completion=false" \
  --data-binary @- <<JSON
{ "source": { "index":"$SRC_INDEX", "query": { "range": { "whenInserted": { "gte": "now-1h" } } } }, "dest": { "index":"$DEST_INDEX" } }
JSON
| jq -r '.task')"
echo "Task ID: $TASK_ID"
curl -sS $AUTH "$BASE_URL/_tasks/$TASK_ID" | jq .

5) Změna aliasu na nový index

✅ Atomická změna aliasu:

curl -sS -X POST $AUTH -H 'Content-Type: application/json' \
  "$BASE_URL/_aliases" \
  --data-binary @- <<JSON
{
  "actions": [
    { "remove": { "index": "$SRC_INDEX",  "alias": "$ALIAS" } },
    { "add":    { "index": "$DEST_INDEX", "alias": "$ALIAS" } }
  ]
}
JSON

6) Doindexování dat vzniklých během kopírování (posledních 10 min)

curl -sS -X POST $AUTH \
  -H 'Content-Type: application/json' \
  "$BASE_URL/_reindex?wait_for_completion=false" \
  --data-binary @- <<JSON
{
  "conflicts": "proceed",
  "source": {
    "index": "$SRC_INDEX",
    "query": {
      "range": { "whenInserted": { "gte": "now-10m" } }
    }
  },
  "dest": { "index": "$DEST_INDEX", "op_type": "create" }
}
JSON

7) Doindexování historie (30 dní)

curl -sS -X POST $AUTH \
  -H 'Content-Type: application/json' \
  "$BASE_URL/_reindex?wait_for_completion=false" \
  --data-binary @- <<JSON
{
  "conflicts": "proceed",
  "source": {
    "index": "$SRC_INDEX",
    "query": {
      "range": { "whenInserted": { "gte": "now-30d" } }
    }
  },
  "dest": { "index": "$DEST_INDEX", "op_type": "create" }
}
JSON

8) Kontroly před přepnutím a po něm (doporučeno)

# počty dokumentů
curl -sS $AUTH "$BASE_URL/$SRC_INDEX/_count"  | jq .
curl -sS $AUTH "$BASE_URL/$DEST_INDEX/_count" | jq .

# zdraví clusteru + shards
curl -sS $AUTH "$BASE_URL/_cluster/health?level=indices" | jq .
curl -sS $AUTH "$BASE_URL/$DEST_INDEX/_settings" | jq .

9) Smazání původního indexu (až po ověření aliasu & dat)

curl -sS -X DELETE $AUTH \
  "$BASE_URL/$SRC_INDEX"

Tip: můžete nejprve /_close místo Delete, a po pár dnech teprve smazat.

Shrnutí bezpečného postupu (krátce)

  1. Vytvoř DEST index se shodným mappingem/settings.
  2. Reindex dat (malé okno, např. now-1h).
  3. Atomicky přepni alias ze SRC_INDEX → DEST_INDEX.
  4. Doindexuj „delta“ (now-10m).
  5. Doindexuj historická data (now-30d).
  6. Po validaci smaž SRC_INDEX.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment