Repro: logstash-docs PR #2102 filter-elasticsearch ES|QL example missing query_type => "esql"

README.md

Repro: logstash-filter-elasticsearch ES|QL example missing query_type => "esql"

This reproduces a documentation bug in elastic/logstash-docs#2102.

The bug

The ES|QL example in docs/plugins/filters/elasticsearch.asciidoc sets query and query_params but never sets query_type. The plugin defaults query_type to "dsl" (source), and in DSL mode it rejects query_params outright at registration (source).

So the documented example, copied verbatim, aborts pipeline startup. The check runs in register before test_connection!, so no Elasticsearch is needed to reproduce it.

Run it

chmod +x run.sh
./run.sh

Requires Docker. The script installs logstash-filter-elasticsearch v4.4.1 (the version the 9.5 docs pin) into a Logstash 8.17.4 container and runs two pipelines.

Expected output

bad.conf (the doc example verbatim) fails at startup:

LogStash::ConfigurationError: `query_params` is not allowed when `query_type => 'dsl'`.
  logstash-filter-elasticsearch-4.4.1/lib/logstash/filters/elasticsearch.rb:453:in `validate_query_settings'
  ...:440:in `validate_dsl_query_settings!'
  ...:199:in `register'

good.conf (same config + query_type => "esql") gets past validation and only fails later at test_connection! because the ES host is fake:

Elastic::Transport::Transport::Error: Connect to localhost:9200 ... Connection refused
  logstash-filter-elasticsearch-4.4.1/lib/logstash/filters/elasticsearch.rb:407:in `test_connection!'
  ...:210:in `register'

The error moving from line 453 (validation) to line 407 (connection) when only query_type => "esql" is added is the proof: the missing query_type is what breaks the documented example.

Fix

Add query_type => "esql" to the example in the docs, and state that it's required for ES|QL queries.

Note

The stock docker.elastic.co/logstash/logstash:8.17.4 image ships a logstash-filter-elasticsearch that fails to load on its own (LoadError: elasticsearch/transport/transport/http/manticore). That's a separate image-packaging issue; run.sh sidesteps it by installing v4.4.1.

bad.conf

# Reproduces the broken ES|QL example from the logstash-docs PR verbatim.
# The docs show `query` + `query_params` with NO `query_type`, which defaults
# to "dsl". In DSL mode the plugin rejects `query_params` at register time,
# so the pipeline aborts at startup (no Elasticsearch required).
input { generator { count => 1 } }
filter {
  elasticsearch {
    hosts => ["https://localhost:9200"]
    api_key => "id:apikey"
    query => '
      FROM food-index
        | WHERE id == ?food_id
    '
    query_params => {
      "food_id" => "[food][id]"
    }
  }
}
output { stdout {} }

good.conf

# Same config as bad.conf, plus the one missing line: query_type => "esql".
# The `query_params` ConfigurationError disappears; the plugin now gets past
# validation and only fails later at test_connection! (because the ES host
# here is fake). That contrast is the proof.
input { generator { count => 1 } }
filter {
  elasticsearch {
    hosts => ["https://localhost:9200"]
    api_key => "id:apikey"
    query_type => "esql"
    query => '
      FROM food-index
        | WHERE id == ?food_id
    '
    query_params => {
      "food_id" => "[food][id]"
    }
  }
}
output { stdout {} }

run.sh

#!/usr/bin/env bash
# Reproduces finding #1 from elastic/logstash-docs PR #2102:
# the filter-elasticsearch ES|QL example is missing `query_type => "esql"`,
# so copied verbatim it aborts pipeline startup with a ConfigurationError.
#
# Usage: ./run.sh
# Requires: Docker running. No Elasticsearch needed (the check is at config
# validation time, before any connection attempt).

set -euo pipefail

IMAGE="docker.elastic.co/logstash/logstash:8.17.4"
# Match the plugin version the 9.5 docs pin. The stock image ships a
# filter-elasticsearch that fails to load on its own (a separate image bug),
# so we install the pinned version explicitly.
PLUGIN_VERSION="4.4.1"
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

run_case() {
  local conf="$1"
  echo "==================================================================="
  echo "Running: $conf"
  echo "==================================================================="
  docker run --rm \
    -v "$DIR/$conf:/usr/share/logstash/pipeline/logstash.conf:ro" \
    -e XPACK_MONITORING_ENABLED=false \
    --entrypoint sh \
    "$IMAGE" \
    -c "bin/logstash-plugin install --version $PLUGIN_VERSION logstash-filter-elasticsearch >/tmp/install.log 2>&1 \
        && logstash 2>&1 \
        | grep -iE 'query_params|ConfigurationError|Connection refused|test_connection|Pipeline error' \
        | head -20"
  echo
}

# bad.conf  -> ConfigurationError: `query_params` is not allowed when `query_type => 'dsl'`.
run_case bad.conf

# good.conf -> that error is gone; fails later at test_connection! instead.
run_case good.conf

echo "Done. 'bad.conf' fails at register (validate_query_settings)."
echo "'good.conf' passes validation and only fails on the (fake) ES connection."