Skip to content

Instantly share code, notes, and snippets.

@sanemat

sanemat/match-multi-tags.md

Last active August 29, 2015 14:09
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save sanemat/5a186db8c69b5d891b82 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save sanemat/5a186db8c69b5d891b82 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
match-multi-tags.md

Expect

matchの複数指定をこう書きたい

<match docker.host.** reformed.docker.containers.** nginx.access.** nginx.error.**>
  type forest
  subtype file_alternative
  <template>
    path /var/log/td-agent/%Y%m%d/${tag}.log.%Y%m%dT%H%M
    (snip)
  </template>
</match>

もしくは:

<match "docker.host.** reformed.docker.containers.** nginx.access.** nginx.error.**">
<match ["docker.host.**", "reformed.docker.containers.**", "nginx.access.**", "nginx.error.**"]>

TL;DR

matchに複数tag指定するときの挙動がtd-agent 2.1.0とtd-agent 2.1.2で替わったが、どこが原因なのかあんまり切り分けられてない。 v1configまわりなんだろうけど、forestかも(?)

詳細

matchに複数tag指定したいとき td-agent 1系では空白区切りで複数指定できる(?)ような記述がググると出てくる

td-agent 2.1.0 では、「\ 」区切りで複数指定できた(バグ挙動っぽいけど)

<match docker.host.**\ reformed.docker.containers.**\ nginx.access.**\ nginx.error.**>
  type forest
  subtype file_alternative
  <template>
    path /var/log/td-agent/%Y%m%d/${tag}.log.%Y%m%dT%H%M

    buffer_type file
    flush_interval 1s

    output_data_type    json
    output_include_time false
    output_include_tag  false
    include_time_key    true
    add_newline         true
  </template>
</match>

td-agent 2.1.2では、

<match docker.host.**\ reformed.docker.containers.**\ nginx.access.**\ nginx.error.**>
<match docker.host.** reformed.docker.containers.** nginx.access.** nginx.error.**>
<match "docker.host.** reformed.docker.containers.** nginx.access.** nginx.error.**">
<match "docker.host.**\ reformed.docker.containers.**\ nginx.access.**\ nginx.error.**">

がいずれもFluent::ConfigParseError

<match "docker.host.**\treformed.docker.containers.**\tnginx.access.**\tnginx.error.**">

はparse出来たけど、後ろ3つのtagがmatchしない

# service td-agent configtest
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/basic_parser.rb:86:in `parse_error!': expected '>' at td-agent.conf line 1,22 (Fluent::ConfigParseError)
  1: <match docker.host.** reformed.docker.containers.** nginx.access.** nginx.error.**>

     ----------------------^
  2:   type forest
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/v1_parser.rb:84:in `parse_element'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/v1_parser.rb:40:in `parse!'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/v1_parser.rb:30:in `parse'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config.rb:30:in `parse'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/supervisor.rb:385:in `apply_system_config'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/supervisor.rb:110:in `initialize'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/command/fluentd.rb:168:in `new'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/command/fluentd.rb:168:in `<top (required)>'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/bin/fluentd:6:in `<top (required)>'
        from /opt/td-agent/embedded/bin/fluentd:23:in `load'
        from /opt/td-agent/embedded/bin/fluentd:23:in `<top (required)>'
        from /usr/sbin/td-agent:7:in `load'
        from /usr/sbin/td-agent:7:in `<main>'

<match "docker.host.** reformed.docker.containers.** nginx.access.** nginx.error.**">

# service td-agent configtest
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/basic_parser.rb:86:in `parse_error!': expected '>' at td-agent.conf line 1,23 (Fluent::ConfigParseError)
  1: <match "docker.host.** reformed.docker.containers.** nginx.access.** nginx.error.**">

     -----------------------^
  2:   type forest
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/v1_parser.rb:84:in `parse_element'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/v1_parser.rb:40:in `parse!'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/v1_parser.rb:30:in `parse'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config.rb:30:in `parse'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/supervisor.rb:385:in `apply_system_config'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/supervisor.rb:110:in `initialize'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/command/fluentd.rb:168:in `new'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/command/fluentd.rb:168:in `<top (required)>'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/bin/fluentd:6:in `<top (required)>'
        from /opt/td-agent/embedded/bin/fluentd:23:in `load'
        from /opt/td-agent/embedded/bin/fluentd:23:in `<top (required)>'
        from /usr/sbin/td-agent:7:in `load'
        from /usr/sbin/td-agent:7:in `<main>'

# service td-agent configtest
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/basic_parser.rb:86:in `parse_error!': expected '>' at td-agent.conf line 1,24 (Fluent::ConfigParseError)
  1: <match "docker.host.**\ reformed.docker.containers.**\ nginx.access.**\ nginx.error.**">

     ------------------------^
  2:   type forest
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/v1_parser.rb:84:in `parse_element'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/v1_parser.rb:40:in `parse!'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config/v1_parser.rb:30:in `parse'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/config.rb:30:in `parse'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/supervisor.rb:385:in `apply_system_config'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/supervisor.rb:110:in `initialize'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/command/fluentd.rb:168:in `new'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/lib/fluent/command/fluentd.rb:168:in `<top (required)>'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.10.57/bin/fluentd:6:in `<top (required)>'
        from /opt/td-agent/embedded/bin/fluentd:23:in `load'
        from /opt/td-agent/embedded/bin/fluentd:23:in `<top (required)>'
        from /usr/sbin/td-agent:7:in `load'
        from /usr/sbin/td-agent:7:in `<main>'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment