Skip to content

Instantly share code, notes, and snippets.

@wyukawa
Last active December 12, 2015 10:29
Show Gist options
  • Select an option

  • Save wyukawa/4759914 to your computer and use it in GitHub Desktop.

Select an option

Save wyukawa/4759914 to your computer and use it in GitHub Desktop.
fluentdのretry処理時刻が過去になっている
# stacktrace
2013-02-12 04:38:39 +0900: failed to communicate hdfs cluster, path: ...
2013-02-12 04:38:39 +0900: temporarily failed to flush the buffer, next retry will be at 2013-02-12 04:37:10 +0900. error="{...
2013-02-12 04:38:39 +0900: /usr/local/lib/ruby/gems/1.9.1/gems/webhdfs-0.5.1/lib/webhdfs/client_v1.rb:278:in `request'
2013-02-12 04:38:39 +0900: /usr/local/lib/ruby/gems/1.9.1/gems/webhdfs-0.5.1/lib/webhdfs/client_v1.rb:232:in `operate_requests'
2013-02-12 04:38:39 +0900: /usr/local/lib/ruby/gems/1.9.1/gems/webhdfs-0.5.1/lib/webhdfs/client_v1.rb:46:in `append'
2013-02-12 04:38:39 +0900: /usr/local/lib/ruby/gems/1.9.1/gems/fluent-plugin-webhdfs-0.1.0/lib/fluent/plugin/out_webhdfs.rb:100:in `send_data'
2013-02-12 04:38:39 +0900: /usr/local/lib/ruby/gems/1.9.1/gems/fluent-plugin-webhdfs-0.1.0/lib/fluent/plugin/out_webhdfs.rb:109:in `write'
2013-02-12 04:38:39 +0900: /usr/local/lib/ruby/gems/1.9.1/gems/fluentd-0.10.30/lib/fluent/buffer.rb:279:in `write_chunk'
2013-02-12 04:38:39 +0900: /usr/local/lib/ruby/gems/1.9.1/gems/fluentd-0.10.30/lib/fluent/buffer.rb:263:in `pop'
2013-02-12 04:38:39 +0900: /usr/local/lib/ruby/gems/1.9.1/gems/fluentd-0.10.30/lib/fluent/output.rb:303:in `try_flush'
2013-02-12 04:38:39 +0900: /usr/local/lib/ruby/gems/1.9.1/gems/fluentd-0.10.30/lib/fluent/output.rb:120:in `run'
# fluentd.conf
<store>
type webhdfs
host ...
port ...
username ...
path ...
field_separator TAB # or 'SPACE', 'COMMA'
output_include_time false
output_include_tag false
output_data_type attr:log
# default val. is 60 on 'BufferedOutput'
flush_interval 1
# default val. is 8*1024*1024 on 'BasicBuffer'
# default val. is 256*1024*1024 on 'TimeSlicedOutput'
buffer_chunk_limit 128M
# default val. is 256 on 'BasicBuffer'
buffer_queue_limit 32
</store>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment