Skip to content

Instantly share code, notes, and snippets.

@larsgeorge
Created August 9, 2011 12:58
Show Gist options
  • Save larsgeorge/1133967 to your computer and use it in GitHub Desktop.
Save larsgeorge/1133967 to your computer and use it in GitHub Desktop.
Use syslog (on MacOS) with Hadoop's userlog/syslog
#...
#Default values
hadoop.tasklog.taskid=null
hadoop.tasklog.iscleanup=false
hadoop.tasklog.noKeepSplits=4
hadoop.tasklog.totalLogFileSize=100
hadoop.tasklog.purgeLogSplits=true
hadoop.tasklog.logsRetainHours=12
log4j.appender.TLA=org.apache.log4j.net.SyslogAppender
log4j.appender.TLA.syslogHost=localhost
log4j.appender.TLA.layout=org.apache.log4j.PatternLayout
log4j.appender.TLA.layout.conversionPattern=%d{ISO8601} %-5p [%t] %c{2} %x - %m%n
#log4j.appender.TLA.Facility=LOCAL1
#log4j.appender.TLA=org.apache.hadoop.mapred.TaskLogAppender
log4j.appender.TLA.taskId=${hadoop.tasklog.taskid}
log4j.appender.TLA.isCleanup=${hadoop.tasklog.iscleanup}
log4j.appender.TLA.totalLogFileSize=${hadoop.tasklog.totalLogFileSize}
#log4j.appender.TLA.layout=org.apache.log4j.PatternLayout
#log4j.appender.TLA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n
#...
# For MacOS:
# - http://d.hatena.ne.jp/MiCHiLU/20110726/1311663353
# - http://superuser.com/questions/131578/how-do-you-enable-syslogd-to-accept-incoming-connections-on-snow-leopard-from-rem
# Note:
# - Makes UI fail to show any logs!
#
# HTTP ERROR: 410
#
# Failed to retrieve syslog log for task: attempt_201108091455_0001_m_000000_0
# RequestURI=/tasklog
#
# Powered by Jetty://
#
# Code expects to have a syslog file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment