I'd like beaver to monitor /var/log with the following concerns:
- applications will create arbitrary trees of logs under
/var/log - I cannot know ahead of time what logs will be present under
/var/log
For instance, let's say I have an erlang application with lager logs to /var/log/erl_app/lags.log, webmachine logs to /var/log/erl_app/webmachine/YEAR_MONTH_DAY_access and a console log to /var/log/erl_app/console.log. The webmachine logs are not rotated, but are rather created at midnight each day. The console and lager logs are rotated periodically. As I add apps to this erlang application they may begin logging under /var/log/erl_app.
What I'd like to do is start beaver once, have it recursiely ship all the logs under /var/log which presently exist and are created in the future. It seems to me that only the path configuration option will suit my needs as the files option assumes I know the filenames I want to ship.
Here's my beaver.conf:
[beaver]
udp_host: 192.168.56.101
upd_port: 9999
path: /var/log/**/*
and the stacktrace when I run beaver, along with versions of the software in question:
blt@logclient1:~$ python --version
Python 2.6.6
blt@logclient1:~$ beaver --version
Info level is on
Beaver 14
blt@logclient1:~$ beaver -c beaver.conf -t udp
[2012-12-20 22:51:35,656] INFO Info level is on
[2012-12-20 22:51:35,656] INFO Processing file portion of config file beaver.conf
[2012-12-20 22:51:35,656] INFO Processing beaver portion of config file beaver.conf
[2012-12-20 22:51:35,677] INFO Logging using the udp transport
[2012-12-20 22:51:35,678] INFO Starting worker...
[2012-12-20 22:51:35,678] INFO Unhandled Exception:
[2012-12-20 22:51:35,678] INFO Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/beaver/worker.py", line 270, in run_worker
l = Worker(file_config, beaver_config, transport.callback, logger, ssh_tunnel=ssh_tunnel)
File "/usr/local/lib/python2.6/dist-packages/beaver/worker.py", line 59, in __init__
% self.folder
AssertionError: /var/log/**/* does not exists
The globstar works as expected from the command line, with shell globstar enabled.
blt@logclient1:~$ shopt -s globstar
blt@logclient1:~$ ls /var/log/**/* | wc -l # trust me, the count is good
64
blt@logclient1:~$ shopt -u globstar
blt@logclient1:~$ ls /var/log/**/* | wc -l
16