Created
April 9, 2020 19:12
-
-
Save richm/8ddaebd6b6a824bd0269c047de1411dd to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{% if rsyslog_flows | length > 0 %} | |
{% for flow in rsyslog_flows %} | |
{% for input_name in flow.inputs %} | |
{% if rsyslog_input_basics.0.name == input_name %} | |
{% if __basics_first_time == true %} | |
if $inputname == "imjournal" then { | |
{% endif %} | |
{% for flow_output in flow.outputs %} | |
call {{ flow_output }} | |
{% endfor %} | |
{% if __basics_first_time == true %} | |
stop | |
} | |
{% set __basics_first_time = false %} | |
{% endif %} | |
{% endif %} | |
{% endfor %} | |
{% endfor %} | |
{% else %} | |
if $inputname == "imjournal" then { | |
call files | |
stop | |
} | |
{% endif %} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
inputs:
type: a
type: b
type: a
outputs:
type: x
type: y
type: x
flows:
inputs: [A, B]
outputs: [X, Y]
inputs: [A, C]
outputs: [X, Z]
This is bad - does "call X" 3 times:
if $input == A then {
call X
call Y
call Z
}
if $input == B then {
call X
call Y
}
if $input == C then {
call X
call Z
}
what we want is a mapping of unique outputs to a unique list of inputs for each output
X: A, B, C
Y: A, B
Z: A, C
Each input has some sort of condition associated with it that uniquely identifies records from that input.
For imfile, we can use the "syslogtag" message property https://www.rsyslog.com/doc/v8-stable/configuration/properties.html#message-properties which is set from the imfile "Tag" property https://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#tag
the rsyslog config condition would be
$syslogtag == "tag value from imfile input config"
e.g.For imjournal, use the "inputname" message property which has the value "imjournal"
For imtcp, use the "inputname" message property which gets its value from the "Name" property in the imtcp input configuration: https://www.rsyslog.com/doc/v8-stable/configuration/modules/imtcp.html#name - note that Name is also used for the stats
For imrelp, use the "inputname" message property which gets its value from the "Name" property in the imrelp input configuration: https://www.rsyslog.com/doc/v8-stable/configuration/modules/imrelp.html#name - NOTE: This isn't specified in the documentation, but that's what the source code does. Also note that the "Name" is used to construct the name used in stats.
Other inputs will have other ways to denote which input this is from. One notable exception may be imkafka - I cannot figure out how to distinguish between two different kafka inputs - it sets syslogtag based on some "key" field in the message, but I don't know where that is set. So the condition may be compound e.g.
($inputname == "imkafka" and something else)
. Similarly with imuxsock - there's $inputname == "imuxsock" but may have to override "Hostname" https://www.rsyslog.com/doc/v8-stable/configuration/modules/imuxsock.html#hostname and use($inputname == "imuxsock" and $hostname == "sock0")
.The point is that the input must have a condition which can be rendered into an rsyslog
if
statement condition and concatenated with conditions from other inputs:assuming "outputA" and "outputB" have the exact same inputs.
for name,inputs,outputs in flows:
for each outputname in outputs:
uniqueoutputs[outputname] = uniqueorderedset(inputs + uniqueoutputs.get(outputname, [])
for each outputname,inputnamelist in uniqueoutputs.items():
output = logging_outputs[outputname]
maybe handle errors here if user specified an output in a flow that is not defined
in logging_outputs
{{ "if " }}
for inputname in inputnamelist:
input = logging_inputs[inputname]
if not loop.first:
{{ " or " }}
{{ input.rsyslog_condition }}
# https://jinja.palletsprojects.com/en/2.11.x/templates/#for
{{ " then { " }}
{{ "call " + output.rsyslog_output }}
{{ "}" }}
{% for each input in inputs %}
{% if there is at least one output for this input %}
if $input in outputs["X"] then {
call X
}
if $input in outputs["Y"] then {
call Y
}
if $input in outputs["Z"] then {
call Z
}