rgerhards at hq
Apr 9, 2012, 11:58 PM
Post #12 of 13
(434 views)
Permalink
|
> -----Original Message-----
> From: rsyslog-bounces [at] lists [mailto:rsyslog-
> bounces [at] lists] On Behalf Of Jo Rhett
> Sent: Tuesday, April 10, 2012 1:47 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] forwarding server template?
>
>
> On Apr 7, 2012, at 5:24 AM, Rainer Gerhards wrote:
> >> I just wanted to add a new answer to this. If I am reading the
> queue
> >> configuration and some online examples right, then I have to put 4-5
> >> lines of queue configuration if I wanted a LinkedList queue for
> >> forwarding to a remote system. This really, really makes the case
> for
> >> defining a forwarding queue in a single place, as this gets very
> silly
> >>
> >> $WorkDirectory /var/spool/rsyslog/fwd-host1
> >> $ActionQueueFileName fwdHost1
> >> $ActionQueueMaxDiskSpace 1g
> >> $ActionQueueSaveOnShutdown on
> >> $ActionQueueType LinkedList
> >> $ActionResumeRetryCount -1
> >> if this and that then @@remoteHost1:514
> >>
> >> $WorkDirectory /var/spool/rsyslog/fwd-host2
> >> $ActionQueueFileName fwdHost2
> >> $ActionQueueMaxDiskSpace 1g
> >> $ActionQueueSaveOnShutdown on
> >> $ActionQueueType LinkedList
> >> $ActionResumeRetryCount -1
> >> if one and another then @@remoteHost2:514
> >>
> >> $WorkDirectory /var/spool/rsyslog/fwd-host1
> >> $ActionQueueFileName fwdHost1
> >> $ActionQueueMaxDiskSpace 1g
> >> $ActionQueueSaveOnShutdown on
> >> $ActionQueueType LinkedList
> >> $ActionResumeRetryCount -1
> >> if more and yetmore then @@remoteHost1:514
> >>
> >> Please please tell me I am misunderstanding and you can easily
> define a
> >> single queue on a per-host basis, no matter how many rules forward
> >> messages to that host.
>
> On Apr 7, 2012, at 5:24 AM, Rainer Gerhards wrote:
> > You are understanding it right ;) The plan is to include named
> actions in v6,
> > which will solve this issue. Anyhow, patches are always welcome.
>
>
> This is perhaps the one thing I understand the least, and I'm looking
> forward to your changes. For now, is the best practice for this to put
> each queue inside a separate file, put the ActionQueue directives at
> the top and put every rule that forwards to that host within the same
> file? Like
Let me try to explain. Please bear with me, as I am so involved with the code
that I may think of something as being "natural" where it isn't.
Let's assume the queue params for a while so that we don't clutter things.
First of all
*.* @@host.example.net #action 1
*.* @@host.example.net #action 2
Means there are TWO forwarding actions, that is if both rules match
(obviously they do in this example), TWO connections to host.example.net are
created. Each of this connections (actually actions) uses their own queue
settings.
If you really want to have a single connection, things get ugly with the
current engine. To do that, you either use a single filter and a single
action, what often is extremely inconvenient. As a work-around, you can use
ruleset inclusion via omrulset[1]. In essence, then a ruleset is defined
which contains the forwarding action like
$ruleset fwd-host
*.* @@host.example.net #action 1
And in the other ruleset, a reference to it is used:
$ruleset whatever
$ActionOmrulesetRulesetName fwd-host
*.* :omruleset:
$ActionOmrulesetRulesetName fwd-host
*.* :omruleset:
This will end up with a single connection and a single set of queue params
(to be set in ruleset fwd-host).
Why did I do it that way? Tried to make it as easy as possible? ;-) Of course
not. It is just that this method could be implemented within the limits of
the pre-v6 engine. Other methods would require a great deal of engine change,
including the config part. And while the config was/is really bad, there was
an even larger demand for new features, so that received priority over the
config. Finally, within v6, I have found some time to gradually improve on
the config. The action part has improved quite a lot in v6, but named actions
are still missing. With them, you do something along these lines:
*.* action(type="omfwd" target="host..." id="fwd-host" queue.param=...))
*.* action(use="fwd-host")
Note how the action is defined the action that contains "id" and re-used by
specifying "use=" in the second filter. This will result in a single action.
If you don't insist on a single queue/connection (and performance-wise this
is a smart idea, if it does not violate any constraints), the usual approach
is to have an include file that contains the queue parameters, like this:
queue-params.inc:
>
> /etc/rsyslog.conf:
> # Include all config files in /etc/rsyslog.d/
> $IncludeConfig /etc/rsyslog.d/*.conf
>
> /etc/rsyslog.d/fwd-host1.conf:
> $WorkDirectory /var/spool/rsyslog/fwd-host1
> $ActionQueueFileName fwdHost1
> $ActionQueueMaxDiskSpace 1g
> $ActionQueueSaveOnShutdown on
> $ActionQueueType LinkedList
> $ActionResumeRetryCount -1
>
> if this and that then @@remoteHost1:514
>
> if more and yetmore then @@remoteHost1:514
>
> if trimore then @@remoteHost1:514
>
> /etc/rsyslog.d/fwd-host2.conf:
> $WorkDirectory /var/spool/rsyslog/fwd-host2
> $ActionQueueFileName fwdHost2
> $ActionQueueMaxDiskSpace 1g
> $ActionQueueSaveOnShutdown on
> $ActionQueueType LinkedList
> $ActionResumeRetryCount -1
>
> if one and another then @@remoteHost2:514
>
> Will that work making it easy to maintain a single queue for each
> remote host?
If you don't insist on a single queue/connection (and performance-wise this
is a smart idea, if it does not violate any constraints), the usual approach
is to have an include file that contains the queue parameters, like this:
/etc/rsyslog.d/queue-params.inc:
$ActionQueueMaxDiskSpace 1g
$ActionQueueSaveOnShutdown on
$ActionQueueType LinkedList
$ActionResumeRetryCount -1
Note that the queue file name is NOT common!
/etc/rsyslog.conf:
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
$WorkDirectory /var/spool/rsyslog/
$ActionQueueFileName fwd1
$includeConfig /etc/rsyslog.d/queue-params.inc
*.* @@host.example.net
$ActionQueueFileName fwd2
$includeConfig /etc/rsyslog.d/queue-params.inc
*.* @@host.example.net
And you can also use the same include if you want to use the same params for
some other host:
$ActionQueueFileName fwd3
$includeConfig /etc/rsyslog.d/queue-params.inc
*.* @@otherhost.example.net
Again, the prime reason for things being as they are is that this way it fits
into the old-style config constraints. In v6, you specify the queue params
inside the scoped action statement, which makes things much more readable.
I hope this clarifies.
Rainer
[1] http://www.rsyslog.com/doc/omruleset.html
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
|
