Mailing List Archive: RSyslog: users

Best way to configure rsyslog as an aggregation center

Index | Next | Previous | View Threaded

scott at perturb

Apr 4, 2012, 3:24 PM

Post #1 of 11 (541 views)
Permalink

Best way to configure rsyslog as an aggregation center

I work for an ISP and we have lots of equipment logging to a central
rsyslog (4.6) server. We're upgrading to rsyslog 5.8 and I want to
re-evaluate my configuration. Right now my config is full of things like:

:FROMHOST, isequal, "router1" -?routers
:FROMHOST, isequal, "10.45.0.7" -?dialup

Etc... about 50 times.

Ideally I'd like to more granular and say 10.45.0.7.warn goes to log #1
file, and 10.45.0.7.critical goes to log #2. Is there a better way to
map different IPs to various log files?

Lastly, if I have all that setup how do I get the local box that's
running rsyslog to log to the various logs? When postfix or the kernel
log a message it doesn't do it via IP so what should my rules match for
that?
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

radu0gheorghe at gmail

Apr 4, 2012, 10:13 PM

Post #2 of 11 (521 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

Hi Scott,

Maybe dynamic file names would help:
http://wiki.rsyslog.com/index.php/Log_Router_syslog_with_Dynamic_File_Names

2012/4/5 Scott Baker <scott [at] perturb>:
> I work for an ISP and we have lots of equipment logging to a central
> rsyslog (4.6) server. We're upgrading to rsyslog 5.8 and I want to
> re-evaluate my configuration. Right now my config is full of things like:
>
> :FROMHOST, isequal, "router1" -?routers
> :FROMHOST, isequal, "10.45.0.7" -?dialup
>
> Etc... about 50 times.
>
> Ideally I'd like to more granular and say 10.45.0.7.warn goes to log #1
> file, and 10.45.0.7.critical goes to log #2. Is there a better way to
> map different IPs to various log files?
>
> Lastly, if I have all that setup how do I get the local box that's
> running rsyslog to log to the various logs? When postfix or the kernel
> log a message it doesn't do it via IP so what should my rules match for
> that?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

david at lang

Apr 4, 2012, 11:09 PM

Post #3 of 11 (523 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

You won't be able to use the dynamic names for to deal with the hostname
part since it's not a 1:1 mapping of host to file (I assume you have
multiple things going to the routers file, and multiple going to dialup,
etc)

create your file template to include the severity in the filename and you
will get the split that you are looking for.

You should also think about upgrading, rsyslgo 4.6 is old enough that it's
no longer really supported (except via a support contract or by RedHat)
and there are a LOT of improvements in later versions

David Lang

On Thu, 5 Apr 2012, Radu Gheorghe wrote:

> Hi Scott,
>
> Maybe dynamic file names would help:
> http://wiki.rsyslog.com/index.php/Log_Router_syslog_with_Dynamic_File_Names
>
> 2012/4/5 Scott Baker <scott [at] perturb>:
>> I work for an ISP and we have lots of equipment logging to a central
>> rsyslog (4.6) server. We're upgrading to rsyslog 5.8 and I want to
>> re-evaluate my configuration. Right now my config is full of things like:
>>
>> :FROMHOST, isequal, "router1" -?routers
>> :FROMHOST, isequal, "10.45.0.7" -?dialup
>>
>> Etc... about 50 times.
>>
>> Ideally I'd like to more granular and say 10.45.0.7.warn goes to log #1
>> file, and 10.45.0.7.critical goes to log #2. Is there a better way to
>> map different IPs to various log files?
>>
>> Lastly, if I have all that setup how do I get the local box that's
>> running rsyslog to log to the various logs? When postfix or the kernel
>> log a message it doesn't do it via IP so what should my rules match for
>> that?
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
>

rgerhards at hq

Apr 5, 2012, 1:14 AM

Post #4 of 11 (527 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

> -----Original Message-----
> From: rsyslog-bounces [at] lists [mailto:rsyslog-
> bounces [at] lists] On Behalf Of Radu Gheorghe
> Sent: Thursday, April 05, 2012 7:13 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Best way to configure rsyslog as an aggregation
> center
>
> Hi Scott,
>
> Maybe dynamic file names would help:
> http://wiki.rsyslog.com/index.php/Log_Router_syslog_with_Dynamic_File_N
> ames

Yup, you can use dynafiles with that. Have a look at the property replacer
doc:

http://www.rsyslog.com/doc/property_replacer.html

to see which properties are available. There are some that contain, for
example, the severity which you can use to build file names (I guess this is
along the lines of "more granualar" that you mentioned).

HTH
Rainer
>
> 2012/4/5 Scott Baker <scott [at] perturb>:
> > I work for an ISP and we have lots of equipment logging to a central
> > rsyslog (4.6) server. We're upgrading to rsyslog 5.8 and I want to
> > re-evaluate my configuration. Right now my config is full of things
> like:
> >
> > :FROMHOST, isequal, "router1" -?routers
> > :FROMHOST, isequal, "10.45.0.7" -?dialup
> >
> > Etc... about 50 times.
> >
> > Ideally I'd like to more granular and say 10.45.0.7.warn goes to log
> #1
> > file, and 10.45.0.7.critical goes to log #2. Is there a better way to
> > map different IPs to various log files?
> >
> > Lastly, if I have all that setup how do I get the local box that's
> > running rsyslog to log to the various logs? When postfix or the
> kernel
> > log a message it doesn't do it via IP so what should my rules match
> for
> > that?
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

scott at perturb

Apr 5, 2012, 8:24 AM

Post #5 of 11 (522 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

On 04/05/2012 01:14 AM, Rainer Gerhards wrote:
> Yup, you can use dynafiles with that. Have a look at the property replacer
> doc:
>
> http://www.rsyslog.com/doc/property_replacer.html
>
> to see which properties are available. There are some that contain, for
> example, the severity which you can use to build file names (I guess this is
> along the lines of "more granualar" that you mentioned).

A real world example:

I have a Linux server that logs two services DHCP on LOCAL1 and RADIUS
on LOCAL7. How do I separate out those into a DHCP.log and a radius.log?

At the same time I have other boxes that just log one server so a simple
match on IP would be plenty sufficient.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

scott at perturb

Apr 5, 2012, 9:03 AM

Post #6 of 11 (522 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

On 04/05/2012 08:24 AM, Scott Baker wrote:
> A real world example:
>
> I have a Linux server that logs two services DHCP on LOCAL1 and RADIUS
> on LOCAL7. How do I separate out those into a DHCP.log and a radius.log?
>
> At the same time I have other boxes that just log one server so a simple
> match on IP would be plenty sufficient.

I guess a better question is how do I filter on two conditions?

:fromhost, isequal, "server.domain.com" and :syslogfacility, isequal, 5

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

trefalgar at realitybytes

Apr 5, 2012, 9:10 AM

Post #7 of 11 (520 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

Quoting Scott Baker <scott [at] perturb>:
> On 04/05/2012 08:24 AM, Scott Baker wrote:
>> A real world example:
>>
>> I have a Linux server that logs two services DHCP on LOCAL1 and RADIUS
>> on LOCAL7. How do I separate out those into a DHCP.log and a radius.log?
>>
>> At the same time I have other boxes that just log one server so a simple
>> match on IP would be plenty sufficient.
>
> I guess a better question is how do I filter on two conditions?
>
> :fromhost, isequal, "server.domain.com" and :syslogfacility, isequal, 5

http://www.rsyslog.com/doc/rsyslog_conf_filter.html

Or specifically ...

if $syslogfacility-text == 'local0' and $msg startswith 'DEVNAME' and
($msg contains 'error1' or $msg contains 'error0') then /var/log/somelog

Jacob

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

scott at perturb

Apr 5, 2012, 9:13 AM

Post #8 of 11 (519 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

On 04/05/2012 09:10 AM, Jacob Steinberger wrote:
>
> http://www.rsyslog.com/doc/rsyslog_conf_filter.html
>
> Or specifically ...
>
> if $syslogfacility-text == 'local0' and $msg startswith 'DEVNAME' and
> ($msg contains 'error1' or $msg contains 'error0') then /var/log/somelog

Perfect thanks!
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

david at lang

Apr 5, 2012, 10:34 AM

Post #9 of 11 (522 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

On Thu, 5 Apr 2012, Scott Baker wrote:

> On 04/05/2012 08:24 AM, Scott Baker wrote:
>> A real world example:
>>
>> I have a Linux server that logs two services DHCP on LOCAL1 and RADIUS
>> on LOCAL7. How do I separate out those into a DHCP.log and a radius.log?
>>
>> At the same time I have other boxes that just log one server so a simple
>> match on IP would be plenty sufficient.
>
> I guess a better question is how do I filter on two conditions?
>
> :fromhost, isequal, "server.domain.com" and :syslogfacility, isequal, 5

There are two possible ways to do this.

1. using rulesets, you can have the first condition take you to a
different ruleset that contains the second condition (very ugly and
probably not the thing you want to do unless you have one common condition
that leads to a bunch of secondary conditions)

2. use the if() filter format. It's slower, but it allows for more complex
expressions.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

jrhett at netconsonance

Apr 5, 2012, 10:37 AM

Post #10 of 11 (522 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

On Apr 5, 2012, at 10:34 AM, david [at] lang wrote:
> 2. use the if() filter format. It's slower, but it allows for more complex expressions.

Really? That sucks. The if() format is a lot easier to read and maintain. Why is it slower? Aren't the two things just different syntax parsers?

--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source and other randomness

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

rgerhards at hq

Apr 5, 2012, 10:39 AM

Post #11 of 11 (525 views)
Permalink

Re: Best way to configure rsyslog as an aggregation center [In reply to]

> -----Original Message-----
> From: rsyslog-bounces [at] lists [mailto:rsyslog-
> bounces [at] lists] On Behalf Of Jo Rhett
> Sent: Thursday, April 05, 2012 7:38 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Best way to configure rsyslog as an aggregation
> center
>
> On Apr 5, 2012, at 10:34 AM, david [at] lang wrote:
> > 2. use the if() filter format. It's slower, but it allows for more
> complex expressions.
>
>
> Really? That sucks. The if() format is a lot easier to read and
> maintain. Why is it slower? Aren't the two things just different
> syntax parsers?

Nope, the more complex logic takes its toll. For example, the simply PRI
based filter is a simple table lookup. Can't do that with a "a and b" thing.
Note that in v6.3+ the if-performance is far better, but not as good as for
the simpler filters.

Rainer
>
> --
> Jo Rhett
> Net Consonance : consonant endings by net philanthropy, open source and
> other randomness
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads