Mailing List Archive: RSyslog: users

Incorrect hostname from %hostname%

Index | Next | Previous | View Threaded

luke.marrott at gmail

Apr 13, 2012, 8:13 AM

Post #1 of 8 (362 views)
Permalink

Incorrect hostname from %hostname%

I am using a combination of rsyslogd and Splunk for syslog in order to
please different requirements within my organization and have ran into a
problem.

The hostnames of some devices is not being recorded correctly.

I've tried both of the following:
#$template default,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
#*.* ?Default

$template DynaFile,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
*.* -?DynaFile

And either way I end up with a directory and file named either "Apr" or
"2012" on a few devices.

If I do a tcpdump I can verify that the source information is coming into
the machine.

Then I tried to do a forward to forward the logs to localhost:10514 just so
I could test if Splunk would get the hostname from a forwarded message.

No luck. However if I turn rsyslogd off and turn Splunk to listen directly
to port 514 it works fine.

So somehow rsyslogd is not getting the hostname correctly.

I am running a bit older version:

rsyslogd 4.6.2, compiled with:
FEATURE_REGEXP: Yes
FEATURE_LARGEFILE: No
FEATURE_NETZIP (message compression): Yes
GSSAPI Kerberos 5 support: Yes
FEATURE_DEBUG (debug build, slow code): No
Atomic operations supported: Yes
Runtime Instrumentation (slow code): No

Thoughts?

Thanks!
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

rgerhards at hq

Apr 13, 2012, 9:57 AM

Post #2 of 8 (345 views)
Permalink

Re: Incorrect hostname from %hostname% [In reply to]

The short answer is that most probably devices send messages in malformed
format.

The long answer - including cures - is provided here:

http://www.rsyslog.com/doc/syslog_parsing.html

It's a long document and it points you to some other resources. All of them
are important if you really want to understand what's going on - and solve
it...

HTH
Rainer

> -----Original Message-----
> From: rsyslog-bounces [at] lists [mailto:rsyslog-
> bounces [at] lists] On Behalf Of Luke Marrott
> Sent: Friday, April 13, 2012 5:14 PM
> To: rsyslog [at] lists
> Subject: [rsyslog] Incorrect hostname from %hostname%
>
> I am using a combination of rsyslogd and Splunk for syslog in order to
> please different requirements within my organization and have ran into
> a
> problem.
>
> The hostnames of some devices is not being recorded correctly.
>
> I've tried both of the following:
> #$template default,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
> #*.* ?Default
>
> $template DynaFile,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
> *.* -?DynaFile
>
> And either way I end up with a directory and file named either "Apr" or
> "2012" on a few devices.
>
> If I do a tcpdump I can verify that the source information is coming
> into
> the machine.
>
> Then I tried to do a forward to forward the logs to localhost:10514
> just so
> I could test if Splunk would get the hostname from a forwarded message.
>
> No luck. However if I turn rsyslogd off and turn Splunk to listen
> directly
> to port 514 it works fine.
>
> So somehow rsyslogd is not getting the hostname correctly.
>
> I am running a bit older version:
>
> rsyslogd 4.6.2, compiled with:
> FEATURE_REGEXP: Yes
> FEATURE_LARGEFILE: No
> FEATURE_NETZIP (message compression): Yes
> GSSAPI Kerberos 5 support: Yes
> FEATURE_DEBUG (debug build, slow code): No
> Atomic operations supported: Yes
> Runtime Instrumentation (slow code): No
>
>
> Thoughts?
>
> Thanks!
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

david at lang

Apr 13, 2012, 2:09 PM

Post #3 of 8 (340 views)
Permalink

Re: Incorrect hostname from %hostname% [In reply to]

log some of the offending messages using the format RSYSLOG_Debug so that
we can see the raw message and how it's parsed.

As Rainer says, it's probably generating a message that doesn't quite
comply with the syslog specs (for example, the syslog spec doesn't include
a year in the timestamp)

Once we see what'd going on, we can look at fixing it.

Rainer, I believe that hostnames are required to have a letter in them
somewhere, so it may be worth tweaking the parser so that if the hostname
field has no letters in it and is a 4 digit number, treat it as the year
part of the timestamp.

David Lang

On Fri, 13 Apr 2012, Luke Marrott wrote:

> Date: Fri, 13 Apr 2012 09:13:48 -0600
> From: Luke Marrott <luke.marrott [at] gmail>
> Reply-To: rsyslog-users <rsyslog [at] lists>
> To: rsyslog [at] lists
> Subject: [rsyslog] Incorrect hostname from %hostname%
>
> I am using a combination of rsyslogd and Splunk for syslog in order to
> please different requirements within my organization and have ran into a
> problem.
>
> The hostnames of some devices is not being recorded correctly.
>
> I've tried both of the following:
> #$template default,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
> #*.* ?Default
>
> $template DynaFile,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
> *.* -?DynaFile
>
> And either way I end up with a directory and file named either "Apr" or
> "2012" on a few devices.
>
> If I do a tcpdump I can verify that the source information is coming into
> the machine.
>
> Then I tried to do a forward to forward the logs to localhost:10514 just so
> I could test if Splunk would get the hostname from a forwarded message.
>
> No luck. However if I turn rsyslogd off and turn Splunk to listen directly
> to port 514 it works fine.
>
> So somehow rsyslogd is not getting the hostname correctly.
>
> I am running a bit older version:
>
> rsyslogd 4.6.2, compiled with:
> FEATURE_REGEXP: Yes
> FEATURE_LARGEFILE: No
> FEATURE_NETZIP (message compression): Yes
> GSSAPI Kerberos 5 support: Yes
> FEATURE_DEBUG (debug build, slow code): No
> Atomic operations supported: Yes
> Runtime Instrumentation (slow code): No
>
>
> Thoughts?
>
> Thanks!
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

rgerhards at hq

Apr 13, 2012, 2:13 PM

Post #4 of 8 (337 views)
Permalink

Re: Incorrect hostname from %hostname% [In reply to]

Nit sure about the letter, but a 4 digit nbr in the range of 2000-2050 should work well. Shouldnt it?

Rainer
"david [at] lang" <david [at] lang> hat geschrieben:log some of the offending messages using the format RSYSLOG_Debug so that
we can see the raw message and how it's parsed.

As Rainer says, it's probably generating a message that doesn't quite
comply with the syslog specs (for example, the syslog spec doesn't include
a year in the timestamp)

Once we see what'd going on, we can look at fixing it.

Rainer, I believe that hostnames are required to have a letter in them
somewhere, so it may be worth tweaking the parser so that if the hostname
field has no letters in it and is a 4 digit number, treat it as the year
part of the timestamp.

David Lang

On Fri, 13 Apr 2012, Luke Marrott wrote:

> Date: Fri, 13 Apr 2012 09:13:48 -0600
> From: Luke Marrott <luke.marrott [at] gmail>
> Reply-To: rsyslog-users <rsyslog [at] lists>
> To: rsyslog [at] lists
> Subject: [rsyslog] Incorrect hostname from %hostname%
>
> I am using a combination of rsyslogd and Splunk for syslog in order to
> please different requirements within my organization and have ran into a
> problem.
>
> The hostnames of some devices is not being recorded correctly.
>
> I've tried both of the following:
> #$template default,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
> #*.* ?Default
>
> $template DynaFile,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
> *.* -?DynaFile
>
> And either way I end up with a directory and file named either "Apr" or
> "2012" on a few devices.
>
> If I do a tcpdump I can verify that the source information is coming into
> the machine.
>
> Then I tried to do a forward to forward the logs to localhost:10514 just so
> I could test if Splunk would get the hostname from a forwarded message.
>
> No luck. However if I turn rsyslogd off and turn Splunk to listen directly
> to port 514 it works fine.
>
> So somehow rsyslogd is not getting the hostname correctly.
>
> I am running a bit older version:
>
> rsyslogd 4.6.2, compiled with:
> FEATURE_REGEXP: Yes
> FEATURE_LARGEFILE: No
> FEATURE_NETZIP (message compression): Yes
> GSSAPI Kerberos 5 support: Yes
> FEATURE_DEBUG (debug build, slow code): No
> Atomic operations supported: Yes
> Runtime Instrumentation (slow code): No
>
>
> Thoughts?
>
> Thanks!
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

david at lang

Apr 13, 2012, 7:02 PM

Post #5 of 8 (341 views)
Permalink

Re: Incorrect hostname from %hostname% [In reply to]

that would work if the date is anywhere close to right

see http://support.microsoft.com/kb/244412 and
http://support.microsoft.com/kb/909264 for an explination of why all
numeric hostnames are not allowed (basically due to DNS limitations, which
apply to all systems)

Many applications (including browsers) will treat an all-numeric value as
an IP address (as a 32 bit integer, not as a dotted decimal address) and
as a result, trying to use an all-numeric hostname will cause 'strange
things' to happen to your software. So even if it is possible, it's not
going to work in many cases and so is a _really_ bad idea.

David Lang

On Fri, 13 Apr 2012, Rainer Gerhards wrote:

> Nit sure about the letter, but a 4 digit nbr in the range of 2000-2050
> should work well. Shouldnt it?
>
> Rainer
> "david [at] lang" <david [at] lang> hat geschrieben:log some of the offending messages using the format RSYSLOG_Debug so that
> we can see the raw message and how it's parsed.
>
> As Rainer says, it's probably generating a message that doesn't quite
> comply with the syslog specs (for example, the syslog spec doesn't include
> a year in the timestamp)
>
> Once we see what'd going on, we can look at fixing it.
>
> Rainer, I believe that hostnames are required to have a letter in them
> somewhere, so it may be worth tweaking the parser so that if the hostname
> field has no letters in it and is a 4 digit number, treat it as the year
> part of the timestamp.
>
> David Lang
>
> On Fri, 13 Apr 2012, Luke Marrott wrote:
>
>> Date: Fri, 13 Apr 2012 09:13:48 -0600
>> From: Luke Marrott <luke.marrott [at] gmail>
>> Reply-To: rsyslog-users <rsyslog [at] lists>
>> To: rsyslog [at] lists
>> Subject: [rsyslog] Incorrect hostname from %hostname%
>>
>> I am using a combination of rsyslogd and Splunk for syslog in order to
>> please different requirements within my organization and have ran into a
>> problem.
>>
>> The hostnames of some devices is not being recorded correctly.
>>
>> I've tried both of the following:
>> #$template default,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
>> #*.* ?Default
>>
>> $template DynaFile,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
>> *.* -?DynaFile
>>
>> And either way I end up with a directory and file named either "Apr" or
>> "2012" on a few devices.
>>
>> If I do a tcpdump I can verify that the source information is coming into
>> the machine.
>>
>> Then I tried to do a forward to forward the logs to localhost:10514 just so
>> I could test if Splunk would get the hostname from a forwarded message.
>>
>> No luck. However if I turn rsyslogd off and turn Splunk to listen directly
>> to port 514 it works fine.
>>
>> So somehow rsyslogd is not getting the hostname correctly.
>>
>> I am running a bit older version:
>>
>> rsyslogd 4.6.2, compiled with:
>> FEATURE_REGEXP: Yes
>> FEATURE_LARGEFILE: No
>> FEATURE_NETZIP (message compression): Yes
>> GSSAPI Kerberos 5 support: Yes
>> FEATURE_DEBUG (debug build, slow code): No
>> Atomic operations supported: Yes
>> Runtime Instrumentation (slow code): No
>>
>>
>> Thoughts?
>>
>> Thanks!
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

david at lang

Apr 13, 2012, 7:10 PM

Post #6 of 8 (338 views)
Permalink

Re: Incorrect hostname from %hostname% [In reply to]

doing a google search for 'all numeric hostname' I find lots of things
that break if you have an all numeric hostname, but a old RFC that relaxes
the prior restriction that the first character of a hostname could not be
a number.

so it looks like they are technically legal according to the RFCs, but
they will break all over the place.

given how rare an all-numeric hostname would be (and how much other stuff
is broken by using one), I think it's reasonable for rsyslog to have a
heuristic that doesn't allow them and assums that a 4-digit number in that
posisiton is the year portion of a timestamp. Especially since quite a
number of devices appear to be sending out logs with timestamps with the
year added to them.

David Lang

On Fri, 13 Apr 2012, Rainer Gerhards wrote:

> Date: Fri, 13 Apr 2012 23:13:15 +0200
> From: Rainer Gerhards <rgerhards [at] hq>
> Reply-To: rsyslog-users <rsyslog [at] lists>
> To: rsyslog [at] lists
> Subject: Re: [rsyslog] Incorrect hostname from %hostname%
>
> Nit sure about the letter, but a 4 digit nbr in the range of 2000-2050 should work well. Shouldnt it?
>
> Rainer
> "david [at] lang" <david [at] lang> hat geschrieben:log some of the offending messages using the format RSYSLOG_Debug so that
> we can see the raw message and how it's parsed.
>
> As Rainer says, it's probably generating a message that doesn't quite
> comply with the syslog specs (for example, the syslog spec doesn't include
> a year in the timestamp)
>
> Once we see what'd going on, we can look at fixing it.
>
> Rainer, I believe that hostnames are required to have a letter in them
> somewhere, so it may be worth tweaking the parser so that if the hostname
> field has no letters in it and is a 4 digit number, treat it as the year
> part of the timestamp.
>
> David Lang
>
> On Fri, 13 Apr 2012, Luke Marrott wrote:
>
>> Date: Fri, 13 Apr 2012 09:13:48 -0600
>> From: Luke Marrott <luke.marrott [at] gmail>
>> Reply-To: rsyslog-users <rsyslog [at] lists>
>> To: rsyslog [at] lists
>> Subject: [rsyslog] Incorrect hostname from %hostname%
>>
>> I am using a combination of rsyslogd and Splunk for syslog in order to
>> please different requirements within my organization and have ran into a
>> problem.
>>
>> The hostnames of some devices is not being recorded correctly.
>>
>> I've tried both of the following:
>> #$template default,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
>> #*.* ?Default
>>
>> $template DynaFile,"/var/log/syslog/%HOSTNAME%/%HOSTNAME%.log"
>> *.* -?DynaFile
>>
>> And either way I end up with a directory and file named either "Apr" or
>> "2012" on a few devices.
>>
>> If I do a tcpdump I can verify that the source information is coming into
>> the machine.
>>
>> Then I tried to do a forward to forward the logs to localhost:10514 just so
>> I could test if Splunk would get the hostname from a forwarded message.
>>
>> No luck. However if I turn rsyslogd off and turn Splunk to listen directly
>> to port 514 it works fine.
>>
>> So somehow rsyslogd is not getting the hostname correctly.
>>
>> I am running a bit older version:
>>
>> rsyslogd 4.6.2, compiled with:
>> FEATURE_REGEXP: Yes
>> FEATURE_LARGEFILE: No
>> FEATURE_NETZIP (message compression): Yes
>> GSSAPI Kerberos 5 support: Yes
>> FEATURE_DEBUG (debug build, slow code): No
>> Atomic operations supported: Yes
>> Runtime Instrumentation (slow code): No
>>
>>
>> Thoughts?
>>
>> Thanks!
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

rgerhards at hq

Apr 16, 2012, 1:44 AM

Post #7 of 8 (331 views)
Permalink

Re: Incorrect hostname from %hostname% [In reply to]

> -----Original Message-----
> From: rsyslog-bounces [at] lists [mailto:rsyslog-
> bounces [at] lists] On Behalf Of david [at] lang
> Sent: Saturday, April 14, 2012 4:10 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Incorrect hostname from %hostname%
>
> doing a google search for 'all numeric hostname' I find lots of things
> that break if you have an all numeric hostname, but a old RFC that
> relaxes
> the prior restriction that the first character of a hostname could not
> be
> a number.
>
> so it looks like they are technically legal according to the RFCs, but
> they will break all over the place.
>
> given how rare an all-numeric hostname would be (and how much other
> stuff
> is broken by using one), I think it's reasonable for rsyslog to have a
> heuristic that doesn't allow them and assums that a 4-digit number in
> that
> posisiton is the year portion of a timestamp. Especially since quite a
> number of devices appear to be sending out logs with timestamps with
> the
> year added to them.

Makes sense. I'd still say that if an all-numeric string of size <> 4
occupies that position, I'd interpret it as hostname?

Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

david at lang

Apr 16, 2012, 1:52 AM

Post #8 of 8 (328 views)
Permalink

Re: Incorrect hostname from %hostname% [In reply to]

On Mon, 16 Apr 2012, Rainer Gerhards wrote:

>> -----Original Message-----
>> From: rsyslog-bounces [at] lists [mailto:rsyslog-
>> bounces [at] lists] On Behalf Of david [at] lang
>> Sent: Saturday, April 14, 2012 4:10 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Incorrect hostname from %hostname%
>>
>> doing a google search for 'all numeric hostname' I find lots of things
>> that break if you have an all numeric hostname, but a old RFC that
>> relaxes
>> the prior restriction that the first character of a hostname could not
>> be
>> a number.
>>
>> so it looks like they are technically legal according to the RFCs, but
>> they will break all over the place.
>>
>> given how rare an all-numeric hostname would be (and how much other
>> stuff
>> is broken by using one), I think it's reasonable for rsyslog to have a
>> heuristic that doesn't allow them and assums that a 4-digit number in
>> that
>> posisiton is the year portion of a timestamp. Especially since quite a
>> number of devices appear to be sending out logs with timestamps with
>> the
>> year added to them.
>
> Makes sense. I'd still say that if an all-numeric string of size <> 4
> occupies that position, I'd interpret it as hostname?

It's either that or decide that there is no hostname provided and that
it's part of the message instead.

when I get around to it, I'm going to do another Cisco fixup parser. I've
got some Cisco devices that put a sequence number ahead of the message (I
need to double check exactly what it is that they send when I get into the
office in the morning)

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads