Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: RSyslog: users

Re: rawmessage forwarding doesn't appear to work -- very detailed

 

 

RSyslog users RSS feed   Index | Next | Previous | View Threaded


jrhett at netconsonance

Apr 20, 2012, 12:00 PM

Post #1 of 13 (731 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed

On Apr 20, 2012, at 9:58 AM, Rainer Gerhards wrote:
> I just tried with that latest 5.8.10 and 5.4.0 (as a random older version). I
> could not reproduce the problem in both cases. I would appreciate if you
> could either 5.8.10 or let me know the version number of a version where you
> had the problem. I'd like to see if I can reproduce - or if it is
> environment-induced.


I am using 5.8.10.

We are trying to forward syslog to Zenoss and that's where it is failing. So I did some testing and gathered some pcaps, and I'm seeing something odd. The forwarded messages appear to have a lot of nulls in the same place where a normal syslog message starts.

Interesting, testing to port 514 with a normal syslogd works fine. It's just when testing to port 1514 that I'm seeing this behavior. Configuration:

# Load the module which spoofs UDP packets when forwarding
$ModLoad omudpspoof
$template Untouched,"%rawmsg%"
$template RawMessage,"%msg:2:2048%\n"
$ActionOMUDPSpoofTargetHost loghost
$ActionOMUDPSpoofTargetPort 1514

# This doesn't work in any scenario
local7.* :omudpspoof:;RSYSLOG_TraditionalFileFormat

# The following two work for a normal syslogd on port 514
local7.* :omudpspoof:;Untouched
local7.* :omudpspoof:

# The following works for forwarding to Zenoss -- although Zenoss doesn't parse it correctly since it isn't in the right format
local7.* :omudpspoof:;RawMessage

HOWEVER, a normal syslog message generated locally by that host works fine (not through the UDP spoof) and Zenoss parses it correctly too. So there is some difference in the message formats.

So I went looking at pcap of the files and found that in a normal syslog message not forwarded through udpspoof, the message consistently starts at byte 43. Before that byte are mostly binary and very few nulls.

The udpspoofed messages don't begin until byte 63. With %msg:2:2048% the message part of the syslog event starts immediately. With %rawmsg% the message beings with the string "<190>" before the date stamp, which appears to break the parsing of the message on the far side.

Here's some examples -- msg:2:2048 created with logger -p local7.info "123 This is another test message" and forwarded via udpspoof

0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 4a 00 x+.)...& .3.:..J.
0010 00 52 00 f2 00 00 40 11 b6 bd 0a d2 82 76 0a d2 .R....@. .....v..
0020 1e ca 00 01 01 01 01 01 01 01 01 01 01 00 01 01 ........ ........
0030 00 01 00 01 01 00 07 7d 05 ea 00 2a 63 a3 31 32 .......} ...*c.12
0040 33 20 54 68 69 73 20 69 73 20 61 6e 6f 74 68 65 3 This i s anothe
0050 72 20 74 65 73 74 20 6d 65 73 73 61 67 65 2e 0a r test m essage..

rawmsg created with logger -p local7.info 123 "This is another test message" and forwarded via udpspoof

0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 4a 00 x+.)...& .3.:..J.
0010 00 79 00 f2 00 00 40 11 b6 9a 0a d2 82 76 0a d2 .y....@. .....v..
0020 1e ca 01 00 01 01 00 00 00 01 01 01 01 00 00 00 ........ ........
0030 01 01 01 00 01 00 02 7d 05 ea 00 51 33 98 3c 31 .......} ...Q3.<1
0040 39 30 3e 41 70 72 20 32 30 20 31 31 3a 30 30 3a 90>Apr 2 0 11:00:
0050 30 36 20 73 32 32 2d 77 77 77 30 38 20 6a 6f 72 06 s22-w ww08 jor

Here's a normal syslog message from the same host sent directly, not forwarded by udpspoof: you might recognize the message ;-)

0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 45 00 x+.)...& .3.:..E.
0010 00 6d 00 00 40 00 40 11 e7 53 0a d2 1e bf 0a d2 .m..@.@. .S......
0020 1e ca cb aa 05 ea 00 59 32 64 41 70 72 20 32 30 .......Y 2dApr 20
0030 20 31 37 3a 35 39 3a 33 37 20 73 32 32 2d 66 32 17:59:3 7 s22-f2
0040 30 31 20 6b 65 72 6e 65 6c 3a 20 69 6d 6b 6c 6f 01 kerne l: imklo
0050 67 20 35 2e 38 2e 31 30 2c 20 6c 6f 67 20 73 6f g 5.8.10 , log so

--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source and other randomness

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


david at lang

Apr 20, 2012, 1:25 PM

Post #2 of 13 (710 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Fri, 20 Apr 2012, Jo Rhett wrote:

> On Apr 20, 2012, at 9:58 AM, Rainer Gerhards wrote:
>> I just tried with that latest 5.8.10 and 5.4.0 (as a random older version). I
>> could not reproduce the problem in both cases. I would appreciate if you
>> could either 5.8.10 or let me know the version number of a version where you
>> had the problem. I'd like to see if I can reproduce - or if it is
>> environment-induced.
>
>
> I am using 5.8.10.
>
> We are trying to forward syslog to Zenoss and that's where it is
> failing. So I did some testing and gathered some pcaps, and I'm seeing
> something odd. The forwarded messages appear to have a lot of nulls in
> the same place where a normal syslog message starts.
>
> Interesting, testing to port 514 with a normal syslogd works fine. It's
> just when testing to port 1514 that I'm seeing this behavior.
> Configuration:
>
> # Load the module which spoofs UDP packets when forwarding
> $ModLoad omudpspoof
> $template Untouched,"%rawmsg%"
> $template RawMessage,"%msg:2:2048%\n"
> $ActionOMUDPSpoofTargetHost loghost
> $ActionOMUDPSpoofTargetPort 1514
>
> # This doesn't work in any scenario
> local7.* :omudpspoof:;RSYSLOG_TraditionalFileFormat
>
> # The following two work for a normal syslogd on port 514
> local7.* :omudpspoof:;Untouched
> local7.* :omudpspoof:
>
> # The following works for forwarding to Zenoss -- although Zenoss doesn't parse it correctly since it isn't in the right format
> local7.* :omudpspoof:;RawMessage
>
> HOWEVER, a normal syslog message generated locally by that host works
> fine (not through the UDP spoof) and Zenoss parses it correctly too. So
> there is some difference in the message formats.
>
> So I went looking at pcap of the files and found that in a normal syslog
> message not forwarded through udpspoof, the message consistently starts
> at byte 43. Before that byte are mostly binary and very few nulls.
>
> The udpspoofed messages don't begin until byte 63. With %msg:2:2048%
> the message part of the syslog event starts immediately. With %rawmsg%
> the message beings with the string "<190>" before the date stamp, which
> appears to break the parsing of the message on the far side.

the string <190> is the encoded priority and severity of the message (in
this case local7.user). A properly formatted syslog message will have this
(try sending a message with the format RSYSLOG_Traditional_Forward_Format
and you should see a similar thing before the timestamp) If this is
breaking the message parsing, the receiving system is broken


> Here's some examples -- msg:2:2048 created with logger -p local7.info "123 This is another test message" and forwarded via udpspoof
>
> 0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 4a 00 x+.)...& .3.:..J.
> 0010 00 52 00 f2 00 00 40 11 b6 bd 0a d2 82 76 0a d2 .R....@. .....v..
> 0020 1e ca 00 01 01 01 01 01 01 01 01 01 01 00 01 01 ........ ........
> 0030 00 01 00 01 01 00 07 7d 05 ea 00 2a 63 a3 31 32 .......} ...*c.12
> 0040 33 20 54 68 69 73 20 69 73 20 61 6e 6f 74 68 65 3 This i s anothe
> 0050 72 20 74 65 73 74 20 6d 65 73 73 61 67 65 2e 0a r test m essage..
>
> rawmsg created with logger -p local7.info 123 "This is another test message" and forwarded via udpspoof
>
> 0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 4a 00 x+.)...& .3.:..J.
> 0010 00 79 00 f2 00 00 40 11 b6 9a 0a d2 82 76 0a d2 .y....@. .....v..
> 0020 1e ca 01 00 01 01 00 00 00 01 01 01 01 00 00 00 ........ ........
> 0030 01 01 01 00 01 00 02 7d 05 ea 00 51 33 98 3c 31 .......} ...Q3.<1
> 0040 39 30 3e 41 70 72 20 32 30 20 31 31 3a 30 30 3a 90>Apr 2 0 11:00:
> 0050 30 36 20 73 32 32 2d 77 77 77 30 38 20 6a 6f 72 06 s22-w ww08 jor
>
> Here's a normal syslog message from the same host sent directly, not forwarded by udpspoof: you might recognize the message ;-)
>
> 0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 45 00 x+.)...& .3.:..E.
> 0010 00 6d 00 00 40 00 40 11 e7 53 0a d2 1e bf 0a d2 .m..@.@. .S......
> 0020 1e ca cb aa 05 ea 00 59 32 64 41 70 72 20 32 30 .......Y 2dApr 20
> 0030 20 31 37 3a 35 39 3a 33 37 20 73 32 32 2d 66 32 17:59:3 7 s22-f2
> 0040 30 31 20 6b 65 72 6e 65 6c 3a 20 69 6d 6b 6c 6f 01 kerne l: imklo
> 0050 67 20 35 2e 38 2e 31 30 2c 20 6c 6f 67 20 73 6f g 5.8.10 , log so

looking at these dumps, I don't think the problem is the <190>, I think
the problem is the three characters before that (Q3. in the text
represnetation), those start at the same point that the timestamp starts
in the last example.

note that the last example is missing the priority/severity information.
This is unfortunantly not an uncommon bug.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


jrhett at netconsonance

Apr 20, 2012, 1:42 PM

Post #3 of 13 (702 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

> the string <190> is the encoded priority and severity of the message (in this case local7.user). A properly formatted syslog message will have this (try sending a message with the format RSYSLOG_Traditional_Forward_Format and you should see a similar thing before the timestamp) If this is breaking the message parsing, the receiving system is broken

I think you mean local7.info? ;-) Thanks for the explanation. Anyway, if I just send a message directly to it the system works fine so this isn't apparently a problem. It's just having issues with the udpspoofed messages.

>> Here's some examples -- msg:2:2048 created with logger -p local7.info "123 This is another test message" and forwarded via udpspoof
>>
>> 0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 4a 00 x+.)...& .3.:..J.
>> 0010 00 52 00 f2 00 00 40 11 b6 bd 0a d2 82 76 0a d2 .R....@. .....v..
>> 0020 1e ca 00 01 01 01 01 01 01 01 01 01 01 00 01 01 ........ ........
>> 0030 00 01 00 01 01 00 07 7d 05 ea 00 2a 63 a3 31 32 .......} ...*c.12
>> 0040 33 20 54 68 69 73 20 69 73 20 61 6e 6f 74 68 65 3 This i s anothe
>> 0050 72 20 74 65 73 74 20 6d 65 73 73 61 67 65 2e 0a r test m essage..
>>
>> rawmsg created with logger -p local7.info 123 "This is another test message" and forwarded via udpspoof
>>
>> 0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 4a 00 x+.)...& .3.:..J.
>> 0010 00 79 00 f2 00 00 40 11 b6 9a 0a d2 82 76 0a d2 .y....@. .....v..
>> 0020 1e ca 01 00 01 01 00 00 00 01 01 01 01 00 00 00 ........ ........
>> 0030 01 01 01 00 01 00 02 7d 05 ea 00 51 33 98 3c 31 .......} ...Q3.<1
>> 0040 39 30 3e 41 70 72 20 32 30 20 31 31 3a 30 30 3a 90>Apr 2 0 11:00:
>> 0050 30 36 20 73 32 32 2d 77 77 77 30 38 20 6a 6f 72 06 s22-w ww08 jor
>>
>> Here's a normal syslog message from the same host sent directly, not forwarded by udpspoof: you might recognize the message ;-)
>>
>> 0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 45 00 x+.)...& .3.:..E.
>> 0010 00 6d 00 00 40 00 40 11 e7 53 0a d2 1e bf 0a d2 .m..@.@. .S......
>> 0020 1e ca cb aa 05 ea 00 59 32 64 41 70 72 20 32 30 .......Y 2dApr 20
>> 0030 20 31 37 3a 35 39 3a 33 37 20 73 32 32 2d 66 32 17:59:3 7 s22-f2
>> 0040 30 31 20 6b 65 72 6e 65 6c 3a 20 69 6d 6b 6c 6f 01 kerne l: imklo
>> 0050 67 20 35 2e 38 2e 31 30 2c 20 6c 6f 67 20 73 6f g 5.8.10 , log so
>
> looking at these dumps, I don't think the problem is the <190>, I think the problem is the three characters before that (Q3. in the text represnetation), those start at the same point that the timestamp starts in the last example.

Actually the timestamp starts like 18 bytes earlier in the normal non-spoofed example. Similar position in the row, but a whole row earlier.

> note that the last example is missing the priority/severity information. This is unfortunantly not an uncommon bug.

That is straight from the following line in rsyslog.conf, so if that is broken then it's an rsyslog bug:

kern.info @[zenhost]:1154

--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source and other randomness

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


david at lang

Apr 20, 2012, 1:52 PM

Post #4 of 13 (700 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Fri, 20 Apr 2012, Jo Rhett wrote:

>> the string <190> is the encoded priority and severity of the message (in this case local7.user). A properly formatted syslog message will have this (try sending a message with the format RSYSLOG_Traditional_Forward_Format and you should see a similar thing before the timestamp) If this is breaking the message parsing, the receiving system is broken
>
> I think you mean local7.info? ;-) Thanks for the explanation. Anyway,
> if I just send a message directly to it the system works fine so this
> isn't apparently a problem. It's just having issues with the udpspoofed
> messages.

oops yes

>>> Here's some examples -- msg:2:2048 created with logger -p local7.info "123 This is another test message" and forwarded via udpspoof
>>>
>>> 0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 4a 00 x+.)...& .3.:..J.
>>> 0010 00 52 00 f2 00 00 40 11 b6 bd 0a d2 82 76 0a d2 .R....@. .....v..
>>> 0020 1e ca 00 01 01 01 01 01 01 01 01 01 01 00 01 01 ........ ........
>>> 0030 00 01 00 01 01 00 07 7d 05 ea 00 2a 63 a3 31 32 .......} ...*c.12
>>> 0040 33 20 54 68 69 73 20 69 73 20 61 6e 6f 74 68 65 3 This i s anothe
>>> 0050 72 20 74 65 73 74 20 6d 65 73 73 61 67 65 2e 0a r test m essage..
>>>
>>> rawmsg created with logger -p local7.info 123 "This is another test message" and forwarded via udpspoof
>>>
>>> 0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 4a 00 x+.)...& .3.:..J.
>>> 0010 00 79 00 f2 00 00 40 11 b6 9a 0a d2 82 76 0a d2 .y....@. .....v..
>>> 0020 1e ca 01 00 01 01 00 00 00 01 01 01 01 00 00 00 ........ ........
>>> 0030 01 01 01 00 01 00 02 7d 05 ea 00 51 33 98 3c 31 .......} ...Q3.<1
>>> 0040 39 30 3e 41 70 72 20 32 30 20 31 31 3a 30 30 3a 90>Apr 2 0 11:00:
>>> 0050 30 36 20 73 32 32 2d 77 77 77 30 38 20 6a 6f 72 06 s22-w ww08 jor
>>>
>>> Here's a normal syslog message from the same host sent directly, not forwarded by udpspoof: you might recognize the message ;-)
>>>
>>> 0000 78 2b cb 29 ff 1b 00 26 b9 33 00 3a 08 00 45 00 x+.)...& .3.:..E.
>>> 0010 00 6d 00 00 40 00 40 11 e7 53 0a d2 1e bf 0a d2 .m..@.@. .S......
>>> 0020 1e ca cb aa 05 ea 00 59 32 64 41 70 72 20 32 30 .......Y 2dApr 20
>>> 0030 20 31 37 3a 35 39 3a 33 37 20 73 32 32 2d 66 32 17:59:3 7 s22-f2
>>> 0040 30 31 20 6b 65 72 6e 65 6c 3a 20 69 6d 6b 6c 6f 01 kerne l: imklo
>>> 0050 67 20 35 2e 38 2e 31 30 2c 20 6c 6f 67 20 73 6f g 5.8.10 , log so
>>
>> looking at these dumps, I don't think the problem is the <190>, I think the problem is the three characters before that (Q3. in the text represnetation), those start at the same point that the timestamp starts in the last example.
>
> Actually the timestamp starts like 18 bytes earlier in the normal non-spoofed example. Similar position in the row, but a whole row earlier.
>
>> note that the last example is missing the priority/severity information. This is unfortunantly not an uncommon bug.
>
> That is straight from the following line in rsyslog.conf, so if that is broken then it's an rsyslog bug:
>
> kern.info @[zenhost]:1154

what is your default template? if you could run a quick test and add
;RSYSLOG_Traditional_Forward_Format to the line and see if you get the
format I am expecting.

the fact that the contents is starting so much later in the omspoof
version is definantly a bug. I wonder what it thinks it's doing? could you
write to a file using the same template and see what shows up in the file?

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


jrhett at netconsonance

Apr 20, 2012, 2:43 PM

Post #5 of 13 (691 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Apr 20, 2012, at 1:52 PM, david [at] lang wrote:
>> That is straight from the following line in rsyslog.conf, so if that is broken then it's an rsyslog bug:
>>
>> kern.info @[zenhost]:1154
>
> what is your default template? if you could run a quick test and add ;RSYSLOG_Traditional_Forward_Format to the line and see if you get the format I am expecting.

# Use traditional timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

All of our logs are in this format, we don't use the modern/better format.

> the fact that the contents is starting so much later in the omspoof version is definantly a bug. I wonder what it thinks it's doing? could you write to a file using the same template and see what shows up in the file?

All file writing works exactly as expected.

local7.* /tmp/rawtest.log;Untouched

$ cat /tmp/rawtest.log
<190>Apr 20 14:39:33 sj2-web08 jorhett: 123 This is another test message$

No linefeed, so that's the prompt at the end there.

$ hexdump -c /tmp/rawtest.log
0000000 < 1 9 0 > A p r 2 0 1 4 : 3
0000010 9 : 3 3 s 2 2 - w w w 0 8 j
0000020 o r h e t t : 1 2 3 T h i s
0000030 i s a n o t h e r t e s t
0000040 m e s s a g e
0000048

So I suspect this is just a udpspoof thing. The very odd thing is that either syslogd deals with this, or it only affects spoofing to a non-standard port.

--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source and other randomness

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


david at lang

Apr 20, 2012, 2:46 PM

Post #6 of 13 (693 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Fri, 20 Apr 2012, Jo Rhett wrote:

> On Apr 20, 2012, at 1:52 PM, david [at] lang wrote:
>>> That is straight from the following line in rsyslog.conf, so if that is broken then it's an rsyslog bug:
>>>
>>> kern.info @[zenhost]:1154
>>
>> what is your default template? if you could run a quick test and add ;RSYSLOG_Traditional_Forward_Format to the line and see if you get the format I am expecting.
>
> # Use traditional timestamp format
> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
>
> All of our logs are in this format, we don't use the modern/better format.

minor detail, you should use Forward instead of File when you are sending
the log to another machine, otherwise the priority/severity info is lost.

>> the fact that the contents is starting so much later in the omspoof version is definantly a bug. I wonder what it thinks it's doing? could you write to a file using the same template and see what shows up in the file?
>
> All file writing works exactly as expected.
>
> local7.* /tmp/rawtest.log;Untouched
>
> $ cat /tmp/rawtest.log
> <190>Apr 20 14:39:33 sj2-web08 jorhett: 123 This is another test message$
>
> No linefeed, so that's the prompt at the end there.
>
> $ hexdump -c /tmp/rawtest.log
> 0000000 < 1 9 0 > A p r 2 0 1 4 : 3
> 0000010 9 : 3 3 s 2 2 - w w w 0 8 j
> 0000020 o r h e t t : 1 2 3 T h i s
> 0000030 i s a n o t h e r t e s t
> 0000040 m e s s a g e
> 0000048
>
> So I suspect this is just a udpspoof thing. The very odd thing is that
> either syslogd deals with this, or it only affects spoofing to a
> non-standard port.

yep, this sure looks like a udpspoof problem.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


jrhett at netconsonance

Apr 20, 2012, 4:22 PM

Post #7 of 13 (692 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Apr 20, 2012, at 2:46 PM, david [at] lang wrote:
> minor detail, you should use Forward instead of File when you are sending the log to another machine, otherwise the priority/severity info is lost.

I just retried with this format and it isn't recognized either, fyi.

--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source and other randomness

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


david at lang

Apr 20, 2012, 4:25 PM

Post #8 of 13 (692 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Fri, 20 Apr 2012, Jo Rhett wrote:

> On Apr 20, 2012, at 2:46 PM, david [at] lang wrote:
>> minor detail, you should use Forward instead of File when you are sending the log to another machine, otherwise the priority/severity info is lost.
>
> I just retried with this format and it isn't recognized either, fyi.

I'm not surprised, but with Forward instead of File you should see the
<##> at the beginning of the log message (that's the only difference
between the two formats)

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


david at lang

Apr 20, 2012, 4:30 PM

Post #9 of 13 (688 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Fri, 20 Apr 2012, david [at] lang wrote:

> On Fri, 20 Apr 2012, Jo Rhett wrote:
>
>> On Apr 20, 2012, at 2:46 PM, david [at] lang wrote:
>>> minor detail, you should use Forward instead of File when you are sending
>>> the log to another machine, otherwise the priority/severity info is lost.
>>
>> I just retried with this format and it isn't recognized either, fyi.
>
> I'm not surprised, but with Forward instead of File you should see the <##>
> at the beginning of the log message (that's the only difference between the
> two formats)

wait a minute, I realized just after I sent this that you probably meant
that if you send standard syslog with the Forward format Zenoss doesn't
work, but if you send standard syslog with the File format Zenoss works.

If this is the case, then instead of using '%rawmesg%' for your spoof
template, use '%timestamp% %hostname% %syslogtag%%msg%'

This should be the same thing, just without the severity/priority tag. If
that's what Zenoss is choaking on, this may fix it (and then you can file
a bug with them :-)


David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


jrhett at netconsonance

Apr 20, 2012, 6:03 PM

Post #10 of 13 (696 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Apr 20, 2012, at 4:30 PM, david [at] lang wrote:
> wait a minute, I realized just after I sent this that you probably meant that if you send standard syslog with the Forward format Zenoss doesn't work, but if you send standard syslog with the File format Zenoss works.
>
> If this is the case, then instead of using '%rawmesg%' for your spoof template, use '%timestamp% %hostname% %syslogtag%%msg%'
>
> This should be the same thing, just without the severity/priority tag. If that's what Zenoss is choaking on, this may fix it (and then you can file a bug with them :-)


No, I've only gotten it working with the message only. It doesn't like FileFormat either (see my message). I can try that later tonight. We're at peak traffic right now ;-)

--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source and other randomness

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


david at lang

Apr 20, 2012, 9:09 PM

Post #11 of 13 (691 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Fri, 20 Apr 2012, Jo Rhett wrote:

> On Apr 20, 2012, at 4:30 PM, david [at] lang wrote:
>> wait a minute, I realized just after I sent this that you probably meant that if you send standard syslog with the Forward format Zenoss doesn't work, but if you send standard syslog with the File format Zenoss works.
>>
>> If this is the case, then instead of using '%rawmesg%' for your spoof template, use '%timestamp% %hostname% %syslogtag%%msg%'
>>
>> This should be the same thing, just without the severity/priority tag. If that's what Zenoss is choaking on, this may fix it (and then you can file a bug with them :-)
>
>
> No, I've only gotten it working with the message only. It doesn't like
> FileFormat either (see my message). I can try that later tonight. We're
> at peak traffic right now ;-)

Ok, I'm trying to make sure I am properly understanding what works and
what doesn't (I know that at some point here I have gotten confused, and
I'm not sure I have it all straightend out yet)

My understanding is that the following scenarios have been tested

1. @hostname:port with FileFormat

works

2. @hostname:port with ForwardFormat

works? I thought the message I was replying to said it did not
work.

3. omspoof with ForwardFormat (%rawmesg%)

does not work

4. omspoof with message only (%msg:2:2000%)

works

5. omspoof with FileFormat (%timestamp% %hostname% %syslogtag%%msg%)


does not work? this is what I think you are saying above



where FileFormat is

Apr 19 12:34:56 hostname application[PID]: log data

and ForwardFormat is

<190>Apr 19 12:34:56 hostname application[PID]: log data

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


rgerhards at hq

Apr 21, 2012, 1:50 AM

Post #12 of 13 (692 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

> >>> kern.info @[zenhost]:1154
> >>
> >> what is your default template? if you could run a quick test and add
> ;RSYSLOG_Traditional_Forward_Format to the line and see if you get the
> format I am expecting.
> >
> > # Use traditional timestamp format
> > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

Be careful, this is the default format for *writing logfiles* - it does not
affect omudpspoof. I have just checked the code, the default format is
hardcoded to RSYSLOG_TraditionalForwardFormat, which should be just right
(with the PRI at the beginning of the message).

I could add some additional debug instrumentation to omudpspoof, mabe that
gets us some additional information about the internal state. Of course, this
means that a debug log must be generated. Should I look into that?

Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


jrhett at netconsonance

Apr 23, 2012, 12:40 PM

Post #13 of 13 (680 views)
Permalink
Re: rawmessage forwarding doesn't appear to work -- very detailed [In reply to]

On Apr 20, 2012, at 1:25 PM, david [at] lang wrote:
> the string <190> is the encoded priority and severity of the message (in this case local7.user). A properly formatted syslog message will have this (try sending a message with the format RSYSLOG_Traditional_Forward_Format and you should see a similar thing before the timestamp) If this is breaking the message parsing, the receiving system is broken

As you'll see in the pcap file I just sent you directly (check your spam folder) the priority remains intact from one system to the other. The missing priority appears to be related to kern messages as processed by rsyslog and is not related to this.

> looking at these dumps, I don't think the problem is the <190>, I think the problem is the three characters before that (Q3. in the text represnetation), those start at the same point that the timestamp starts in the last example.


So the pcap file I just sent shows the receipt of the local7 message (with priority) and forwarded message from the rsyslog server's perspective. I hope this can show you what you need to know to get this identified.

--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source and other randomness

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

RSyslog users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.