Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: RSyslog: users

adding a tag to log messages

 

 

RSyslog users RSS feed   Index | Next | Previous | View Threaded


martyleisner at yahoo

Aug 7, 2012, 12:28 PM

Post #1 of 7 (210 views)
Permalink
adding a tag to log messages

Right before the log message is delivered to a file, I want to intercept the message and append
a hash to the message.

I want to hash the entire message (including the time) and append the hash to the line
(along with "secret sauce")

I want to use a plugin to do this.

What's a good way to do this?
I was thinking of writing a strgen module, but it doesn't seem to have
access to whole line before written to the log file.

I can customize the config file to do whatever I need it to do.


Marty
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


david at lang

Aug 7, 2012, 10:42 PM

Post #2 of 7 (200 views)
Permalink
Re: adding a tag to log messages [In reply to]

On Tue, 7 Aug 2012, marty wrote:

> Right before the log message is delivered to a file, I want to intercept the message and append
> a hash to the message.
>
> I want to hash the entire message (including the time) and append the hash to the line
> (along with "secret sauce")
>
> I want to use a plugin to do this.
>
> What's a good way to do this?
> I was thinking of writing a strgen module, but it doesn't seem to have
> access to whole line before written to the log file.
>
> I can customize the config file to do whatever I need it to do.

I don't know enough to answer this and Rainer is on vacation for a week,
so you will probably have to wait for him to get back (and possibly as
again if he misses it in his pile of unread messages when he gets back)

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


briank at talksum

Aug 8, 2012, 6:09 AM

Post #3 of 7 (203 views)
Permalink
Re: adding a tag to log messages [In reply to]

Off the top of my head, I'd say the mmnormalize plugin is probably a good
one to look at to get some ideas. It definitely has access to the entire
message as it's passing the message to liblognorm. mmjsonparse also works
with the entire message to pass it to a json parser. Neither of these
modules rewrite the message itself but they might give you a good starting
place.

Brian

On Wed, Aug 8, 2012 at 1:42 AM, <david [at] lang> wrote:

> On Tue, 7 Aug 2012, marty wrote:
>
> Right before the log message is delivered to a file, I want to intercept
>> the message and append
>> a hash to the message.
>>
>> I want to hash the entire message (including the time) and append the
>> hash to the line
>> (along with "secret sauce")
>>
>> I want to use a plugin to do this.
>>
>> What's a good way to do this?
>> I was thinking of writing a strgen module, but it doesn't seem to have
>> access to whole line before written to the log file.
>>
>> I can customize the config file to do whatever I need it to do.
>>
>
> I don't know enough to answer this and Rainer is on vacation for a week,
> so you will probably have to wait for him to get back (and possibly as
> again if he misses it in his pile of unread messages when he gets back)
>
> David Lang
>
> ______________________________**_________________
> rsyslog mailing list
> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
> What's up with rsyslog? Follow https://twitter.com/rgerhards
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


briank at talksum

Aug 8, 2012, 6:11 AM

Post #4 of 7 (200 views)
Permalink
Re: adding a tag to log messages [In reply to]

Thinking about it a little more - both mmjsonparse and mmnormalize produce
properties derived from the message that are then accessable from templates
- perhaps you could do something similar, and assign the hash to a property
that you could then include in a template for the outputs you want the hash
in?

Brian

On Wed, Aug 8, 2012 at 9:09 AM, Brian Knox <briank [at] talksum> wrote:

> Off the top of my head, I'd say the mmnormalize plugin is probably a good
> one to look at to get some ideas. It definitely has access to the entire
> message as it's passing the message to liblognorm. mmjsonparse also works
> with the entire message to pass it to a json parser. Neither of these
> modules rewrite the message itself but they might give you a good starting
> place.
>
> Brian
>
>
> On Wed, Aug 8, 2012 at 1:42 AM, <david [at] lang> wrote:
>
>> On Tue, 7 Aug 2012, marty wrote:
>>
>> Right before the log message is delivered to a file, I want to intercept
>>> the message and append
>>> a hash to the message.
>>>
>>> I want to hash the entire message (including the time) and append the
>>> hash to the line
>>> (along with "secret sauce")
>>>
>>> I want to use a plugin to do this.
>>>
>>> What's a good way to do this?
>>> I was thinking of writing a strgen module, but it doesn't seem to have
>>> access to whole line before written to the log file.
>>>
>>> I can customize the config file to do whatever I need it to do.
>>>
>>
>> I don't know enough to answer this and Rainer is on vacation for a week,
>> so you will probably have to wait for him to get back (and possibly as
>> again if he misses it in his pile of unread messages when he gets back)
>>
>> David Lang
>>
>> ______________________________**_________________
>> rsyslog mailing list
>> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
>> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>
>
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


rgerhards at hq

Aug 8, 2012, 7:19 AM

Post #5 of 7 (201 views)
Permalink
Re: adding a tag to log messages [In reply to]

Strgen has full access and is a good place for this. See tools/smfile for an example.

Rg





Von Samsung Mobile gesendet

marty <martyleisner [at] yahoo> hat geschrieben:
Right before the log message is delivered to a file, I want to intercept the message and append
a hash to the message.

I want to hash the entire message (including the time) and append the hash to the line
(along with "secret sauce")

I want to use a plugin to do this.

What's a good way to do this?
I was thinking of writing a strgen module, but it doesn't seem to have
access to whole line before written to the log file.

I can customize the config file to do whatever I need it to do.


Marty
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


martyleisner at yahoo

Aug 9, 2012, 6:32 PM

Post #6 of 7 (204 views)
Permalink
Re: adding a tag to log messages [In reply to]

I looked at a strgen method.  

But it seems there is more processing possible after a strgen module is called
(I need the whole line with the timestamp).  The format of the timestamps are very

changeable.

I'm looking to compute a cryptographic hash of the final line before its delivered
to the log files but after all processing is done on the line.  Is there a way to

guarantee a strgen method is called after all other processing?


I suppose I could modify stream.c:doWriteCall (and perhaps add a means to turn

on this extra processing).  But that wouldn't be a plugin but hacking the rsyslog
source.


Would this comply with gplv3 if rsyslog is used locally and never distributed?
The object is be able distinguish valid log lines from  hacker injected log lines
(i.e. the hash algorithm would not be public).


Marty




>________________________________
> From: Rainer Gerhards <rgerhards [at] hq>
>To: "martyleisner [at] yahoo" <martyleisner [at] yahoo>; "rsyslog [at] lists" <rsyslog [at] lists>
>Sent: Wednesday, August 8, 2012 10:19 AM
>Subject: AW: [rsyslog] adding a tag to log messages
>
>
>
>Strgen has full access and is a good place for this. See tools/smfile for an example.
>
>
>Rg
>
>
>
>
>
>
>
>
>
>
>Von Samsung Mobile gesendet
>marty <martyleisner [at] yahoo> hat geschrieben:
>Right before the log message is delivered to a file, I want to intercept the message and append
>a hash to the message.
>
>I want to hash the entire message (including the time) and append the hash to the line
>(along with "secret sauce")
>
>I want to use a plugin to do this.
>
>What's a good way to do this?
>I was thinking of writing a strgen module, but it doesn't seem to have
>access to whole line before written to the log file.
>
>I can customize the config file to do whatever I need it to do.
>
>
>Marty
>_______________________________________________
>rsyslog mailing list
>http://lists.adiscon.net/mailman/listinfo/rsyslog
>http://www.rsyslog.com/professional-services/
>What's up with rsyslog? Follow https://twitter.com/rgerhards
>
>
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


david at lang

Aug 10, 2012, 12:19 PM

Post #7 of 7 (200 views)
Permalink
Re: adding a tag to log messages [In reply to]

On Thu, 9 Aug 2012, marty wrote:

> Would this comply with gplv3 if rsyslog is used locally and never distributed?
> The object is be able distinguish valid log lines from  hacker injected log lines
> (i.e. the hash algorithm would not be public).

The GPL only comes in to play when you distribute the code. If you only
ever use it locally the GPL is never triggered.

However, if you are really working to defend against the files being
modified, simply hashing each line independantly is not good enough.

You should take a look at logtools
http://www.rsyslog.com/logtools-0-1-0-released

This implements hashing of log file lines in a way that prevents
individual lines from being removed or added.

Since this is already available in rsyslog, you can look at it for how to
implement your own solution if you still want to roll your own.

David Lang

RSyslog users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.