david at lang
Oct 11, 2011, 10:07 PM
Post #2 of 4
(298 views)
Permalink
|
|
Re: templates not working on second-hop relay?
[In reply to]
|
|
On Tue, 11 Oct 2011, Gregory K. Ruiz-Ade wrote:
> I'm currently using rsyslog 4.6.5, as packaged by IUS for CentOS 5.6.
>
> I've replicated this problem on rsyslog 5.8.5, compiled from sources on CentOS 5.6 and Ubuntu 11.04.
>
> I've been testing a configuration where there's two hops in my logging.
>
> Basically, client -> local loghost -> archive loghost.
>
> On the client, I'm using a template to add a tag to the start of the $msg property. On the servers, I then look for this tag for dynafile purposes, and use a second template to strip out the tag from $msg before it's written to file or passed to a database. On a simple client -> loghost setup, this works fine.
>
> However, when I then added an archive loghost, which the local loghost relays everything to, none of the templates seem to be working.
>
> The client config can be simplified to:
>
> #---------------------------------------
> $template SiteIDForwardFormat, "<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%[SITE:datacenter/dev]%msg:R,ERE,3,FIELD::sp-if-no-1st-sp%%msg%"
>
> *.* @@loghost:1514;SiteIDForwardFormat
> #---------------------------------------
>
> The loghost config is, essentially:
>
> #---------------------------------------
> $ModLoad imtcp.so
> $InputTCPServerRun 1514
>
> $CreateDirs on
>
> $template SiteIDTaggedMsg, "%timestamp% %hostname% %syslogtag%%msg:R,ERE,3,FIELD:(\[SITE:([-/a-zA-Z0-9]+)\] ){0,1}(.*)$--end%\n"
>
> $template SiteIDTaggedPath, "/data/syslog/logs/%msg:R,ERE,2,BLANK:(\[SITE:([-/a-zA-Z0-9]+)\] ){0,1}(.*)$--end%/%hostname:::secpath-replace%/%$year%/%$month%/syslog.log"
>
> *.* ?SiteIDTaggedPath;SiteIDTaggedMsg
>
> *.* @@logarchive
> #---------------------------------------
>
> For all intents and purposes, the archive loghost has the exact same config as the local loghost, sans the @@logarchive action.
>
> On the local loghost, messages are being properly filed into the expected dynafiles paths (/data/syslog/logs/datacenter/dev/hostname/year/month/syslog.log) and the "[SITE:something]" text is correctly removed from the messages written to the files.
>
> On the archive loghost, however, the dynafiles are missing the site component (ending up as /data/syslog/logs/hostname/year/month/syslog.log), and all the messages in the files still have the "[SITE:something]" text prepended to the $msg property.
>
> Why is the second (archive) log host not able to properly parse the messages with the templates? am I missing something, or is the relay step munging the event in such a way as the EREs are no longer working as I think they should?
>
> Thanks for any help,
I think that what you end up doing is manipulating the logfile too much on
the relay system.
make a temporary output file on the relay system to log with the debug
format (RSYSLOG_DEBUG IIRC) and you will see everything that is arriving,
then decide if you need to manipulate it more before sending it on to the
archive server.
remember that each step in the process is working with what it was sent,
not what the software originally wrote to the log.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
|
