steve.chupack at dealer
Sep 11, 2011, 1:50 PM
Views: 432
Permalink
|
|
Unable to implement on-disk queuing
|
|
Hi,
I have rsyslogd up and running and logging to mysql with loganalyzer as a front end. Very cool.
However, I can't seem to get on-disk queuing working, which would be nice if mysql goes down or a table is locked because I'm purging records during a nightly maintenance cron.
I never see any queue files created. I dropped the size of the memory queue to something ridiculously small, but still no queue files even after 20 minutes.
I followed the instructions given here: http://www.rsyslog.com/doc/rsyslog_high_database_rate.html along with a couple of other tutorials on rsyslogd and queuing.
I'm quite sure I'm just completely missing something obvious. Details below... And thanks to anyone who might be able to point out what I'm doing wrong.
-Steve
MY CONFIG, please, no flames re the horrendously complex rules ;-)
# if you experience problems, check
# http://www.rsyslog.com/troubleshoot for assistance
# rsyslog v3: load input modules
# If you do not load inputs, nothing happens!
# You may need to set the module load path if modules are not found.
$ModLoad immark # provides --MARK-- message capability
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog # kernel logging (formerly provided by rklogd)
$ModLoad ommysql
# SC 2010.11.11: configure disk caching in case mysql is unavailable
$MainMsgQueueSize 5
$WorkDirectory /var/log/rsyslogq # default location for work (spool) files
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName dbq # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
# send snmpd INFO messages to the dustbin
if $syslogseverity-text == 'info' and $syslogtag contains 'snmpd' then ~
$template vtfw,"insert into vtfw (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL
if ($fromhost-ip contains '10.128.255') then :ommysql:localhost,Syslog,root,mysqldb44;vtfw
& ~
#if ($fromhost-ip contains '10.128.255') then ~
$template vt1hs1_switches,"insert into vt1hs1_switches (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL
if $fromhost-ip contains '10.128.0' or $hostname contains_i 'core01' or $hostname contains_i 'core02' then :ommysql:localhost,Syslog,root,mysqldb44;vt1hs1_switches
#if ($fromhost-ip contains '10.128.0' or $source=='vt1hs1-dc216-core01' or $source=='vt1hs1-dc216-core02') then /var/log/switches
& ~
#if $fromhost-ip contains '10.128.0' or $hostname contains_i 'core01' or $hostname contains_i 'core02' then ~
$template vt1hs1_wifi,"insert into vt1hs1_wifi (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL
if $fromhost-ip contains '10.128.244' then :ommysql:localhost,Syslog,root,mysqldb44;vt1hs1_wifi
& ~
#if $fromhost-ip contains '10.128.244' then ~
$template vt1hs1_vsphere,"insert into vt1hs1_vsphere (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL
if ($fromhost-ip == '10.128.24.40' or $fromhost-ip == '10.128.24.44' or $fromhost-ip == '10.128.24.48' or $fromhost-ip == '10.128.24.52' or $fromhost-ip == '10.128.24.72' or $fromhost-ip == '10.128.24.76' or $fromhost-ip == '10.128.24.80' or $fromhost-ip == '10.128.24.84') and ($syslogseverity-text == 'warning' or $syslogseverity-text == 'warn' or $syslogseverity-text == 'err' or $syslogseverity-text == 'error' or $syslogseverity-text == 'crit' or $syslogseverity-text == 'alert' or $syslogseverity-text == 'emergency' or $syslogseverity-text == 'panic') then :ommysql:localhost,Syslog,root,mysqldb44;vt1hs1_vsphere
& ~
if ($fromhost-ip == '10.128.24.40' or $fromhost-ip == '10.128.24.44' or $fromhost-ip == '10.128.24.48' or $fromhost-ip == '10.128.24.52' or $fromhost-ip == '10.128.24.72' or $fromhost-ip == '10.128.24.76' or $fromhost-ip == '10.128.24.80' or $fromhost-ip == '10.128.24.84') and ($syslogseverity-text == 'notice') then ~
#*.* :ommysql:localhost,Syslog,root,mysqldb44
*.notice :ommysql:localhost,Syslog,root,mysqldb44
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none -/var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* -/var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit -/var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
# Remote Logging (we use TCP for reliable delivery)
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$WorkDirectory /rsyslog/spool # where to place spool files
#$ActionQueueFileName uniqName # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList # run asynchronously
#$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ######### Receiving Messages from Remote Hosts ##########
# TCP Syslog Server:
# provides TCP syslog reception and GSS-API (if compiled to support it)
$ModLoad imtcp.so # load module
$InputTCPServerRun 1470 # start up TCP listener at port 514
# UDP Syslog Server:
$ModLoad imudp.so # provides UDP syslog reception
$UDPServerRun 514 # start a UDP syslog server at standard port 514
SOME DEBUG OUTPUT:
(see attached)
Steve Chupack | IT Systems Administrator
V: 877.327.8422 x 1242
Steve.Chupack [at] dealer | www.dealer.com
|
rsyslog-debug.txt
Unable to implement on-disk queuing
