skyeh at uidaho
Jan 20, 2012, 11:14 AM
Post #6 of 6
(204 views)
Permalink
|
The syslog events are indexed in Splunk with a specific sourcetype, in our
case 'Rancid'. We run RANCID via cron once a day. Therefore, our Splunk
reports are on a daily schedule. Here is a list of the reports that we run.
Consistent errors, where the same device has an error for the last two
weeks. Run daily, for the last 2 weeks. Query: sourcetype="Rancid"
"rancid-run" error | stats count by device | where count > 13
Excessive errors, where we have a large number of errors in a single run.
Run daily, for the last 24 hours. Query: sourcetype="Rancid" rancid Errors >
7
Did not finish, where we do not see the end of the run for our three RANCID
groups. Run daily, for the last 24 hours. Query: sourcetype="Rancid"
rancid-run ending: | stats count by date_mday | where count != 3
The reports are only emailed to our network team if the event count is
greater than zero. In other words, we only get notified if there is a
problem.
If you want to discuss this further, lets take it off-list.
Skye.
On 1/20/12 12:52 AM, "Krzysztof Zygmunt" <krzysztof.zygmunt [at] gmail>
wrote:
> Hi,
>
> It looks very interesting, can you tell me what do you do with those
> syslog messages
> on splunk side ? How do you generate any reports from such information ?
>
> cheers
> kris
>
> 2012/1/18 Hagen, Skye <skyeh [at] uidaho>:
>> We took a completely different tact on notification. We backup over 1600
>> devices nightly, and only want notification if there is a problem, and in
>> some cases, only when there is a major problem.
>>
>> We use syslog with Splunk for a majority of our reporting. What I did was to
>> wrap 'rancid-run' in a shell script that will take the logs, massage them,
>> and send the results to syslog.
>>
>> We have created a number of Splunk reports, such as a report that tells us
>> if there are excessive errors, or if a backup has had successive failures.
>>
>> If anyone is interested, I have attached the script.
>>
>> Skye.
>>
>>
>>
>>
>>
>> On 1/16/12 5:58 PM, "Michael Lee" <michael.lee [at] mincom> wrote:
>>
>>> Hi,
>>>
>>> Really appreciate your reply. Yeah, however I am trying to list out all
>>> the
>>> devices and backup status in a list. By the way, do you know anyway to
>>> send
>>> the configuration diff using html format in an EMAIL similar to what we
>>> see on
>>> VIEWVC?
>>>
>>>
>>> BR,
>>> Michael
>>> -----Original Message-----
>>> From: shouldbe q931 [mailto:shouldbeq931 [at] gmail]
>>> Sent: Tuesday, January 17, 2012 2:55 AM
>>> To: Michael Lee
>>> Cc: rancid-discuss [at] shrubbery
>>> Subject: Re: [rancid] Email notification with RANCID
>>>
>>> On Mon, Jan 16, 2012 at 5:33 AM, Michael Lee <michael.lee [at] mincom>
>>> wrote:
>>>> Hi all,
>>>>
>>>>
>>>>
>>>> I am new to RANCID, hopefully you guys can help. I have been trying to
>>>> find
>>>> some way to make RANCID notify daily through mail which devices is backup
>>>> successful and which backup is not successful.
>>>>
>>>> Is there currently a way to do this. I notice the capability at the
>>>> moment
>>>> only send notification on the changes of the configuration.
>>>>
>>>>
>>>>
>>>> Many thanks in advance,
>>>>
>>>>
>>>>
>>>> Br,
>>>>
>>>> Michael
>>>>
>>>>
>>>
>>> rancid will send a notification email if it has not been able to
>>> contact a device for 24hours, the below is from control_rancid
>>>
>>>
>>> ----------------------------------------------------------------------------
>>> --
>>> -----------------
>>> # If any machines have not been reached within the last $OLDTIME
>>> # hours, mail out a list of them.
>>> cd $DIR/configs
>>> rm -f $DIR/routers.failed
>>> if [ "X$OLDTIME" = "X" ] ; then
>>> OLDTIME=24
>>> fi
>>>
>>> ----------------------------------------------------------------------------
>>> --
>>> -----------------
>>>
>>>
>>> This transmission is for the intended addressee only and is confidential
>>> information. If you have received this transmission in error, please
>>> notify
>>> the sender and delete the transmission. The contents of this e-mail are
>>> the
>>> opinion of the writer only and are not endorsed by the Mincom Group of
>>> companies unless expressly stated otherwise.
>>>
>>>
>>> _______________________________________________
>>> Rancid-discuss mailing list
>>> Rancid-discuss [at] shrubbery
>>> http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
>>
>>
>> _______________________________________________
>> Rancid-discuss mailing list
>> Rancid-discuss [at] shrubbery
>> http://www.shrubbery.net/mailman/listinfo.cgi/rancid-discuss
|
RunRANCID.sh.txt
