Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Varnish: Dev

varnishstatd stats daemon

 

 

Varnish dev RSS feed   Index | Next | Previous | View Threaded


jspam at skopis

Mar 26, 2012, 9:13 PM

Post #1 of 3 (500 views)
Permalink
varnishstatd stats daemon

Hello,

I made a daemon to collect stats from varnishstat and calculate 1/5/15
min averages, and 1/5/15 min averages of the deltas. The stats are
then exported over HTTP.

I intend on using the check_varnish.pl script (queries varnishstatd)
to monitor our varnish instance using nagios.

The code is here: https://github.com/johnskopis/varnishstatd

I would love to hear your comments/suggestions.


Thanks,
John

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev


andrea.campi at zephirworks

Mar 26, 2012, 11:40 PM

Post #2 of 3 (451 views)
Permalink
Re: varnishstatd stats daemon [In reply to]

On Tue, Mar 27, 2012 at 6:13 AM, John Skopis <jspam [at] skopis> wrote:
> Hello,
>
> I made a daemon to collect stats from varnishstat and calculate 1/5/15
> min averages, and 1/5/15 min averages of the deltas. The stats are
> then exported over HTTP.
>
> I intend on using the check_varnish.pl script (queries varnishstatd)
> to monitor our varnish instance using nagios.
>
> The code is here: https://github.com/johnskopis/varnishstatd
>
> I would love to hear your comments/suggestions.

Reading the name I was hoping you were going in the direction I'm
thinking of going :)
That is, pushing statistics to statsd [1] and from there to Graphite.

My rationale is that we are going to use statsd for other components
of our architecture; but even looking at just Varnish, it has
potential to be more scalable. In particular, I'm looking at
collecting more in-depth stats via the SHM.
Last year I did a spike on that [2,3] and while my naive approach
broke down under moderate load, it looks like if I keep processing at
a minimum and just fire off UDP I should be able to keep up with a
much higher traffic.

I'm not ready to work on that right now, but I will get there. If this
is something that might interest you, maybe we can combine efforts.

Andrea


1: https://github.com/etsy/statsd
2: https://github.com/andreacampi/varnish-rb
3: https://github.com/andreacampi/newrelic_varnish

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev


jspam at skopis

Mar 27, 2012, 6:48 AM

Post #3 of 3 (453 views)
Permalink
Re: varnishstatd stats daemon [In reply to]

On Tue, Mar 27, 2012 at 1:40 AM, Andrea Campi
<andrea.campi [at] zephirworks> wrote:

> Reading the name I was hoping you were going in the direction I'm
> thinking of going :)
> That is, pushing statistics to statsd [1] and from there to Graphite.

Sadly, we are still using cacti (and pnp4nagios) to monitor
performance information. At this point I am more interested in
monitoring individual counters in nagios. I will be interested in
collecting performance data next.

I planned on trying graphite. I am no expert in tornado but it seems
they have a UDP transport so similar to how the stats are updated an
event would fire and trigger sending the stats to graphite.

Do most people use statsD instead of just sending rows to graphite?

>
> My rationale is that we are going to use statsd for other components
> of our architecture; but even looking at just Varnish, it has
> potential to be more scalable. In particular, I'm looking at
> collecting more in-depth stats via the SHM.
> Last year I did a spike on that [2,3] and while my naive approach
> broke down under moderate load, it looks like if I keep processing at
> a minimum and just fire off UDP I should be able to keep up with a
> much higher traffic.

Why did it break down under load?

In my testing I thought it would be faster to use FFI to get at the
stats...but the callback function added quite a bit of overhead.

>
> I'm not ready to work on that right now, but I will get there. If this
> is something that might interest you, maybe we can combine efforts.
>
> Andrea
>
>
> 1: https://github.com/etsy/statsd
> 2: https://github.com/andreacampi/varnish-rb
> 3: https://github.com/andreacampi/newrelic_varnish
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev [at] varnish-cache
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

Varnish dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.