Mailing List Archive: Varnish: Misc

Varnish 3.0.2 on FreeBSD 8-STABLE: Constant Restart

Index | Next | Previous | View Threaded

lists at hub

Jul 13, 2012, 10:51 AM

Post #1 of 7 (275 views)
Permalink

Varnish 3.0.2 on FreeBSD 8-STABLE: Constant Restart

Morning …

OS: FreeBSD 8.3-STABLE #2: Mon Jul 2 21:20:02 UTC 2012

I am trying to use varnish as a caching front end to a wordpress site … found some wordpress specific templetes for vcl_recv / vcl_fetch, to provide me with a base, and everything starts up, and stays running fine, *until* I change my DNS to have the domain go through Varnish … and then it starts rebooting …

This test through, before I switched DNS, I had an uptime (according to varnish stat) of just over 1 hr … as soon as I switched DNS over, uptime kept resetting to 0, with (after switching DNS back again), I'm currently seeing:

0+00:07:44
Hitrate ratio: 0 0 0
Hitrate avg: 0.0000 0.0000 0.0000

And, according to ps, you can see the Child much younger then the Mgr:

root [at] lb:/var/log # ps aux | grep varnish
www 52281 0.0 0.0 355674084 6432 ?? IJ 5:34PM 0:00.16 varnishd: Varnish-Chld lb1.hospitalityleaders.com (varnishd)
root 74714 0.0 0.0 98032 2636 ?? SsJ 4:21PM 0:00.13 varnishd: Varnish-Mgr lb1.hospitalityleaders.com (varnishd)
root 71856 0.0 0.0 540 328 2 S+J 5:44PM 0:00.00 grep varnish

I'm new with this, so not really sure how best to debug … I did see something from PHK in one of my searches talking about an issue with cookies … ? But I haven't been able to re-find that posting …

I've also, so far, been unable to find a core file, although I'm currently running a find across the whole file system to see if there isn't one …

Any suggestions on where to start debugging this, or more information I can provide, would be appreciated …

Thank you ...
_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

lists at hub

Jul 17, 2012, 6:35 PM

Post #2 of 7 (265 views)
Permalink

Re: Varnish 3.0.2 on FreeBSD 8-STABLE: Constant Restart [In reply to]

Okay, I'm lost here … what sort of site is Varnish good for … ? A news web site where nobody every logs into the site … ?

For instance, the impression I'm getting is that you couldn't use Varnish in front of a Facebook like site, since none of the hits would be anonymous? But would be potentially useful in front of a Slashdot like site, where alot of the hits *are* anonymous?

The reason I started to look at Varnish was a client asked about it in terms of the W3T Cache that is available for Wordpress … it apparently has the ability to 'invalidate cache' in a varnish server, so as to force it to reload from apache … but I don't know enough about Wordpress to know if I can even safely disable cookies, but without disabling cookies, nothing will get cached, making Varnish good as a load balancer, but losing the major benefit …

… no?

On 2012-07-13, at 11:53 PM, Nick Tailor wrote:

> Also you only want anonymous visits cached. All cookies with session Id should be passed to apache and then you want look trying to setup should ur backend fail run off the cache only
>
> Sent from my iPhone
>
> On Jul 13, 2012, at 10:51 AM, Marc Fournier <lists [at] hub> wrote:
>
>>
>> Morning …
>>
>> OS: FreeBSD 8.3-STABLE #2: Mon Jul 2 21:20:02 UTC 2012
>>
>> I am trying to use varnish as a caching front end to a wordpress site … found some wordpress specific templetes for vcl_recv / vcl_fetch, to provide me with a base, and everything starts up, and stays running fine, *until* I change my DNS to have the domain go through Varnish … and then it starts rebooting …
>>
>> This test through, before I switched DNS, I had an uptime (according to varnish stat) of just over 1 hr … as soon as I switched DNS over, uptime kept resetting to 0, with (after switching DNS back again), I'm currently seeing:
>>
>> 0+00:07:44
>> Hitrate ratio: 0 0 0
>> Hitrate avg: 0.0000 0.0000 0.0000
>>
>> And, according to ps, you can see the Child much younger then the Mgr:
>>
>> root [at] lb:/var/log # ps aux | grep varnish
>> www 52281 0.0 0.0 355674084 6432 ?? IJ 5:34PM 0:00.16 varnishd: Varnish-Chld lb1.hospitalityleaders.com (varnishd)
>> root 74714 0.0 0.0 98032 2636 ?? SsJ 4:21PM 0:00.13 varnishd: Varnish-Mgr lb1.hospitalityleaders.com (varnishd)
>> root 71856 0.0 0.0 540 328 2 S+J 5:44PM 0:00.00 grep varnish
>>
>> I'm new with this, so not really sure how best to debug … I did see something from PHK in one of my searches talking about an issue with cookies … ? But I haven't been able to re-find that posting …
>>
>> I've also, so far, been unable to find a core file, although I'm currently running a find across the whole file system to see if there isn't one …
>>
>> Any suggestions on where to start debugging this, or more information I can provide, would be appreciated …
>>
>> Thank you ...
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc [at] varnish-cache
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

robert at shilston

Jul 18, 2012, 12:49 AM

Post #3 of 7 (217 views)
Permalink

Re: Varnish 3.0.2 on FreeBSD 8-STABLE: Constant Restart [In reply to]

Marc,

We successfully run Varnish in front of a large Wordpress site and achieve a good hit rate. At a very high level, most access is anonymous, and we simple PIPE access if there's wp-admin in the URL. We happen to use a custom plugin to manage submitting bans to Varnish, but I believe off the shelf ones are now available.

However, undertaking putting Varnish in front of any site without a strong understanding of how the site works will not yield the best results. Only by understanding the traffic and requests will you be able to safely get a high degree of caching.

Rob

On 18 Jul 2012, at 02:35, Marc Fournier wrote:

>
> Okay, I'm lost here … what sort of site is Varnish good for … ? A news web site where nobody every logs into the site … ?
>
> For instance, the impression I'm getting is that you couldn't use Varnish in front of a Facebook like site, since none of the hits would be anonymous? But would be potentially useful in front of a Slashdot like site, where alot of the hits *are* anonymous?
>
> The reason I started to look at Varnish was a client asked about it in terms of the W3T Cache that is available for Wordpress … it apparently has the ability to 'invalidate cache' in a varnish server, so as to force it to reload from apache … but I don't know enough about Wordpress to know if I can even safely disable cookies, but without disabling cookies, nothing will get cached, making Varnish good as a load balancer, but losing the major benefit …
>
> … no?
>
>
>
> On 2012-07-13, at 11:53 PM, Nick Tailor wrote:
>
>> Also you only want anonymous visits cached. All cookies with session Id should be passed to apache and then you want look trying to setup should ur backend fail run off the cache only
>>
>> Sent from my iPhone
>>
>> On Jul 13, 2012, at 10:51 AM, Marc Fournier <lists [at] hub> wrote:
>>
>>>
>>> Morning …
>>>
>>> OS: FreeBSD 8.3-STABLE #2: Mon Jul 2 21:20:02 UTC 2012
>>>
>>> I am trying to use varnish as a caching front end to a wordpress site … found some wordpress specific templetes for vcl_recv / vcl_fetch, to provide me with a base, and everything starts up, and stays running fine, *until* I change my DNS to have the domain go through Varnish … and then it starts rebooting …
>>>
>>> This test through, before I switched DNS, I had an uptime (according to varnish stat) of just over 1 hr … as soon as I switched DNS over, uptime kept resetting to 0, with (after switching DNS back again), I'm currently seeing:
>>>
>>> 0+00:07:44
>>> Hitrate ratio: 0 0 0
>>> Hitrate avg: 0.0000 0.0000 0.0000
>>>
>>> And, according to ps, you can see the Child much younger then the Mgr:
>>>
>>> root [at] lb:/var/log # ps aux | grep varnish
>>> www 52281 0.0 0.0 355674084 6432 ?? IJ 5:34PM 0:00.16 varnishd: Varnish-Chld lb1.hospitalityleaders.com (varnishd)
>>> root 74714 0.0 0.0 98032 2636 ?? SsJ 4:21PM 0:00.13 varnishd: Varnish-Mgr lb1.hospitalityleaders.com (varnishd)
>>> root 71856 0.0 0.0 540 328 2 S+J 5:44PM 0:00.00 grep varnish
>>>
>>> I'm new with this, so not really sure how best to debug … I did see something from PHK in one of my searches talking about an issue with cookies … ? But I haven't been able to re-find that posting …
>>>
>>> I've also, so far, been unable to find a core file, although I'm currently running a find across the whole file system to see if there isn't one …
>>>
>>> Any suggestions on where to start debugging this, or more information I can provide, would be appreciated …
>>>
>>> Thank you ...
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc [at] varnish-cache
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc [at] varnish-cache
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

apj at mutt

Jul 18, 2012, 12:57 AM

Post #4 of 7 (256 views)
Permalink

Re: Varnish 3.0.2 on FreeBSD 8-STABLE: Constant Restart [In reply to]

On Tue, Jul 17, 2012 at 09:35:34PM -0400, Marc Fournier wrote:
>
> Okay, I'm lost here â€¦ what sort of site is Varnish good for â€¦ ? A news web
> site where nobody every logs into the site â€¦ ?

Varnish can be used for that, you just have to tell it to vary the cache based
on the login.

> For instance, the impression I'm getting is that you couldn't use Varnish in
> front of a Facebook like site, since none of the hits would be anonymous?
> But would be potentially useful in front of a Slashdot like site, where alot
> of the hits *are* anonymous?

You can use varnish in front of both (I believe that facebook actually uses
varnish to some extent)

> The reason I started to look at Varnish was a client asked about it in terms
> of the W3T Cache that is available for Wordpress â€¦ it apparently has the
> ability to 'invalidate cache' in a varnish server, so as to force it to
> reload from apache â€¦ but I don't know enough about Wordpress to know if I can
> even safely disable cookies, but without disabling cookies, nothing will get
> cached, making Varnish good as a load balancer, but losing the major benefit

Nothing will be cached *in the default configuration*, if Cookie-headers are
present in the request or Set-Cookie headers in the response.

Your task is to write the configuration that fits your application needs.

--
Andreas

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

daniel.carrillo at gmail

Jul 18, 2012, 1:01 AM

Post #5 of 7 (265 views)
Permalink

Re: Varnish 3.0.2 on FreeBSD 8-STABLE: Constant Restart [In reply to]

2012/7/18 Marc Fournier <lists [at] hub>:
>
> Okay, I'm lost here … what sort of site is Varnish good for … ? A news web site where nobody every logs into the site … ?
>
> For instance, the impression I'm getting is that you couldn't use Varnish in front of a Facebook like site, since none of the hits would be anonymous? But would be potentially useful in front of a Slashdot like site, where alot of the hits *are* anonymous?
>
> The reason I started to look at Varnish was a client asked about it in terms of the W3T Cache that is available for Wordpress … it apparently has the ability to 'invalidate cache' in a varnish server, so as to force it to reload from apache … but I don't know enough about Wordpress to know if I can even safely disable cookies, but without disabling cookies, nothing will get cached, making Varnish good as a load balancer, but losing the major benefit …

Hi,

You don't need disable cookies, you could "pass" requests with
specific cookies (wordpress_logged_in and others)

In the other hand, you need a wordpres plugin in order to purge new
comments, posts, etc. Check this one (includes VCL example):

http://wordpress.org/extend/plugins/cd34-varnish-esi/

There is several plugins doing the job, take a look on them.

Hope this helps.
Kind regards.

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

lists at hub

Jul 18, 2012, 1:21 AM

Post #6 of 7 (261 views)
Permalink

Re: Varnish 3.0.2 on FreeBSD 8-STABLE: Constant Restart [In reply to]

Okay, at this point, I think I have to be missing something fairly obvious here, so maybe I'm asking the question wrong …

I'm not even to the point right now of caring about whether or not I have a good hit rate … right now, I'm wondering why it consistently crashes:

www 62495 0.0 0.0 355672036 6204 ?? IJ 6:43AM 0:01.32 varnishd: Varnish-Chld lb1.hospitalityleaders.com (varnishd)
root 74714 0.0 0.0 98032 2364 ?? SsJ Fri04PM 0:08.97 varnishd: Varnish-Mgr lb1.hospitalityleaders.com (varnishd)

If I hit the site with varnishlog running, I see:

0 CLI - Rd ping
0 CLI - Wr 200 19 PONG 1342598723 1.0
0 CLI - Rd ping
0 CLI - Wr 200 19 PONG 1342598726 1.0
0 CLI - Rd ping
0 CLI - Wr 200 19 PONG 1342598729 1.0
0 CLI - Rd ping
0 CLI - Wr 200 19 PONG 1342598732 1.0
0 WorkThread - 0x7fffff5fbbb0 start
0 WorkThread - 0x7fffff3fabb0 start
0 WorkThread - 0x7fffff1f9bb0 start
0 WorkThread - 0x7ffffedf7bb0 start
0 CLI - Rd vcl.load "boot" ./vcl.M1UEoiQY.so
0 WorkThread - 0x7ffffebf6bb0 start
0 CLI - Wr 200 36 Loaded "./vcl.M1UEoiQY.so" as "boot"
0 CLI - Rd vcl.use "boot"
0 CLI - Wr 200 0
0 CLI - Rd start
0 Debug - "Acceptor is kqueue"
0 CLI - Wr 200 0
0 WorkThread - 0x7ffffe5f3bb0 start
0 WorkThread - 0x7ffffe3f2bb0 start
0 WorkThread - 0x7ffffe1f1bb0 start
0 WorkThread - 0x7ffffdff0bb0 start
0 WorkThread - 0x7ffffddefbb0 start
0 CLI - Rd ping
0 CLI - Wr 200 19 PONG 1342598737 1.0
0 CLI - Rd ping
0 CLI - Wr 200 19 PONG 1342598740 1.0

And /var/log/debug.log on the machine shows:

Jul 18 06:43:11 lb1 varnishd[74714]: Child cleanup complete
Jul 18 08:04:23 lb1 varnishd[74714]: Child cleanup complete

Now, I just modified my vcl_recv function so that all it does is 'return (pass);', and it seems to work, so I'm going to step through the code I had, to see at what point it does break …

The code I'm using as my starting point is what is available on github:

https://github.com/mattiasgeniar/varnish-3.0-configuration-templates

I'm not sure what I was expecting, but figured like most code, there was some sort of syntactically checker, or log error message, that would indicate what/where in the config a bug was, not it just crashing :(

As I said, I will go through and see if I can't narrow down where the crash itself is occurring by slowly enabling small chunks of the config …

Thanks ...

On 2012-07-18, at 3:49 AM, Robert Shilston wrote:

> Marc,
>
> We successfully run Varnish in front of a large Wordpress site and achieve a good hit rate. At a very high level, most access is anonymous, and we simple PIPE access if there's wp-admin in the URL. We happen to use a custom plugin to manage submitting bans to Varnish, but I believe off the shelf ones are now available.
>
> However, undertaking putting Varnish in front of any site without a strong understanding of how the site works will not yield the best results. Only by understanding the traffic and requests will you be able to safely get a high degree of caching.
>
>
> Rob
>
> On 18 Jul 2012, at 02:35, Marc Fournier wrote:
>
>>
>> Okay, I'm lost here … what sort of site is Varnish good for … ? A news web site where nobody every logs into the site … ?
>>
>> For instance, the impression I'm getting is that you couldn't use Varnish in front of a Facebook like site, since none of the hits would be anonymous? But would be potentially useful in front of a Slashdot like site, where alot of the hits *are* anonymous?
>>
>> The reason I started to look at Varnish was a client asked about it in terms of the W3T Cache that is available for Wordpress … it apparently has the ability to 'invalidate cache' in a varnish server, so as to force it to reload from apache … but I don't know enough about Wordpress to know if I can even safely disable cookies, but without disabling cookies, nothing will get cached, making Varnish good as a load balancer, but losing the major benefit …
>>
>> … no?
>>
>>
>>
>> On 2012-07-13, at 11:53 PM, Nick Tailor wrote:
>>
>>> Also you only want anonymous visits cached. All cookies with session Id should be passed to apache and then you want look trying to setup should ur backend fail run off the cache only
>>>
>>> Sent from my iPhone
>>>
>>> On Jul 13, 2012, at 10:51 AM, Marc Fournier <lists [at] hub> wrote:
>>>
>>>>
>>>> Morning …
>>>>
>>>> OS: FreeBSD 8.3-STABLE #2: Mon Jul 2 21:20:02 UTC 2012
>>>>
>>>> I am trying to use varnish as a caching front end to a wordpress site … found some wordpress specific templetes for vcl_recv / vcl_fetch, to provide me with a base, and everything starts up, and stays running fine, *until* I change my DNS to have the domain go through Varnish … and then it starts rebooting …
>>>>
>>>> This test through, before I switched DNS, I had an uptime (according to varnish stat) of just over 1 hr … as soon as I switched DNS over, uptime kept resetting to 0, with (after switching DNS back again), I'm currently seeing:
>>>>
>>>> 0+00:07:44
>>>> Hitrate ratio: 0 0 0
>>>> Hitrate avg: 0.0000 0.0000 0.0000
>>>>
>>>> And, according to ps, you can see the Child much younger then the Mgr:
>>>>
>>>> root [at] lb:/var/log # ps aux | grep varnish
>>>> www 52281 0.0 0.0 355674084 6432 ?? IJ 5:34PM 0:00.16 varnishd: Varnish-Chld lb1.hospitalityleaders.com (varnishd)
>>>> root 74714 0.0 0.0 98032 2636 ?? SsJ 4:21PM 0:00.13 varnishd: Varnish-Mgr lb1.hospitalityleaders.com (varnishd)
>>>> root 71856 0.0 0.0 540 328 2 S+J 5:44PM 0:00.00 grep varnish
>>>>
>>>> I'm new with this, so not really sure how best to debug … I did see something from PHK in one of my searches talking about an issue with cookies … ? But I haven't been able to re-find that posting …
>>>>
>>>> I've also, so far, been unable to find a core file, although I'm currently running a find across the whole file system to see if there isn't one …
>>>>
>>>> Any suggestions on where to start debugging this, or more information I can provide, would be appreciated …
>>>>
>>>> Thank you ...
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc [at] varnish-cache
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc [at] varnish-cache
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

apj at mutt

Jul 18, 2012, 2:06 AM

Post #7 of 7 (255 views)
Permalink

Re: Varnish 3.0.2 on FreeBSD 8-STABLE: Constant Restart [In reply to]

On Wed, Jul 18, 2012 at 04:21:59AM -0400, Marc Fournier wrote:
>
> I'm not sure what I was expecting, but figured like most code, there was some
> sort of syntactically checker, or log error message, that would indicate
> what/where in the config a bug was, not it just crashing :(

There is, but obviously it has missing something in your config.
Crashes will both be syslogged and registered by the master process so you can
view them using "panic.show" from varnishadmin.

--
Andreas

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads