Mailing List Archive: Varnish: Misc

max_connections, backend_busy, and queueing requests

Index | Next | Previous | View Threaded

jpotter-varnish at codepuppy

May 22, 2012, 8:05 AM

Post #1 of 4 (367 views)
Permalink

max_connections, backend_busy, and queueing requests

Hi List,

I'm having a hard time finding documentation or other users who've done what I'm trying to do, although it seems pretty straight-forward and I must be missing something (or have something misconfigured?).

Let's say that I have a simple setup of 1 varnish server and 5 backend nodes defined in a single director; nothing fancy other than each backend's max_connections being set to 100. This setup would allow for 500 concurrent requests when everything is healthy.

As soon as there are 500 concurrent requests, varnish kicks any additional requests to vcl_error and we send out a 5xx error for the requests. And varnishstats increments the backend_busy/"Backend conn. too many" counter.

How do I get Varnish to queue up the requests instead, so that when a backend worker connection becomes available it gets sent to a backend? (Currently running Varnish 2; planning to migrate to Varnish 3 shortly. If the answer is "upgrade to varnish 3", that's useful to know too!)

It may help to know that, in our case, we expect most of the requests to go to the backend -- i.e. while we do use cacheing, most cacheable requests have been pushed off to other hosts. If the only way to do this is configure varnish such that it only handles 500 frontend requests at a time, that would be okay (but not ideal).

Thanks!
-Jeff

P.S. Thanks for a great piece of software -- Varnish is pretty amazing.
_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

perbu at varnish-software

May 22, 2012, 9:14 AM

Post #2 of 4 (341 views)
Permalink

Re: max_connections, backend_busy, and queueing requests [In reply to]

Hi Jeff.

On Tue, May 22, 2012 at 5:05 PM, Jeff Potter
<jpotter-varnish [at] codepuppy>wrote:

>
> As soon as there are 500 concurrent requests, varnish kicks any additional
> requests to vcl_error and we send out a 5xx error for the requests. And
> varnishstats increments the backend_busy/"Backend conn. too many" counter.
>
> How do I get Varnish to queue up the requests instead, so that when a
> backend worker connection becomes available it gets sent to a backend?

Varnish, in it's current state doesn't do this, I'm afraid. Since Varnish
uses one thread, and thus has some overhead, for each connection it
wouldn't be ideal to have them idly wait. The current trunk code uses
separate threads to fetch content from the backend so having some sort of
queue there shouldn't be to hard. I don't know of any plans to implement
this, however.

--
Per Buer
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
*Varnish makes websites fly!*
Whitepapers <http://www.varnish-software.com/whitepapers> |
Video<http://www.youtube.com/watch?v=x7t2Sp174eI> |
Twitter <https://twitter.com/varnishsoftware>

jpotter-varnish at codepuppy

May 23, 2012, 2:39 PM

Post #3 of 4 (341 views)
Permalink

Re: max_connections, backend_busy, and queueing requests [In reply to]

Hi Per,

>> As soon as there are 500 concurrent requests, varnish kicks any additional requests to vcl_error and we send out a 5xx error for the requests. And varnishstats increments the backend_busy/"Backend conn. too many" counter. How do I get Varnish to queue up the requests instead, so that when a backend worker connection becomes available it gets sent to a backend?
>
> Varnish, in it's current state doesn't do this, I'm afraid. Since Varnish uses one thread, and thus has some overhead, for each connection it wouldn't be ideal to have them idly wait. The current trunk code uses separate threads to fetch content from the backend so having some sort of queue there shouldn't be to hard. I don't know of any plans to implement this, however.

Thanks for the reply. If I were to implement this and submit it, would Varnish be willing to accept the patch into the mainline? And, presumably an implementation would want two config variables: the max number of requests to allow on the queue and the max time allowed on queue until vcl_error is triggered?

Best,
Jeff

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

perbu at varnish-software

May 24, 2012, 12:36 AM

Post #4 of 4 (350 views)
Permalink

Re: max_connections, backend_busy, and queueing requests [In reply to]

On Wed, May 23, 2012 at 11:39 PM, Jeff Potter <jpotter-varnish [at] codepuppy
> wrote:

>
>
> Thanks for the reply. If I were to implement this and submit it, would
> Varnish be willing to accept the patch into the mainline? And, presumably
> an implementation would want two config variables: the max number of
> requests to allow on the queue and the max time allowed on queue until
> vcl_error is triggered?
>

For me, at least, this sounds feasible as long as the implementation is
clean and fits into trunk. It must pass General Muster, aka PHK. See
https://www.varnish-cache.org/trac/wiki/Contributing for details.

--
Per Buer, CEO
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
*Varnish makes websites fly!*
Whitepapers <http://www.varnish-software.com/whitepapers> |
Video<http://www.youtube.com/watch?v=x7t2Sp174eI> |
Twitter <https://twitter.com/varnishsoftware>

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads