Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Varnish: Bugs

#792: Bandwidth management / rate-limiting

 

 

Varnish bugs RSS feed   Index | Next | Previous | View Threaded


varnish-bugs at varnish-cache

Oct 7, 2010, 2:21 AM

Post #1 of 7 (1094 views)
Permalink
#792: Bandwidth management / rate-limiting

#792: Bandwidth management / rate-limiting
-------------------------+--------------------------------------------------
Reporter: tmagnien | Owner: phk
Type: enhancement | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: trunk
Severity: normal | Keywords: bandwidth rate-limit
-------------------------+--------------------------------------------------
Hi,

Here is a patch to allow for rate-limiting on specific objects / url /
etc. It uses vmod and looks like this in the VCL :


{{{
sub vcl_recv {
if ( req.url ~ "^/bigone.ts$" ) {
std.set_bandwidth(20000);
}
}
}}}


This will limit the delivery at 20000 bytes/sec.

The idea behind this (eh, who would like to slow down delivery with a web-
accelerator ?) is the following : mobile users watching a video usually
watch the first N seconds and stop. But as they start watching, the device
downloads at full speed in order to fill up its cache. The result is that
a lot of downloaded data are never used, so why wouldn't we limit the
download rate to the rate at which the video is encoded ? This allows to
decrease the overall data transfer, which is not a bad idea on mobile
networks.

Note that this is just a "testcase" patch : the "sleep(1)" would better be
something more precise but that's the main idea.

All remarks welcome.


Thanks,


Thierry

--
Ticket URL: <http://varnish-cache.org/trac/ticket/792>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
http://lists.varnish-cache.org/mailman/listinfo/varnish-bugs


varnish-bugs at varnish-cache

Oct 7, 2010, 4:29 AM

Post #2 of 7 (1058 views)
Permalink
Re: #792: Bandwidth management / rate-limiting [In reply to]

#792: Bandwidth management / rate-limiting
-------------------------+--------------------------------------------------
Reporter: tmagnien | Owner: phk
Type: enhancement | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: trunk
Severity: normal | Keywords: bandwidth rate-limit
-------------------------+--------------------------------------------------

Comment(by tmagnien):

After rereading, I think I should use TIM_real() to get timestamp before
and after write operation to take into account the time used for sending
data, e.g. : sleep(1 - write_duration). This would allow for a more
precise bandwidth management.

--
Ticket URL: <http://varnish-cache.org/trac/ticket/792#comment:1>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
http://lists.varnish-cache.org/mailman/listinfo/varnish-bugs


thebog at gmail

Oct 7, 2010, 11:25 AM

Post #3 of 7 (1064 views)
Permalink
Re: #792: Bandwidth management / rate-limiting [In reply to]

Another, equally important, use for this is protecting against "attacks".

I remember once we had a proxy against our site that did prefetch
(yes, somebody has written such a bad boy, and yes, they are called
M$).
They used 100 Mbit/s filling thousands of articles that their cache
thought might get read again :)

Ab

On Thu, Oct 7, 2010 at 1:29 PM, Varnish <varnish-bugs [at] varnish-cache> wrote:
> #792: Bandwidth management / rate-limiting
> -------------------------+--------------------------------------------------
>  Reporter:  tmagnien     |       Owner:  phk
>     Type:  enhancement  |      Status:  new
>  Priority:  normal       |   Milestone:
> Component:  varnishd     |     Version:  trunk
>  Severity:  normal       |    Keywords:  bandwidth rate-limit
> -------------------------+--------------------------------------------------
>
> Comment(by tmagnien):
>
>  After rereading, I think I should use TIM_real() to get timestamp before
>  and after write operation to take into account the time used for sending
>  data, e.g. : sleep(1 - write_duration). This would allow for a more
>  precise bandwidth management.
>
> --
> Ticket URL: <http://varnish-cache.org/trac/ticket/792#comment:1>
> Varnish <http://varnish-cache.org/>
> The Varnish HTTP Accelerator
>
> _______________________________________________
> varnish-bugs mailing list
> varnish-bugs [at] varnish-cache
> http://lists.varnish-cache.org/mailman/listinfo/varnish-bugs
>

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
http://lists.varnish-cache.org/mailman/listinfo/varnish-bugs


varnish-bugs at varnish-cache

Mar 7, 2011, 4:24 AM

Post #4 of 7 (907 views)
Permalink
Re: #792: Bandwidth management / rate-limiting [In reply to]

#792: Bandwidth management / rate-limiting
-------------------------+--------------------------------------------------
Reporter: tmagnien | Owner: phk
Type: enhancement | Status: new
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Keywords: bandwidth rate-limit
-------------------------+--------------------------------------------------
Changes (by kristian):

* milestone: => Later


Old description:

> Hi,
>
> Here is a patch to allow for rate-limiting on specific objects / url /
> etc. It uses vmod and looks like this in the VCL :
>

> {{{
> sub vcl_recv {
> if ( req.url ~ "^/bigone.ts$" ) {
> std.set_bandwidth(20000);
> }
> }
> }}}
>

> This will limit the delivery at 20000 bytes/sec.
>
> The idea behind this (eh, who would like to slow down delivery with a
> web-accelerator ?) is the following : mobile users watching a video
> usually watch the first N seconds and stop. But as they start watching,
> the device downloads at full speed in order to fill up its cache. The
> result is that a lot of downloaded data are never used, so why wouldn't
> we limit the download rate to the rate at which the video is encoded ?
> This allows to decrease the overall data transfer, which is not a bad
> idea on mobile networks.
>
> Note that this is just a "testcase" patch : the "sleep(1)" would better
> be something more precise but that's the main idea.
>
> All remarks welcome.
>

> Thanks,
>

> Thierry

New description:

Hi,

Here is a patch to allow for rate-limiting on specific objects / url /
etc. It uses vmod and looks like this in the VCL :


{{{
sub vcl_recv {
if ( req.url ~ "^/bigone.ts$" ) {
std.set_bandwidth(20000);
}
}
}}}


This will limit the delivery at 20000 bytes/sec.

The idea behind this (eh, who would like to slow down delivery with a web-
accelerator ?) is the following : mobile users watching a video usually
watch the first N seconds and stop. But as they start watching, the device
downloads at full speed in order to fill up its cache. The result is that
a lot of downloaded data are never used, so why wouldn't we limit the
download rate to the rate at which the video is encoded ? This allows to
decrease the overall data transfer, which is not a bad idea on mobile
networks.

Note that this is just a "testcase" patch : the "sleep(1)" would better be
something more precise but that's the main idea.

All remarks welcome.


Thanks,


Thierry

--

Comment:

Sorry for the response-time. We've discussed this back and forth several
times, but apparently never got back to you, please accept our apologies.

It doesn't look like we'll add this for Varnish 3.0, and given the nature
of vmods, it might be a more suitable place in the future, but for now,
I'm just moving this to 'later'.

--
Ticket URL: <http://varnish-cache.org/trac/ticket/792#comment:2>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs


varnish-bugs at varnish-cache

Aug 15, 2011, 4:25 AM

Post #5 of 7 (820 views)
Permalink
Re: #792: Bandwidth management / rate-limiting [In reply to]

#792: Bandwidth management / rate-limiting
----------------------------------+-----------------------------------------
Reporter: tmagnien | Owner: phk
Type: enhancement | Status: closed
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution: invalid
Keywords: bandwidth rate-limit |
----------------------------------+-----------------------------------------
Changes (by phk):

* status: new => closed
* resolution: => invalid


Comment:

We revisted the oldest tickets on the bug-wash today, and finally(!)
made our mind up about this one:

I'm closing this ticket as invalid, because it is a feature request.

I have put a back-link to it from wiki:Future_Feature, please read long
explanation there.

Sorry about taking so long before dealing with this ticket.

--
Ticket URL: <http://www.varnish-cache.org/trac/ticket/792#comment:3>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs


varnish-bugs at varnish-cache

Nov 27, 2012, 2:12 AM

Post #6 of 7 (552 views)
Permalink
Re: #792: Bandwidth management / rate-limiting [In reply to]

#792: Bandwidth management / rate-limiting
----------------------------------+----------------------
Reporter: tmagnien | Owner: phk
Type: enhancement | Status: closed
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution: invalid
Keywords: bandwidth rate-limit |
----------------------------------+----------------------

Comment (by ruben):

Replying to [comment:3 phk]:
> We revisted the oldest tickets on the bug-wash today, and finally(!)
> made our mind up about this one:
>
> I'm closing this ticket as invalid, because it is a feature request.
>
> I have put a back-link to it from wiki:Future_Feature, please read long
explanation there.

The correct link for this is wiki:Future_VMODS#RateLimiting

>
> Sorry about taking so long before dealing with this ticket.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/792#comment:4>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs


varnish-bugs at varnish-cache

Apr 22, 2014, 2:16 PM

Post #7 of 7 (214 views)
Permalink
Re: #792: Bandwidth management / rate-limiting [In reply to]

#792: Bandwidth management / rate-limiting
----------------------------------+----------------------
Reporter: tmagnien | Owner: phk
Type: enhancement | Status: closed
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution: invalid
Keywords: bandwidth rate-limit |
----------------------------------+----------------------

Comment (by keretamalam):

I am about to leave and cannot look into this further right now, but I do
want to mention that relying on pkg-config itself is not a problem (as you
said, we also use it later), the problem is that on some systems, the .pc
file for pkg-config is supplying wrong information (whereas on yours, it's
botan-config that is wrong), so that's why I'm trying to make it work with
both (while one of them is bad).
[http://maskodoq.blogspot.com/2014/03/unit-link-terbaik-di-indonesia.html
Unit Link Terbaik di Indonesia Commonwealth Life Investra Link]
[http://maskodoq.blogspot.com/2013/07/CiptoJunaedy.html Cipto Junaedy]
[http://etnisjawa.blogspot.com/2013/08/apakah-cipto-junaedy-bohong.html
Cipto Junaedy]

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/792#comment:5>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

Varnish bugs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.