Mailing List Archive: Varnish: Bugs

#1090: Latency for streaming using byte range requests

Index | Next | Previous | View Threaded

varnish-bugs at varnish-cache

Feb 6, 2012, 1:49 AM

Post #1 of 2 (70 views)
Permalink

#1090: Latency for streaming using byte range requests

#1090: Latency for streaming using byte range requests
--------------------+-------------------------------------------------------
Reporter: helmer | Type: enhancement
Status: new | Priority: normal
Milestone: | Component: varnishd
Version: 3.0.2 | Severity: normal
Keywords: |
--------------------+-------------------------------------------------------
This ticket regards the streaming support, the version used is from the
streaming branch, 3.0.2-streaming.2 (Dec 8).

We have been looking at using the new streaming support in Varnish to
accelerate Widevine content. A Widevine player gets content roughly in the
following fashion:

The player first makes a regular http GET for the file, reads the first
(approx) 100k bytes, closes the connection and then makes a new http
request, specifying a byte range starting somewhere in the file and with
an open ending.

To mimic this we tried:
{{{
> curl -o /dev/null -s -m 1 http://test_server/blob.wvm
}}}
where the media file blob.wvm is about 650 MB (the complete file won't be
transferred to Varnish within the 1 second limit used by
curl, -m 1), and then:
{{{
> curl -o /dev/null -s -H 'Range: bytes=1234567-'
http://test_server/blob.wvm
}}}
This results in two GET to the origin server for the complete file. Since
Varnish asks the origin for the complete file, users will experience a
(sometimes considerable) latency when the file is not in cache. For this
use case support for byte range requests to the origin would be of great
value (to our understanding not implemented, suggested enhancement).

Information about our environment: CentoS 5.5, we compiled the source
"plain vanilla" with gcc 4.1.2. The default.vcl was used with streaming
enabled:
{{{
sub vcl_fetch {
set beresp.do_stream = true;
}
}}}
We started varnishd with
{{{
/usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -s
malloc,256M -s persistent,/var/varnish/varnish.cache,2048M -p
default_ttl=7200 -T 127.0.0.1:2000 -a 0.0.0.0:8080
}}}

I added Martin as cc, since he has been working on this.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1090>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

varnish-bugs at varnish-cache

Feb 6, 2012, 3:59 AM

Post #2 of 2 (70 views)
Permalink

Re: #1090: Latency for streaming using byte range requests [In reply to]

#1090: Latency for streaming using byte range requests
----------------------+-----------------------------------------------------
Reporter: helmer | Type: enhancement
Status: closed | Priority: normal
Milestone: | Component: varnishd
Version: 3.0.2 | Severity: normal
Resolution: invalid | Keywords:
----------------------+-----------------------------------------------------
Changes (by martin):

* status: new => closed
* resolution: => invalid

Comment:

In Varnish 3.0.2s, when a range request is made, the do_stream flag is
turned off. So the request becomes just like a normal range request
without any streaming. That is probably the cause of the latency you are
experiencing, where the entire object will have to be fetched into Varnish
before the range request can be delivered to the client.

When the streaming support is released in the main branch of Varnish, we
plan on having range support for the requests also when doing streaming
delivery. This will then mean that the requesting client will have a
streaming delivery of the requested range as the bytes of this range
becomes available in the cache. The backend(/origin) request will still be
for the full object.

There is a patch available for the Varnish 3.0S that was posted on the
varnish-dev mailing list some time ago by Thomas Souvignet that implements
this mode as well.

Your requested feature of byte range requests against the origin server is
not something that is being worked on, as this would require support for
partial objects which Varnish doesn't have. So Varnish will continue to
request the full object from the origin also when streaming.

As this is a feature request and we only use trac for bugs, I will close
this ticket.

Regards,
Martin Blix Grydeland

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1090#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads