Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Apache: Dev

DoS with mod_deflate & range requests

 

 

First page Previous page 1 2 3 Next page Last page  View All Apache dev RSS feed   Index | Next | Previous | View Threaded


sf at sfritsch

Aug 23, 2011, 4:08 AM

Post #1 of 56 (22194 views)
Permalink
DoS with mod_deflate & range requests

http://seclists.org/fulldisclosure/2011/Aug/175

I haven't looked into it so far. And I am not sure I will have time today.


ruediger.pluem at vodafone

Aug 23, 2011, 5:11 AM

Post #2 of 56 (21846 views)
Permalink
RE: DoS with mod_deflate & range requests [In reply to]

> -----Original Message-----
> From: Stefan Fritsch [mailto:sf [at] sfritsch]
> Sent: Dienstag, 23. August 2011 13:09
> To: dev [at] httpd
> Subject: DoS with mod_deflate & range requests
>
> http://seclists.org/fulldisclosure/2011/Aug/175
>
> I haven't looked into it so far. And I am not sure I will
> have time today.


After checking the attack script and the code this has IMHO
nothing to do with mod_deflate but only with the byterange filter.

But I admit that haven't run the script to check.

The host is seen as vulnerable if it replies to a range request that requests
the whole entity via a range "0-" with a partial response.
A possible problem is that the output bucket brigade gets transformed
in a "one bucket per byte" brigade and thus into a brigade with many
buckets. Futhermore the created range response has a lot of buckets
with boundaries, strings allocated from r->pool.
So it might be advisable if we limit the number of ranges we accept
contained in a Range header.
As a further optimization we could check for "0-" ranges and once we
hit one just reply with the full response instead of a partial response.

Regards

Rüdiger


lazy404 at gmail

Aug 23, 2011, 5:15 AM

Post #3 of 56 (21831 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

2011/8/23 Stefan Fritsch <sf [at] sfritsch>:
> http://seclists.org/fulldisclosure/2011/Aug/175
>
> I haven't looked into it so far. And I am not sure I will have time today.
>

it is sending HEAD requests with lots of ranges
HEAD / HTTP/1.1
Host: xxxx
Range:bytes=0-,5-1,5-2,5-3,.....

the code in
ap_byterange_filter()
http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c
creates a bucket for every range element,

the number of buckets is limited by the size of the document in
published code but I think it can be enchanced by
using 1-2,1-3,..1-doc_size,2-1,2-2, 2-doc_size

doeas Range in HEAD request have any sense at all ?


lazy404 at gmail

Aug 23, 2011, 6:56 AM

Post #4 of 56 (21844 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

2011/8/23 Lazy <lazy404 [at] gmail>:
> 2011/8/23 Stefan Fritsch <sf [at] sfritsch>:
>> http://seclists.org/fulldisclosure/2011/Aug/175
>>
>> I haven't looked into it so far. And I am not sure I will have time today.
>>
>
> it is sending HEAD requests with lots of  ranges
> HEAD / HTTP/1.1
> Host: xxxx
> Range:bytes=0-,5-1,5-2,5-3,.....
>
> the code in
> ap_byterange_filter()
> http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c
> creates a bucket for every range element,
>
> the number of buckets is limited by the size of the document in
> published code but I think it can be enchanced by
> using 1-2,1-3,..1-doc_size,2-1,2-2, 2-doc_size
>
> doeas Range in HEAD request have any sense at all ?

quick fix bellow made it immune to this dos

diff -ru modules/http/byterange_filter.c.org
modules/http/byterange_filter.c |less
--- byterange_filter.c 2011-02-13 15:32:19.000000000 +0100
+++ modules/http/byterange_filter.c 2011-08-23 15:54:37.000000000 +0200
@@ -320,6 +320,7 @@
const char *if_range;
const char *match;
const char *ct;
+ char * tmp;
int num_ranges;

if (r->assbackwards) {
@@ -373,14 +374,13 @@
}
}

- if (!ap_strchr_c(range, ',')) {
- /* a single range */
- num_ranges = 1;
- }
- else {
- /* a multiple range */
- num_ranges = 2;
- }
+ /* count ranges, exit if more then 10 */
+ tmp=range+6;
+ num_ranges=1;
+ while(*++tmp)
+ if(*tmp == ',')
+ if(++num_ranges > 10)
+ return 0;

r->status = HTTP_PARTIAL_CONTENT;
r->range = range + 6;


isoma at jellybaby

Aug 23, 2011, 7:00 AM

Post #5 of 56 (21831 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On Tue, Aug 23, 2011 at 02:15:16PM +0200, Lazy wrote:
> 2011/8/23 Stefan Fritsch <sf [at] sfritsch>:
> > http://seclists.org/fulldisclosure/2011/Aug/175
> >
> > I haven't looked into it so far. And I am not sure I will have time today.
> >
>
> it is sending HEAD requests with lots of ranges
> HEAD / HTTP/1.1
> Host: xxxx
> Range:bytes=0-,5-1,5-2,5-3,.....
…
> doeas Range in HEAD request have any sense at all ?

One /possible/ use is as an equivalent for a conditional GET, ie
GET / HTTP/1.1
Host: xxx
Range: bytes=1024-
If-Range: "foo"

…to which the correct response should I think be either 200 or 206 depending
on whether the document is modified.

But it's a pretty odd case. I can't imagine any published client or proxy
that would make such a request. It would in any case be acceptable to
return a 200 response instead; RFC 2616 states that "A server MAY ignore
the Range header"

Tim Bannister


wrowe at rowe-clan

Aug 23, 2011, 11:18 AM

Post #6 of 56 (21837 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On 8/23/2011 6:08 AM, Stefan Fritsch wrote:
> http://seclists.org/fulldisclosure/2011/Aug/175
>
> I haven't looked into it so far. And I am not sure I will have time today.

Until range can be completely addressed, avoiding excessive numbers of
ranges (tricky) or overlapping ranges (pretty straightforward)...
what about simply disabling deflate on range requests?


sf at sfritsch

Aug 23, 2011, 11:49 AM

Post #7 of 56 (21841 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On Tuesday 23 August 2011, William A. Rowe Jr. wrote:
> On 8/23/2011 6:08 AM, Stefan Fritsch wrote:
> > http://seclists.org/fulldisclosure/2011/Aug/175
> >
> > I haven't looked into it so far. And I am not sure I will have
> > time today.
>
> Until range can be completely addressed, avoiding excessive numbers
> of ranges (tricky) or overlapping ranges (pretty
> straightforward)... what about simply disabling deflate on range
> requests?

There is this PR:

https://issues.apache.org/bugzilla/show_bug.cgi?id=49772

"mod_deflate kicks itself out on Content-Range responses but not on
multipart/byteranges"

This may be one of the issues. But as Rüdiger pointed out, there is
also an issue without mod_deflate.

From looking at the code, I think the problem is the bucket structs.
With N the number of requested ranges, the initial brigade is
partitioned into 2*N buckets at the maximum. Then those buckets are
copied into the output brigade N times, which means that O(N^2)
buckets are created. The data is not copied, and only N "A-B" strings
are allocated from the pool. But the sum of those is limited by
LimitRequestFieldSize, so it shouldn't be a problem.

Maybe the byte-range filter should call ap_pass_brigade every 10
ranges or so? Then the buckets should be freed earlier (at least if
all filters down the chain behave correctly).


mohameddawaina at gmail

Aug 23, 2011, 11:52 AM

Post #8 of 56 (21832 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

please tell me how to unsubscribe from this mailing list

On Tue, Aug 23, 2011 at 9:49 PM, Stefan Fritsch <sf [at] sfritsch> wrote:

> On Tuesday 23 August 2011, William A. Rowe Jr. wrote:
> > On 8/23/2011 6:08 AM, Stefan Fritsch wrote:
> > > http://seclists.org/fulldisclosure/2011/Aug/175
> > >
> > > I haven't looked into it so far. And I am not sure I will have
> > > time today.
> >
> > Until range can be completely addressed, avoiding excessive numbers
> > of ranges (tricky) or overlapping ranges (pretty
> > straightforward)... what about simply disabling deflate on range
> > requests?
>
> There is this PR:
>
> https://issues.apache.org/bugzilla/show_bug.cgi?id=49772
>
> "mod_deflate kicks itself out on Content-Range responses but not on
> multipart/byteranges"
>
> This may be one of the issues. But as Rüdiger pointed out, there is
> also an issue without mod_deflate.
>
> From looking at the code, I think the problem is the bucket structs.
> With N the number of requested ranges, the initial brigade is
> partitioned into 2*N buckets at the maximum. Then those buckets are
> copied into the output brigade N times, which means that O(N^2)
> buckets are created. The data is not copied, and only N "A-B" strings
> are allocated from the pool. But the sum of those is limited by
> LimitRequestFieldSize, so it shouldn't be a problem.
>
> Maybe the byte-range filter should call ap_pass_brigade every 10
> ranges or so? Then the buckets should be freed earlier (at least if
> all filters down the chain behave correctly).
>
>


--
Mohamed D. Sulaiman
Farmer's Commercial Bank
Banking Technology Unit
Cell: (+249) 923407600
Website: http://www.mohameddawaina.site40.net


wrowe at rowe-clan

Aug 23, 2011, 12:32 PM

Post #9 of 56 (21829 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On 8/23/2011 1:49 PM, Stefan Fritsch wrote:
>
> From looking at the code, I think the problem is the bucket structs.
> With N the number of requested ranges, the initial brigade is
> partitioned into 2*N buckets at the maximum. Then those buckets are
> copied into the output brigade N times, which means that O(N^2)
> buckets are created. The data is not copied, and only N "A-B" strings
> are allocated from the pool. But the sum of those is limited by
> LimitRequestFieldSize, so it shouldn't be a problem.
>
> Maybe the byte-range filter should call ap_pass_brigade every 10
> ranges or so? Then the buckets should be freed earlier (at least if
> all filters down the chain behave correctly).

I suggest we should be parsing and reassembling the list before we
start the bucket logic. I'd also suggest the following...

This example from the spec...

- Several legal but not canonical specifications of the second 500
bytes (byte offsets 500-999, inclusive):
bytes=500-600,601-999
bytes=500-700,601-999

does not say the last is 200 bytes and 400 bytes, but is explicitly the
second 500 bytes.

I propose we satisfy range requests in the only sensible manner, returning
the ranges in sequence, using a linked list of buckets and combining all
ranges or another mechanism to work out the applicable ranges.

The range processing is limited to some 4000 parts (consisting entirely
of invalid -, segments), and as a practical matter much less than 2500.
Reassemble the list of ranges in sequence as a pre-parsing step, and we
can much more efficiently generate the response with no duplication.

The spec is ambiguous but nowhere suggested that duplicate ranges would
be legitimate.


ames.greg at gmail

Aug 23, 2011, 2:00 PM

Post #10 of 56 (21829 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On Tue, Aug 23, 2011 at 3:32 PM, William A. Rowe Jr. <wrowe [at] rowe-clan>wrote:

>
> I suggest we should be parsing and reassembling the list before we
> start the bucket logic.
>


> I propose we satisfy range requests in the only sensible manner, returning
> the ranges in sequence,
>

yeah, overlapping ranges should be merged up front. That ought to completely
fix the issue.

fwiw, I played with the script to create the attack and noticed this:

Content-Length: 950883

That's probably the sum of the overlapping ranges. My original file is 3929
bytes long.

Greg


wrowe at rowe-clan

Aug 23, 2011, 2:34 PM

Post #11 of 56 (21832 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On 8/23/2011 4:00 PM, Greg Ames wrote:
>
> On Tue, Aug 23, 2011 at 3:32 PM, William A. Rowe Jr. wrote:
>
> I suggest we should be parsing and reassembling the list before we
> start the bucket logic.
>
> I propose we satisfy range requests in the only sensible manner, returning
> the ranges in sequence,
>
> yeah, overlapping ranges should be merged up front. That ought to completely fix the issue.

So the only remaining question; are we free to reorder them into sequence?

Even in the most pedantic case, I believe that the total array shouldn't
ever exceed 1024, because in those cases a large number of the acceptable
expected ranges should be in the nnn-nnn, format, or 8 characters long,
out of our MAX_LINE_LENGTH of some 8190. If we argue that asking for
single bytes is simply wrong, we should probably allocate some 16 ranges
and grow the list by a power of four, resulting in a max of some 4 allocs
and maximum memory consumption of less than 64k per request.


h.reindl at thelounge

Aug 23, 2011, 3:12 PM

Post #12 of 56 (21819 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

Am 23.08.2011 20:52, schrieb Mohamed Dawaina:
> please tell me how to unsubscribe from this mailing list

what about looking in the mail-header or login with your acount
you have registered?

Precedence: bulk
Reply-To: dev [at] httpd
list-help: <mailto:dev-help [at] httpd>
list-unsubscribe: <mailto:dev-unsubscribe [at] httpd>
List-Post: <mailto:dev [at] httpd>
List-Id: <dev.httpd.apache.org>
Attachments: signature.asc (0.26 KB)


sf at sfritsch

Aug 23, 2011, 3:28 PM

Post #13 of 56 (21811 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On Tuesday 23 August 2011, William A. Rowe Jr. wrote:
> On 8/23/2011 4:00 PM, Greg Ames wrote:
> > On Tue, Aug 23, 2011 at 3:32 PM, William A. Rowe Jr. wrote:
> > I suggest we should be parsing and reassembling the list
> > before we start the bucket logic.
> >
> > I propose we satisfy range requests in the only sensible
> > manner, returning the ranges in sequence,
> >
> > yeah, overlapping ranges should be merged up front. That ought to
> > completely fix the issue.
>
> So the only remaining question; are we free to reorder them into
> sequence?

Good point. I haven't seen anything in the RFC about that. I guess
that there are at least some clients that will be broken by that.

Nevertheless, I have done a first try at a patch. The necessary
modification to only merge and not reorder should be small. I have
done only limited testing, so there are probably some bugs. There are
no tests with multiple ranges in the test-framework, yet.

> Even in the most pedantic case, I believe that the total array
> shouldn't ever exceed 1024, because in those cases a large number
> of the acceptable expected ranges should be in the nnn-nnn,
> format, or 8 characters long, out of our MAX_LINE_LENGTH of some
> 8190. If we argue that asking for single bytes is simply wrong,
> we should probably allocate some 16 ranges and grow the list by a
> power of four, resulting in a max of some 4 allocs and maximum
> memory consumption of less than 64k per request.

Just counting the commas in the header line seems acceptable to me. In
any case, single byte ranges are explicitly mentioned in the RFC as
example, so we probably should not disallow those.
Attachments: merge_ranges.diff (9.37 KB)


fielding at gbiv

Aug 23, 2011, 6:34 PM

Post #14 of 56 (21801 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On Aug 23, 2011, at 2:34 PM, William A. Rowe Jr. wrote:

> On 8/23/2011 4:00 PM, Greg Ames wrote:
>>
>> On Tue, Aug 23, 2011 at 3:32 PM, William A. Rowe Jr. wrote:
>>
>> I suggest we should be parsing and reassembling the list before we
>> start the bucket logic.
>>
>> I propose we satisfy range requests in the only sensible manner, returning
>> the ranges in sequence,
>>
>> yeah, overlapping ranges should be merged up front. That ought to completely fix the issue.
>
> So the only remaining question; are we free to reorder them into sequence?

And the spec says ...

When a client requests multiple ranges in one request, the
server SHOULD return them in the order that they appeared in the
request.

My suggestion is to reject any request with overlapping ranges or more
than five ranges with a 416, and to send 200 for any request with 4-5
ranges. There is simply no need to support random access in HTTP.

....Roy


sf at sfritsch

Aug 23, 2011, 11:38 PM

Post #15 of 56 (21785 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On Wednesday 24 August 2011, Roy T. Fielding wrote:
> My suggestion is to reject any request with overlapping ranges or
> more than five ranges with a 416, and to send 200 for any request
> with 4-5 ranges. There is simply no need to support random access
> in HTTP.

Even with WebDAV?

BTW, I think Adobe reader does range requests to download single pages
from a pdf. It would be interesting to see what kind of requests it
does, but I don't have it installed right now.


ruediger.pluem at vodafone

Aug 24, 2011, 1:05 AM

Post #16 of 56 (21780 views)
Permalink
RE: DoS with mod_deflate & range requests [In reply to]

> -----Original Message-----
> From: Stefan Fritsch
> Sent: Mittwoch, 24. August 2011 00:28
> To: dev [at] httpd
> Subject: Re: DoS with mod_deflate & range requests
>
> On Tuesday 23 August 2011, William A. Rowe Jr. wrote:
> > On 8/23/2011 4:00 PM, Greg Ames wrote:
> > > On Tue, Aug 23, 2011 at 3:32 PM, William A. Rowe Jr. wrote:
> > > I suggest we should be parsing and reassembling the list
> > > before we start the bucket logic.
> > >
> > > I propose we satisfy range requests in the only sensible
> > > manner, returning the ranges in sequence,
> > >
> > > yeah, overlapping ranges should be merged up front. That ought to
> > > completely fix the issue.
> >
> > So the only remaining question; are we free to reorder them into
> > sequence?
>
> Good point. I haven't seen anything in the RFC about that. I guess
> that there are at least some clients that will be broken by that.
>
> Nevertheless, I have done a first try at a patch. The necessary
> modification to only merge and not reorder should be small. I have
> done only limited testing, so there are probably some bugs. There are
> no tests with multiple ranges in the test-framework, yet.
>
> > Even in the most pedantic case, I believe that the total array
> > shouldn't ever exceed 1024, because in those cases a large number
> > of the acceptable expected ranges should be in the nnn-nnn,
> > format, or 8 characters long, out of our MAX_LINE_LENGTH of some
> > 8190. If we argue that asking for single bytes is simply wrong,
> > we should probably allocate some 16 ranges and grow the list by a
> > power of four, resulting in a max of some 4 allocs and maximum
> > memory consumption of less than 64k per request.
>
> Just counting the commas in the header line seems acceptable
> to me. In
> any case, single byte ranges are explicitly mentioned in the RFC as
> example, so we probably should not disallow those.

Patch looks good, but some comments:

As far as I can see the following range request would not get merged:

Range: bytes=0-0,1-1,2-2

into a 0-2 range as need_sort would remain 0. OTOH

Range: bytes=0-0,0-1,1-2

would get get merged into a 0-2 range.

Using boundary and !boundary in the later if statements to decide whether a request
is multi range or single range is IMHO bad as boundary is set based on the old number
ranges and not based on the number of merged ranges. So multiple ranges in the beginning
might get merged to a single range in the end.

Regards

Rüdiger


isoma at jellybaby

Aug 24, 2011, 8:35 AM

Post #17 of 56 (21770 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On Tue, Aug 23, 2011, Roy T. Fielding wrote:
> And the spec says ...
>
> When a client requests multiple ranges in one request, the
> server SHOULD return them in the order that they appeared in the
> request.
>
> My suggestion is to reject any request with overlapping ranges or more
> than five ranges with a 416, and to send 200 for any request with 4-5
> ranges. There is simply no need to support random access in HTTP.

Deshpande & Zeng in http://dx.doi.org/10.1145/500141.500197 describe a
method for "streaming" JPEG 2000 documents over HTTP, using many more than
5 ranges in a single request.
A client that knows about any server-side limit could make multiple
requests each with a small number of ranges, but discovering that limit
will add latency and take more code.

Tim Bannister


jim at jaguNET

Aug 24, 2011, 8:41 AM

Post #18 of 56 (21743 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On Aug 23, 2011, at 9:34 PM, Roy T. Fielding wrote:

> On Aug 23, 2011, at 2:34 PM, William A. Rowe Jr. wrote:
>
>> On 8/23/2011 4:00 PM, Greg Ames wrote:
>>>
>>> On Tue, Aug 23, 2011 at 3:32 PM, William A. Rowe Jr. wrote:
>>>
>>> I suggest we should be parsing and reassembling the list before we
>>> start the bucket logic.
>>>
>>> I propose we satisfy range requests in the only sensible manner, returning
>>> the ranges in sequence,
>>>
>>> yeah, overlapping ranges should be merged up front. That ought to completely fix the issue.
>>
>> So the only remaining question; are we free to reorder them into sequence?
>
> And the spec says ...
>
> When a client requests multiple ranges in one request, the
> server SHOULD return them in the order that they appeared in the
> request.
>
> My suggestion is to reject any request with overlapping ranges

+1

> or more
> than five ranges with a 416, and to send 200 for any request with 4-5
> ranges. There is simply no need to support random access in HTTP.

-0


Dirk-Willem.van.Gulik at bbc

Aug 24, 2011, 8:46 AM

Post #19 of 56 (21755 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On 24 Aug 2011, at 16:35, Tim Bannister wrote:

> On Tue, Aug 23, 2011, Roy T. Fielding wrote:
>> And the spec says ...
>>
>> When a client requests multiple ranges in one request, the
>> server SHOULD return them in the order that they appeared in the
>> request.
>>
>> My suggestion is to reject any request with overlapping ranges or more
>> than five ranges with a 416, and to send 200 for any request with 4-5
>> ranges. There is simply no need to support random access in HTTP.
>
> Deshpande & Zeng in http://dx.doi.org/10.1145/500141.500197 describe a
> method for "streaming" JPEG 2000 documents over HTTP, using many more than
> 5 ranges in a single request.
> A client that knows about any server-side limit could make multiple
> requests each with a small number of ranges, but discovering that limit
> will add latency and take more code.

Agreed - I've seen many 10's of ranges in a single request for things like clever PDF pagination (or tiny TIFF quicklooks for the pages), clever http fake streaming and clever use of jumping to i-Frames.

I think we just need to sit this out - and accept up to RequestFieldSize limit bytes on that line - and then do a sort & merge overlaps as needed.

And then it is fine.

Dw


jim at jaguNET

Aug 24, 2011, 8:48 AM

Post #20 of 56 (21768 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On Aug 24, 2011, at 4:05 AM, Plüm, Rüdiger, VF-Group wrote:

>
> Patch looks good, but some comments:
>
> As far as I can see the following range request would not get merged:
>
> Range: bytes=0-0,1-1,2-2
>
> into a 0-2 range as need_sort would remain 0. OTOH
>
> Range: bytes=0-0,0-1,1-2
>
> would get get merged into a 0-2 range.
>
> Using boundary and !boundary in the later if statements to decide whether a request
> is multi range or single range is IMHO bad as boundary is set based on the old number
> ranges and not based on the number of merged ranges. So multiple ranges in the beginning
> might get merged to a single range in the end.

+1…

Suggestion: Let's fold the patch, as-is, into trunk, tune it there
and then backport to 2.x...


ruediger.pluem at vodafone

Aug 24, 2011, 8:55 AM

Post #21 of 56 (21761 views)
Permalink
RE: DoS with mod_deflate & range requests [In reply to]

> -----Original Message-----
> From: Dirk-Willem van Gulik [mailto:Dirk-Willem.van.Gulik [at] bbc]
> Sent: Mittwoch, 24. August 2011 17:46
> To: dev [at] httpd
> Subject: Re: DoS with mod_deflate & range requests
>
>
> On 24 Aug 2011, at 16:35, Tim Bannister wrote:
>
> > On Tue, Aug 23, 2011, Roy T. Fielding wrote:
> >> And the spec says ...
> >>
> >> When a client requests multiple ranges in one request, the
> >> server SHOULD return them in the order that they appeared in the
> >> request.
> >>
> >> My suggestion is to reject any request with overlapping
> ranges or more
> >> than five ranges with a 416, and to send 200 for any
> request with 4-5
> >> ranges. There is simply no need to support random access in HTTP.
> >
> > Deshpande & Zeng in http://dx.doi.org/10.1145/500141.500197
> describe a
> > method for "streaming" JPEG 2000 documents over HTTP, using
> many more than
> > 5 ranges in a single request.
> > A client that knows about any server-side limit could make multiple
> > requests each with a small number of ranges, but
> discovering that limit
> > will add latency and take more code.
>
> Agreed - I've seen many 10's of ranges in a single request
> for things like clever PDF pagination (or tiny TIFF
> quicklooks for the pages), clever http fake streaming and
> clever use of jumping to i-Frames.
>
> I think we just need to sit this out - and accept up to
> RequestFieldSize limit bytes on that line - and then do a
> sort & merge overlaps as needed.

Hm. If I got it right what Roy says above about the spec sorting and merging is
not an option as we need to stick to the order and number of ranges the client
requested. But we can deny overlapping with a 416.
Or we do a 416 as well if merging would change something.

Regards

Rüdiger


jim at jaguNET

Aug 24, 2011, 9:01 AM

Post #22 of 56 (21751 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

Sorting isn't allowed but I get the impression that merging is OK…
Roy can confirm…

If not, then some sort of runtime limit on the number of allowable
ranges plus a 416 w/ overlapping ranges makes the most sense.

On Aug 24, 2011, at 11:55 AM, Plüm, Rüdiger, VF-Group wrote:
>
> Hm. If I got it right what Roy says above about the spec sorting and merging is
> not an option as we need to stick to the order and number of ranges the client
> requested. But we can deny overlapping with a 416.
> Or we do a 416 as well if merging would change something.
>
> Regards
>
> Rüdiger
>


ruediger.pluem at vodafone

Aug 24, 2011, 9:02 AM

Post #23 of 56 (21755 views)
Permalink
RE: DoS with mod_deflate & range requests [In reply to]

> -----Original Message-----
> From: Jim Jagielski [mailto:jim [at] jaguNET]
> Sent: Mittwoch, 24. August 2011 17:48
> To: dev [at] httpd
> Subject: Re: DoS with mod_deflate & range requests
>
>
> On Aug 24, 2011, at 4:05 AM, Plüm, Rüdiger, VF-Group wrote:
>
> >
> > Patch looks good, but some comments:
> >
> > As far as I can see the following range request would not
> get merged:
> >
> > Range: bytes=0-0,1-1,2-2
> >
> > into a 0-2 range as need_sort would remain 0. OTOH
> >
> > Range: bytes=0-0,0-1,1-2
> >
> > would get get merged into a 0-2 range.
> >
> > Using boundary and !boundary in the later if statements to
> decide whether a request
> > is multi range or single range is IMHO bad as boundary is
> set based on the old number
> > ranges and not based on the number of merged ranges. So
> multiple ranges in the beginning
> > might get merged to a single range in the end.
>
> +1...
>
> Suggestion: Let's fold the patch, as-is, into trunk, tune it there
> and then backport to 2.x...
>

Based on Roy's comment about the spec I think we cannot optimize this way.
I think we can only detect if something weird goes on (overlapping, merging
would result in smaller number of ranges, excessive number of ranges, whereas
"excessive" needs to be configurable with a sane default) and reply with a 416 then.

Regards

Rüdiger


ruediger.pluem at vodafone

Aug 24, 2011, 9:05 AM

Post #24 of 56 (21750 views)
Permalink
RE: DoS with mod_deflate & range requests [In reply to]

> -----Original Message-----
> From: Jim Jagielski [mailto:jim [at] jaguNET]
> Sent: Mittwoch, 24. August 2011 18:02
> To: dev [at] httpd
> Subject: Re: DoS with mod_deflate & range requests
>
> Sorting isn't allowed but I get the impression that merging is OK...
> Roy can confirm...

But merging might require sorting...

>
> If not, then some sort of runtime limit on the number of allowable
> ranges plus a 416 w/ overlapping ranges makes the most sense.
>
> On Aug 24, 2011, at 11:55 AM, Plüm, Rüdiger, VF-Group wrote:
> >
> > Hm. If I got it right what Roy says above about the spec
> sorting and merging is
> > not an option as we need to stick to the order and number
> of ranges the client
> > requested. But we can deny overlapping with a 416.
> > Or we do a 416 as well if merging would change something.
> >
> > Regards
> >
> > Rüdiger
> >
>
>

Regards

Rüdiger


wrowe at rowe-clan

Aug 24, 2011, 9:22 AM

Post #25 of 56 (21767 views)
Permalink
Re: DoS with mod_deflate & range requests [In reply to]

On 8/24/2011 10:55 AM, "Plüm, Rüdiger, VF-Group" wrote:
>
> Hm. If I got it right what Roy says above about the spec sorting and merging is
> not an option as we need to stick to the order and number of ranges the client
> requested.

No. Merging -is- recommended.



10.4.17 416 Requested Range Not Satisfiable

A server SHOULD return a response with this status code if a request
included a Range request-header field (section 14.35), and none of
the range-specifier values in this field overlap the current extent
of the selected resource, and the request did not include an If-Range
request-header field. (For byte-ranges, this means that the first-
byte-pos of all of the byte-range-spec values were greater than the
current length of the selected resource.)

Note; the FIRST byte-pos is invalid. It doesn't suggest that 416 can be
used for semantically valid ranges longer than the document length, or
to represent that valid byte ranges overlapped. In fact, if ONE range
can be satisfied, 416 is not appropriate.

So if ONE range can be satisfied, it is to be returned (this is reiterated
in 14.16), which may obviously be out-of-sequence.

I would suggest we simply ignore/extend for all overlapping ranges
rather than rejecting them.

E.g.

0-, 40-50 becomes 0-
0-499, 400-599 becomes 0-599
1000-1075, 200-250, 1051-1100 becomes 1000-1100, 200-250

-Sorting- is to be avoided, although as its a SHOULD, I am happy to break
that recommendation if there is no reasonably efficient solution to the
server side. As Roy suggests, random access on the server side is very
dubious. A client issuing rich requests should assemble them and must
respect the individual Content-Range response headers.

First page Previous page 1 2 3 Next page Last page  View All Apache dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.