Mailing List Archive: Varnish: Bugs

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.

Index | Next | Previous | View Threaded

varnish-bugs at varnish-cache

Jun 20, 2011, 12:08 PM

Post #1 of 15 (592 views)
Permalink

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.
-------------------+--------------------------------------------------------
Reporter: david | Type: defect
Status: new | Priority: high
Milestone: | Component: build
Version: 3.0.0 | Severity: critical
Keywords: |
-------------------+--------------------------------------------------------
Hello,

With the introduction of gzip support in Varnish 3, we have run into a
problem where ETags are incorrectly duplicated between two different
objects. I'm not sure I need to provide much explanation here except to
remind the devs that ETags are unique, and need to be different for each
variant of a page (including a variant caused by gzip). You can read more
about it in an old bug report here:
https://issues.apache.org/bugzilla/show_bug.cgi?id=39727

The only workaround I could come up with was to revert to storing two
copies of a page in cache (2.1.5 behavior) by turning off Varnish's gzip
support, and putting the Accept-Encoding normalization code back into
vcl_recv.

If there is another workaround, please let me know.

Regards,
-david

--
Ticket URL: <http://www.varnish-cache.org/trac/ticket/940>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

varnish-bugs at varnish-cache

Jun 20, 2011, 12:26 PM

Post #2 of 15 (574 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.
-------------------+--------------------------------------------------------
Reporter: david | Type: defect
Status: new | Priority: high
Milestone: | Component: build
Version: 3.0.0 | Severity: critical
Keywords: |
-------------------+--------------------------------------------------------

Comment(by phk):

Just when I thought HTTP couldn't become more crappy as a protocol to
transfer data.

As I read the apache discussion there are two possible workarounds and no
solutions:

1) twist the etag and make it weak: you can do this in vcl_deliver if
there is a transfer-encoding header.

2) Delete the etag entirely. Ditto.

I'll need to think more about this, but right now I don't see what else we
can do.

--
Ticket URL: <http://www.varnish-cache.org/trac/ticket/940#comment:1>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

varnish-bugs at varnish-cache

Jun 20, 2011, 12:30 PM

Post #3 of 15 (582 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.
-------------------+--------------------------------------------------------
Reporter: david | Type: defect
Status: new | Priority: high
Milestone: | Component: build
Version: 3.0.0 | Severity: critical
Keywords: |
-------------------+--------------------------------------------------------

Comment(by david):

Hi phk,

Thanks for the ideas. Changing the etag for gzipped variants in
vcl_deliver was the first thing I tried. It neutered my hit ratio because
all of the if-modified-since and if-none-match headers started to not
work.

My thought, of how you can fix this in Varnish is to store a second etag
header for the gunzipped copy - Varnish will probably need to fabricate
this tag, perhaps just adding something like "nogz" to the etag you have
stored now would be sufficient. The problem comes in how to look it up
when you get one of those if-none-match headers. Good luck.

Thanks again,
-david

--
Ticket URL: <http://varnish-cache.org/trac/ticket/940#comment:2>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

varnish-bugs at varnish-cache

Jun 20, 2011, 12:52 PM

Post #4 of 15 (574 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

varnish-bugs at varnish-cache

Jun 20, 2011, 12:58 PM

Post #5 of 15 (575 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.
-------------------+--------------------------------------------------------
Reporter: david | Type: defect
Status: new | Priority: high
Milestone: | Component: build
Version: 3.0.0 | Severity: critical
Keywords: |
-------------------+--------------------------------------------------------

Comment(by david):

oh! I didn't even think about changing it back in vcl_recv. I'll try that.

You used the word inverse, and I read it as reverse, and thought "huh,
that's an interesting idea, reverse the same etag for gzip variant" - then
I realized that's not what you were saying. Maybe it is an idea you can
think about, to tag a gzip-vs-unzip variant. Just reverse the etag for one
of them. :)

--
Ticket URL: <http://www.varnish-cache.org/trac/ticket/940#comment:4>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

varnish-bugs at varnish-cache

Aug 29, 2011, 3:50 AM

Post #6 of 15 (510 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

varnish-bugs at varnish-cache

Aug 29, 2011, 12:04 PM

Post #7 of 15 (514 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.
-------------------+--------------------------------------------------------
Reporter: david | Type: defect
Status: new | Priority: high
Milestone: | Component: build
Version: 3.0.0 | Severity: critical
Keywords: |
-------------------+--------------------------------------------------------

Comment(by david):

Hey Tollef,

I modified my VCL to change the etag for gz content on the way out and
back in, as was discussed on this ticket some time ago. Has the code
been patched to hand out varying etags now? I have the 3.0.1 RC, but I
cannot get it to compile to test it. I sent in my compiler errors on
another (support) ticket.

Regards,
-david

--
Ticket URL: <http://www.varnish-cache.org/trac/ticket/940#comment:6>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

varnish-bugs at varnish-cache

Sep 1, 2011, 1:05 AM

Post #8 of 15 (503 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

varnish-bugs at varnish-cache

Sep 1, 2011, 12:23 PM

Post #9 of 15 (506 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.
-------------------+--------------------------------------------------------
Reporter: david | Type: defect
Status: new | Priority: high
Milestone: | Component: build
Version: 3.0.0 | Severity: critical
Keywords: |
-------------------+--------------------------------------------------------

Comment(by david):

phk,

My entire website became problematic for many of my users after installing
Varnish 3 because of this ETag issue. We had hundreds of support requests
on our support board complaining of "Content encoding" errors. If you
intend to leave this flaky behavior as the default, I would recommend
changing the default VCL to work-around the bug. Expecting users to find
this ticket and come up with their own solution doesn't sit well with me.
Ultimately, it's your decision to serve conflicting etags or not.

--
Ticket URL: <http://www.varnish-cache.org/trac/ticket/940#comment:8>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

varnish-bugs at varnish-cache

Sep 3, 2012, 3:34 AM

Post #10 of 15 (313 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

varnish-bugs at varnish-cache

Oct 6, 2012, 8:38 AM

Post #11 of 15 (256 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

varnish-bugs at varnish-cache

Oct 6, 2012, 8:47 AM

Post #12 of 15 (257 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

varnish-bugs at varnish-cache

Jan 29, 2013, 6:30 AM

Post #13 of 15 (142 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.
----------------------+---------------------
Reporter: david | Owner: martin
Type: defect | Status: new
Priority: high | Milestone:
Component: build | Version: 3.0.0
Severity: critical | Resolution:
Keywords: |
----------------------+---------------------

Comment (by slink):

Replying to [comment:2 david]:

> Thanks for the ideas. Changing the etag for gzipped variants in
vcl_deliver was the first thing I tried. It neutered my hit ratio because
all of the if-modified-since and if-none-match headers started to not
work.

@david: I have written down some thoughts in [wiki:ETags] and I wonder if
you had tried changing the ETag for '''un'''gzipped variants to `W/`eak or
removing it, but leaving it as it is for gzipped.

In practice, most clients should `Accept-Encoding: gzip` so we definitely
want Etags to allow for
*
INM `Range` for this case, but I believe removing Etags for un-gzipped or
making them weak should not make that big a difference.

@anyone else: I'd appreciate feedback on [wiki:ETags]

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/940#comment:12>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

varnish-bugs at varnish-cache

Jan 29, 2013, 1:16 PM

Post #14 of 15 (141 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

#940: ETag for gzip'd variant identical to ETag of ungzipped variant.
----------------------+---------------------
Reporter: david | Owner: martin
Type: defect | Status: new
Priority: high | Milestone:
Component: build | Version: 3.0.0
Severity: critical | Resolution:
Keywords: |
----------------------+---------------------

Comment (by david):

Hi slink,

It's been a year and a half since I wrote the code to handle this problem.
I just added/removed the prefix on the way out/in. Honestly, I think that
should be part of the default VCL because this will eventually bite
someone else. Tollef has a fresh copy if my VCL if you want to take a look
at how I did it. Suggestions warmly welcome.

As for removing the ETag for ungzipped docs, well, I may try that in the
future, but for now that code seems just fine. I have other problems to
tackle. :)

Thank you and regards,
-david

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/940#comment:13>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

varnish-bugs at varnish-cache

Jan 29, 2013, 1:20 PM

Post #15 of 15 (141 views)
Permalink

Re: #940: ETag for gzip'd variant identical to ETag of ungzipped variant. [In reply to]

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads