Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Varnish: Dev

VUG5 IMS presentation

 

 

Varnish dev RSS feed   Index | Next | Previous | View Threaded


slink at schokola

Mar 23, 2012, 6:35 AM

Post #1 of 7 (585 views)
Permalink
VUG5 IMS presentation

Hi,

I've been asked to mail it to this list.

Thanks to everyone at VUG5 dev day for the good discussion.

Nils
Attachments: VUG5_IMS_uplex.pdf (77.6 KB)


sky at crucially

Mar 23, 2012, 7:51 AM

Post #2 of 7 (572 views)
Permalink
Re: VUG5 IMS presentation [In reply to]

I will re-iterate, I have no idea why this is so complex and no one has been able to explain it.

If we have last-modified from in a grace object, add if-modified-since. If you don't like it, remove if-modified-since in vcl_miss. This gives you control.

If you get a 304 and no headers have changed (most likely) then you don't need to copy the object, just update timestamps.

Go through vcl_fetch to allow people to change the vcl or introduce a new function for that. (the one problem of going through vcl_fetch is people might try to set headers unnecessarily.)

The patch to this is literally tiny, combined with pure background 304 re-validation I think it is 200 lines, and that includes using background threads.

Artur
Sent via BlackBerry by AT&T

-----Original Message-----
From: Nils Goroll <slink [at] schokola>
Sender: varnish-dev-bounces [at] varnish-cache
Date: Fri, 23 Mar 2012 14:35:05
To: <varnish-dev [at] varnish-cache>
Subject: VUG5 IMS presentation

Hi,

I've been asked to mail it to this list.

Thanks to everyone at VUG5 dev day for the good discussion.

Nils

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev


slink at schokola

Mar 23, 2012, 8:09 AM

Post #3 of 7 (575 views)
Permalink
Re: VUG5 IMS presentation [In reply to]

Hi Sky and all,

I really should have sent a brief note on what was discussed at VUG5 today:

All the slides on VCL access to the stale_obj etc. were are really only meant as
a basis for the discussion we had. My understanding of the consensus is that IMS
feature should first go in as a minimal solution: In vcl_miss, you could drop
the If-* headers if you wanted to disable IMS, otherwise it would just be
implicitly enabled. obj in vcl_fetch would be the new object created from the
304 response with any missing headers copied from the stale object.

If needed, the stale object could be made accessible through a vmod and if we
found actual use cases for this in real world projects later, we could still
agree on inclusion of access functions in VCL.

> If you get a 304 and no headers have changed (most likely) then you don't need to copy the object, just update timestamps.

It is important to update Expires/Cache-Control to "extend" the lifetime of the
refreshed object (
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5 ). We must not
change a live object header because it is not protected (by locks). So we really
need to have a new object and dup (either reference or copy) the body.

Nils

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev


sky at crucially

Mar 23, 2012, 8:11 AM

Post #4 of 7 (574 views)
Permalink
Re: VUG5 IMS presentation [In reply to]

Only if people change the cache-control. Expires could be turned into a uint64 and safely updated, though expires is generally frowned upon.

Artur
------Original Message------
From: Nils Goroll
To: Artur Bergman
Cc: varnish-dev [at] varnish-cache
Subject: Re: VUG5 IMS presentation
Sent: Mar 23, 2012 08:09

Hi Sky and all,

I really should have sent a brief note on what was discussed at VUG5 today:

All the slides on VCL access to the stale_obj etc. were are really only meant as
a basis for the discussion we had. My understanding of the consensus is that IMS
feature should first go in as a minimal solution: In vcl_miss, you could drop
the If-* headers if you wanted to disable IMS, otherwise it would just be
implicitly enabled. obj in vcl_fetch would be the new object created from the
304 response with any missing headers copied from the stale object.

If needed, the stale object could be made accessible through a vmod and if we
found actual use cases for this in real world projects later, we could still
agree on inclusion of access functions in VCL.

> If you get a 304 and no headers have changed (most likely) then you don't need to copy the object, just update timestamps.

It is important to update Expires/Cache-Control to "extend" the lifetime of the
refreshed object (
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5 ). We must not
change a live object header because it is not protected (by locks). So we really
need to have a new object and dup (either reference or copy) the body.

Nils

Sent via BlackBerry by AT&T
_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev


aotto at mosso

Mar 23, 2012, 8:48 AM

Post #5 of 7 (573 views)
Permalink
Re: VUG5 IMS presentation [In reply to]

Agreed, but if you change Cache-Control, and an Expires exists on the obj, you must also change that, or the client may not cache the entity because of a past dated Expires header. The trouble with Expires is that it states an absolute point in time for the expiry, versus Cache-Control with is treated relative to the Date: of the delivery.

One solution I have used is to not actually change the Expires header in the cached object (no no issues with locking), but instead dynamically rewrite the Expires header upon delivery in accordance with the difference between the generated Date: header and the max-age parameter from Cache-Control. This way the Expires and Cache-Control headers always agree.

Adrian

On Mar 23, 2012, at 8:11 AM, sky [at] crucially wrote:

> Only if people change the cache-control. Expires could be turned into a uint64 and safely updated, though expires is generally frowned upon.
>
> Artur
> ------Original Message------
> From: Nils Goroll
> To: Artur Bergman
> Cc: varnish-dev [at] varnish-cache
> Subject: Re: VUG5 IMS presentation
> Sent: Mar 23, 2012 08:09
>
> Hi Sky and all,
>
> I really should have sent a brief note on what was discussed at VUG5 today:
>
> All the slides on VCL access to the stale_obj etc. were are really only meant as
> a basis for the discussion we had. My understanding of the consensus is that IMS
> feature should first go in as a minimal solution: In vcl_miss, you could drop
> the If-* headers if you wanted to disable IMS, otherwise it would just be
> implicitly enabled. obj in vcl_fetch would be the new object created from the
> 304 response with any missing headers copied from the stale object.
>
> If needed, the stale object could be made accessible through a vmod and if we
> found actual use cases for this in real world projects later, we could still
> agree on inclusion of access functions in VCL.
>
>> If you get a 304 and no headers have changed (most likely) then you don't need to copy the object, just update timestamps.
>
> It is important to update Expires/Cache-Control to "extend" the lifetime of the
> refreshed object (
> http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5 ). We must not
> change a live object header because it is not protected (by locks). So we really
> need to have a new object and dup (either reference or copy) the body.
>
> Nils
>
> Sent via BlackBerry by AT&T
> _______________________________________________
> varnish-dev mailing list
> varnish-dev [at] varnish-cache
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev


_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev


geoff at uplex

Mar 26, 2012, 1:32 AM

Post #6 of 7 (556 views)
Permalink
Re: VUG5 IMS presentation [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 3/23/12 4:11 PM, sky [at] crucially wrote:
> Only if people change the cache-control. Expires could be turned
> into a uint64 and safely updated, though expires is generally
> frowned upon.

To be sure I understand your argument, are you saying that in the
"almost-always" case, headers from the 304 response do not in fact
change anything about the refreshed object, so we could continue using
it without any of the copying or refcounting? Or that some information
about the object could be safely updated, despite the general rule
that objects in cache are never modified? The quotation above sounds
something like the latter, but if I read you right: convert Expires to
a uint64, which can be updated in the cached object, and convert it
back to text form on delivery.

It seems worthwhile to check for the case where the 304 response
doesn't change anything -- then we could continue to use the old
object, update its TTL etc., and discard the new object created from
the beresp.

But I'd be cautious about anything that involves updating the old
object, and that's the general answer to your question -- we don't
want to break the rule that objects in cache are not modified.

Your questions about this make sense, on first blush you wouldn't
expect validating requests to get involved in things like storage and
stevedores. If the common case can avoid that, I'm all for it,
although we'd still need the capability for the unusual case, since
304 responses could contain any headers at all.

But we can't do this in a way that breaks the concurrency model, that
would cause disruptions that would be far worse and certainly not
worth it.


Best,
Geoff
- --
UPLEX Systemoptimierung
Schwanenwik 24
22087 Hamburg
http://uplex.de/
Mob: +49-176-63690917
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJPcCmSAAoJEOUwvh9pJNURlScP/3hNxrkjSjP5jwgRPxlXKVOb
Ro9lL2TMfX2qHcKGVLzvB/0uBqZgW9Iy4jqUydZRZlMdUXEMsVWfQQyBx7I1xy0t
e9UcAveFyeXcsGkXfVcTDDTi09CrCDbbxanhpm4kaGTY3jjjqNh5EfQByYwwI8D7
MzdiKJ0k5tZtRlIjbhAkJ8XfDZkUhJzXA7ypV6tHnB8/jDv3K8Fj+ihTsIbGrAh+
PpSR5vpt9wi2b8S5Ujz/0BvZEW7pW+OpF0BXs4e8/dSIqEu1OOSwl7AnN964qjKf
qK4UOSCNCZRJ8qGfn/PuGO8YnRBXtoXqbtGU1h2Kjhnk7sEUCQ5Ys3b1ytV0ckId
WMKH8pT+fq1/08FNnQBj8VQyLAcfsqnFj38q/3h9wzUMx+V0b60eUXhRazv0yXOZ
hAhEUR/CvxXaHoK3gBnbdR4FC0BHU5Ut8oUeYtpxbonjePVQINa34Ck2Sljxm3ay
J0hECr5g2new59e3QT0H66dKG9RkBUkJPsASpi3weGMtPpMSDSFxLNJZ0sYD231C
PfsdoCLQ4/qI9tOKSqXXSD/eBwkrW1awrA3ROfeeSIn8QUXYftUywfzamShYMpVw
qBtKyVxFXw3Ey1ylE9mgd0QCWQloFvsbcPa8Z03Rwboo0SawKvpEjNv5uVizNqR8
uxzbZPO61RIuq5X5dxcj
=6Dto
-----END PGP SIGNATURE-----

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev


slink at schokola

Mar 27, 2012, 10:23 PM

Post #7 of 7 (551 views)
Permalink
Re: VUG5 IMS presentation [In reply to]

Hi,

On 03/23/12 04:48 PM, Adrian Otto wrote:
> versus Cache-Control with is treated relative to the Date: of the delivery.

Actually Cache-Control max-age is defined not only relative to Date:, but also
relative to Age:. So if that is set appropriately, also max-age specifies an
absolute point in time.

What this means for IMS is that we really need to update headers for the general
case, but Sky has rightly pointed out that we could further optimize for the
case that no header changes, which might be the only case for some setups (but
not for all).

Nils

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

Varnish dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.