Mailing List Archive: Varnish: Dev

Synthetic objects

Index | Next | Previous | View Threaded

phk at phk

Jun 27, 2013, 3:28 AM

Post #1 of 8 (134 views)
Permalink

Synthetic objects

I think I have made up my mind about how to do synthetic objects in
V4, but I'd like to run the idea past you guys first.

In vcl_backend_(fetch|response){} you do:

return (synth(777));

Then you end up in vcl_backend_synth{} where, like in vcl_error{}, you
can flesh out the object with headers and a body.

The object will have a TTL of zero by default, but you can change that
if you want to cache the object (unless it's a pass of course).

Good or bad idea ?

Poul-Henning

PS: Yes, I do realize that together with std.fileread() this makes
Varnish a webserver, but I trust^H^H^H^H^Hhope that people will
restrain themselves

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk [at] FreeBSD | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

varnish at bsdchicks

Jun 27, 2013, 12:32 PM

Post #2 of 8 (121 views)
Permalink

Re: Synthetic objects [In reply to]

If we're changing things, it would be useful to have something in synth
other than a 3 digit status code.

I wouldn't mind a 64-bit (or 32-bit if you must) argument to synth(),
either instead of the status code, or in addition to.

This would give a little more headroom for choices of synthetic bodies.

On Thu, Jun 27, 2013 at 3:28 AM, Poul-Henning Kamp <phk [at] phk>wrote:

>
> I think I have made up my mind about how to do synthetic objects in
> V4, but I'd like to run the idea past you guys first.
>
> In vcl_backend_(fetch|response){} you do:
>
> return (synth(777));
>
> Then you end up in vcl_backend_synth{} where, like in vcl_error{}, you
> can flesh out the object with headers and a body.
>
> The object will have a TTL of zero by default, but you can change that
> if you want to cache the object (unless it's a pass of course).
>
> Good or bad idea ?
>
> Poul-Henning
>
> PS: Yes, I do realize that together with std.fileread() this makes
> Varnish a webserver, but I trust^H^H^H^H^Hhope that people will
> restrain themselves
>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk [at] FreeBSD | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev [at] varnish-cache
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>
>

phk at phk

Jul 3, 2013, 5:39 AM

Post #3 of 8 (113 views)
Permalink

Re: Synthetic objects [In reply to]

In message <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw [at] mail>
, "Rogier 'DocWilco' Mulhuijzen" writes:

>If we're changing things, it would be useful to have something in synth
>other than a 3 digit status code.

We came up with an idea for this in the IRC channel:

We'll allow any number of digits (up to 32bits) but only the last
three will be used for the HTTP status code.

So you could for instance do
return (synth(10001404));

and that would become a 404, but you get extra bits for your
own purposes.

Also, please notice that you have bereq to look at.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk [at] FreeBSD | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

dridi.boukelmoune at zenika

Jul 3, 2013, 6:38 AM

Post #4 of 8 (111 views)
Permalink

Re: Synthetic objects [In reply to]

Hi,

I'm not sure I understand the purpose, is it for a separations of:
- true errors (from backends/varnish or hand made)
- on-the-fly static responses (redirects, static files...)

In this case it could be interesting to jump to vcl_backend_synth from vcl_recv:
if (req.url ~ "...") {
return(synth(42302));
}

The syntax looks inconsistent, I'd rather have something like:
return(transition [, args])

Examples:
return(lookup);
return(synth, 10001404);
return(error, 10001503);
return(error, 10001503, "message");

It would also help to have constants for magic status codes:
===
// outside of sub functions
const BACKEND_DOWN = 503;
const MAINTENANCE = 1503;

sub vcl_backend_response {
if (beresp looks maintenance-ish) {
return(error, MAINTENANCE);
}
}

sub vcl_error {
if (obj.status == BACKEND_DOWN) {
synthetic "dammit";
}
if (obj.status == MAINTENANCE) {
synthetic std.fileread("maintenance.html");
}
}
===

About Varnish turning into a web server, there's nothing to worry
about with just that, synthetic+std.fileread can't handle most
binary files.

Best Regards,
Dridi

On Wed, Jul 3, 2013 at 2:39 PM, Poul-Henning Kamp <phk [at] phk> wrote:
> In message <CANTTouWnQnqtL6N5z=roCe-zxju=wdtOie-a7xwkmo1=CijKtw [at] mail>
> , "Rogier 'DocWilco' Mulhuijzen" writes:
>
>>If we're changing things, it would be useful to have something in synth
>>other than a 3 digit status code.
>
> We came up with an idea for this in the IRC channel:
>
> We'll allow any number of digits (up to 32bits) but only the last
> three will be used for the HTTP status code.
>
> So you could for instance do
> return (synth(10001404));
>
> and that would become a 404, but you get extra bits for your
> own purposes.
>
> Also, please notice that you have bereq to look at.
>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk [at] FreeBSD | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev [at] varnish-cache
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

phk at phk

Jul 3, 2013, 8:14 AM

Post #5 of 8 (113 views)
Permalink

Re: Synthetic objects [In reply to]

In message <CABtDKm6zNsQE2Pb=nvJErKxYqwSwvkCo6vhjwGzH8OHZtEDABQ [at] mail>
, Dridi Boukelmoune writes:

>I'm not sure I understand the purpose, is it for a separations of:
>- true errors (from backends/varnish or hand made)
>- on-the-fly static responses (redirects, static files...)

Until now vcl_error{} has lived in sort of a no-mans-land between
the client side and the backend side of varnish.

What I'm proposing to do in VCL4 is to split it into
vcl_error{} = client side
and
vcl_backend_synth{} = backend side.

So from vcl_recv{} you'd go to vcl_error{} to deliver a one-off
synthetic response to a single client. That object cannot and will
not be cached.

vcl_backend_synth{} on the other hand, will allow you to create objects
which _can_ be cached, although by default the will probably not be.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk [at] FreeBSD | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

varnish at bsdchicks

Jul 3, 2013, 8:32 AM

Post #6 of 8 (111 views)
Permalink

Re: Synthetic objects [In reply to]

(Oops, think I forgot to hit reply all on my previous email)

In addition to do_esi, what about do_gzip. Especially with ESI in the mix,
compression would be nice to have. But even without ESI, things can get
pretty big. Having synthetics of 100K is not unheard of, with inline images
and such.
On Jul 3, 2013 8:26 AM, "Poul-Henning Kamp" <phk [at] phk> wrote:

> In message <
> CANTTouUPmhbMi4eqq77g-UOFKADagYqxNmdmTVsOTrvsHEp0QQ [at] mail>
> , "Rogier 'DocWilco' Mulhuijzen" writes:
>
> >Can the backend synthetics get the esi flag set? It would be useful to be
> >able to generate synthetic pages with esi includes in them.
>
> Ohh... Neat idea!
>
> I won't promise this in the first iteration, but we should certainy not
> forget that idea...
>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk [at] FreeBSD | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
>

phk at phk

Jul 3, 2013, 8:49 AM

Post #7 of 8 (112 views)
Permalink

Re: Synthetic objects [In reply to]

In message <CANTTouX+q9PT-vnNQYpDJwm=6YpsnJLAQ8FD5ZgWAnsMfUy4hw [at] mail>
, "Rogier 'DocWilco' Mulhuijzen" writes:

>In addition to do_esi, what about do_gzip. Especially with ESI in the mix,
>compression would be nice to have. But even without ESI, things can get
>pretty big. Having synthetics of 100K is not unheard of, with inline images
>and such.

Hand the devil a finger... :-)

I'll add it to the list

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk [at] FreeBSD | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-dev mailing list
varnish-dev [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

varnish at bsdchicks

Jul 3, 2013, 8:50 AM

Post #8 of 8 (111 views)
Permalink

Re: Synthetic objects [In reply to]

I prefer daemon over devil...
On Jul 3, 2013 8:49 AM, "Poul-Henning Kamp" <phk [at] phk> wrote:

> In message <CANTTouX+q9PT-vnNQYpDJwm=
> 6YpsnJLAQ8FD5ZgWAnsMfUy4hw [at] mail>
> , "Rogier 'DocWilco' Mulhuijzen" writes:
>
> >In addition to do_esi, what about do_gzip. Especially with ESI in the mix,
> >compression would be nice to have. But even without ESI, things can get
> >pretty big. Having synthetics of 100K is not unheard of, with inline
> images
> >and such.
>
> Hand the devil a finger... :-)
>
> I'll add it to the list
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk [at] FreeBSD | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
>

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads