Mailing List Archive: Varnish: Misc

Question about srcbody in VCL_conf

Index | Next | Previous | View Threaded

gordon.schmidt at unister-gmbh

Jul 2, 2012, 4:10 AM

Post #1 of 7 (282 views)
Permalink

Question about srcbody in VCL_conf

Dear Varnish-Team,

while trying to optimize varnish performance for my company i've
noticed, that removing all comments from the vcls used in our
configuration tribles the requests per seconds from an easy ab test with
100% cache-misses. On an ab test with 100% cache-hits the improvement
was about 8-10%. I was a little bit shocked, because i've read that the
vcls are compiled to c and execution shouldn't depend on the amount of
comments.

I took a deeper look and found out, that the compiler appends a char
array of the vcl source (srcbody) to the VCL_conf. Which operations need
this data?

If these operations are not crucial for an production environment (e.g.
if it's just used for tracing and source output with -C), could you
please add an option to prevent the compiler to add such additional
information to the VCL_conf?
These operations would have to be reviewed, so they still work with this
option activated (if with less verbosity because of the missing data).
I know it's very helpful to have such informations in an staging
environment, but i wouldn't use tracing in production and so i might not
need this extra data there. The performance gain on the other hand could
be noticeable.

I might get a performance boost with the recent version of varnish
already by just minifying the vcl's in the deployment, but there still
would be a lot of data left in the VCL_conf, that may not be needed imo.

I hope you can shed some light on this matter.

Regards
Gordon

--
E-Mail-Signatur

*Gordon Schmidt*
Softwareentwickler - QA

Unister GmbH
Barfußgässchen 11 | 04109 Leipzig

Telefon: +49 (0)341 65050 - 25757
gordon.schmidt [at] unister <mailto:gordon.schmidt [at] unister>
www.unister.de <http://www.unister.de>

Vertretungsberechtigter Geschäftsführer: Thomas Wagner
Amtsgericht Leipzig, HRB: 19056

phk at phk

Jul 30, 2012, 6:44 AM

Post #2 of 7 (276 views)
Permalink

Re: Question about srcbody in VCL_conf [In reply to]

In message <4FF1818A.6080403 [at] unister-gmbh>, Gordon Schmidt writes:

>while trying to optimize varnish performance for my company i've
>noticed, that removing all comments from the vcls used in our
>configuration tribles the requests per seconds from an easy ab test with
>100% cache-misses.

How big is your VCL with/without comments ?

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk [at] FreeBSD | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

gordon.schmidt at unister-gmbh

Jul 31, 2012, 1:10 AM

Post #3 of 7 (260 views)
Permalink

Re: Question about srcbody in VCL_conf [In reply to]

On 30.07.2012 15:44, Poul-Henning Kamp wrote:
> In message <4FF1818A.6080403 [at] unister-gmbh>, Gordon Schmidt writes:
>
>> while trying to optimize varnish performance for my company i've
>> noticed, that removing all comments from the vcls used in our
>> configuration tribles the requests per seconds from an easy ab test with
>> 100% cache-misses.
> How big is your VCL with/without comments ?
>
Hi,

my vcl config is set up to provide caching and load balancing for about
20 portals with several backends each.
By now I have written a small python script to minify the vcl config by
removing comments and empty lines and also resolving includes.
So i'm using just one VCL-File with 33316 bytes instead of a
configuration of 110 vcl files with 41635 bytes in sum.

Regards
Gordon

--
E-Mail-Signatur

*Gordon Schmidt*
Softwareentwickler - QA

Unister GmbH
Barfußgässchen 11 | 04109 Leipzig

Telefon: +49 (0)341 65050 - 25757
gordon.schmidt [at] unister <mailto:gordon.schmidt [at] unister>
www.unister.de <http://www.unister.de>

Vertretungsberechtigter Geschäftsführer: Thomas Wagner
Amtsgericht Leipzig, HRB: 19056

phk at phk

Jul 31, 2012, 1:44 AM

Post #4 of 7 (277 views)
Permalink

Re: Question about srcbody in VCL_conf [In reply to]

In message <501792EA.5060901 [at] unister-gmbh>, Gordon Schmidt writes:

>By now I have written a small python script to minify the vcl config by
>removing comments and empty lines and also resolving includes.
>So i'm using just one VCL-File with 33316 bytes instead of a
>configuration of 110 vcl files with 41635 bytes in sum.

It sounds really strange that this should have the performance
effect you claim...

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk [at] FreeBSD | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

gordon.schmidt at unister-gmbh

Jul 31, 2012, 2:24 AM

Post #5 of 7 (253 views)
Permalink

Re: Question about srcbody in VCL_conf [In reply to]

On 31.07.2012 10:44, Poul-Henning Kamp wrote:
> In message <501792EA.5060901 [at] unister-gmbh>, Gordon Schmidt writes:
>
>> By now I have written a small python script to minify the vcl config by
>> removing comments and empty lines and also resolving includes.
>> So i'm using just one VCL-File with 33316 bytes instead of a
>> configuration of 110 vcl files with 41635 bytes in sum.
> It sounds really strange that this should have the performance
> effect you claim...
I didn't get the 8%-10% by minifying - the performance gain with
minified vcls is lower.
Just for testing, I removed the part in the vcc_compile.c responsable
for adding scrbody to the compiled object to get the full performance
improvement.

My changes for testing were:
index 66d89f5..e51833a 100644
--- a/lib/libvcl/vcc_compile.c
+++ b/lib/libvcl/vcc_compile.c
@@ -343,16 +343,7 @@ EmitStruct(const struct vcc *tl)
}
Fc(tl, 0, "};\n");

- Fc(tl, 0, "\nconst char *srcbody[%u] = {\n", tl->nsources);
- VTAILQ_FOREACH(sp, &tl->sources, list) {
- Fc(tl, 0, " /* ");
- EncString(tl->fc, sp->name, NULL, 0);
- Fc(tl, 0, "*/\n");
- Fc(tl, 0, "\t");
- EncString(tl->fc, sp->b, sp->e, 1);
- Fc(tl, 0, ",\n");
- }
- Fc(tl, 0, "};\n");
+ Fc(tl, 0, "\nconst char *srcbody[%u] = NULL;\n");

Fc(tl, 0, "\nstatic struct director\t*directors[%d];\n",
tl->ndirector);

Regards
Gordon

--
E-Mail-Signatur

*Gordon Schmidt*
Softwareentwickler - QA

Unister GmbH
Barfußgässchen 11 | 04109 Leipzig

Telefon: +49 (0)341 65050 - 25757
gordon.schmidt [at] unister <mailto:gordon.schmidt [at] unister>
www.unister.de <http://www.unister.de>

Vertretungsberechtigter Geschäftsführer: Thomas Wagner
Amtsgericht Leipzig, HRB: 19056

phk at phk

Jul 31, 2012, 2:37 AM

Post #6 of 7 (274 views)
Permalink

Re: Question about srcbody in VCL_conf [In reply to]

In message <5017A456.7090905 [at] unister-gmbh>, Gordon Schmidt writes:

>> It sounds really strange that this should have the performance
>> effect you claim...
>I didn't get the 8%-10% by minifying - the performance gain with
>minified vcls is lower.
>Just for testing, I removed the part in the vcc_compile.c responsable
>for adding scrbody to the compiled object to get the full performance
>improvement.

The only thing you have done is remove the memory footprint by a
few kilobytes.

If that radically changes your performance it can only be because
it allows you to operate entirely inside the L1/L2/L3 cache of your
CPU.

That would indicate that your benchmark is so trivially simple that
it is invalid for any realistisk workload with more than one URL.
>

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk [at] FreeBSD | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

gordon.schmidt at unister-gmbh

Jul 31, 2012, 6:07 AM

Post #7 of 7 (254 views)
Permalink

Re: Question about srcbody in VCL_conf [In reply to]

On 31.07.2012 11:37, Poul-Henning Kamp wrote:
> In message <5017A456.7090905 [at] unister-gmbh>, Gordon Schmidt writes:
>
>>> It sounds really strange that this should have the performance
>>> effect you claim...
>> I didn't get the 8%-10% by minifying - the performance gain with
>> minified vcls is lower.
>> Just for testing, I removed the part in the vcc_compile.c responsable
>> for adding scrbody to the compiled object to get the full performance
>> improvement.
> The only thing you have done is remove the memory footprint by a
> few kilobytes.
>
> If that radically changes your performance it can only be because
> it allows you to operate entirely inside the L1/L2/L3 cache of your
> CPU.
>
> That would indicate that your benchmark is so trivially simple that
> it is invalid for any realistisk workload with more than one URL.
I haven't said anything different. The test is simple as i already
stated in my very first post.

Still - an option to reduce the memory footprint, when the additional
data is not needed, would be nice.

Btw. the situation becomes relevant, when you have one or only few
resources that are not cachable and will hit much more frequently than
all the other resources.

Regards
Gordon

--
E-Mail-Signatur

*Gordon Schmidt*
Softwareentwickler - QA

Unister GmbH
Barfußgässchen 11 | 04109 Leipzig

Telefon: +49 (0)341 65050 - 25757
gordon.schmidt [at] unister <mailto:gordon.schmidt [at] unister>
www.unister.de <http://www.unister.de>

Vertretungsberechtigter Geschäftsführer: Thomas Wagner
Amtsgericht Leipzig, HRB: 19056

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads