Mailing List Archive: Varnish: Misc

Varnish using more RAM than allocated

Index | Next | Previous | View Threaded

Travis.Crowder at penton

Jun 22, 2012, 11:45 AM

Post #1 of 4 (407 views)
Permalink

Varnish using more RAM than allocated

I recall seeing on the mailing list before a reason why Varnish would use more RAM than allocated due to storing something outside of the defined allocated memory.

For example, I have a box that is allocated 6G of RAM, but it is using about 12G virtual memory.

>top
9101 nobody 20 0 11.7g 6.8g 81m S 6.0 88.5 1270:21 varnishd

Varnish is invoked via:
/usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,6G -T xxx.xxx.xxx.xxx:2000 -a :80

Does anyone know what I am talking about?

Thanks,
Travis

lee at leetrout

Jun 22, 2012, 1:42 PM

Post #2 of 4 (386 views)
Permalink

Re: Varnish using more RAM than allocated [In reply to]

I generally don't worry about the high virtual memory usage. (and perhaps
that will bite me some day ^_^)

There are other memory requirements such as the threads (-s is only for the
object cache). Beyond that, there is also transient storage for short lived
requests which defaults to an unlimited malloc. Given that, I would say the
resident memory size looks about right.
https://www.varnish-cache.org/trac/wiki/Performance &
https://www.varnish-cache.org/docs/3.0/reference/varnishd.html#transient-storage

You could also dig in with pmap ;)

On Fri, Jun 22, 2012 at 2:45 PM, Crowder, Travis
<Travis.Crowder [at] penton>wrote:

> I recall seeing on the mailing list before a reason why Varnish would
> use more RAM than allocated due to storing something outside of the defined
> allocated memory.****
>
> ** **
>
> For example, I have a box that is allocated 6G of RAM, but it is using
> about 12G virtual memory.****
>
> ** **
>
> >top****
>
> 9101 nobody 20 0 11.7g 6.8g 81m S 6.0 88.5 1270:21 varnishd****
>
> ** **
>
> Varnish is invoked via:****
>
> /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -s
> malloc,6G -T xxx.xxx.xxx.xxx:2000 -a :80****
>
> ** **
>
> Does anyone know what I am talking about?****
>
> ** **
>
> Thanks,****
>
> Travis****
>
> ** **
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc [at] varnish-cache
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>

perbu at varnish-software

Jun 22, 2012, 2:03 PM

Post #3 of 4 (388 views)
Permalink

Re: Varnish using more RAM than allocated [In reply to]

On Fri, Jun 22, 2012 at 8:45 PM, Crowder, Travis
<Travis.Crowder [at] penton>wrote:

> I recall seeing on the mailing list before a reason why Varnish would
> use more RAM than allocated due to storing something outside of the defined
> allocated memory.****
>
> ** **
>
> For example, I have a box that is allocated 6G of RAM, but it is using
> about 12G virtual memory.
>

That doesn't make sense. Virtual memory is virtual. You're not using it,
Varnish has mapped it into it's memory space. Just ignore it.

Also, note that allocating memory to a backend (6G) does not include the
amount of memory needed for the various internal structures - more or less
1k per object.

--
Per Buer
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
*Varnish makes websites fly!*
Whitepapers <http://www.varnish-software.com/whitepapers> |
Video<http://www.youtube.com/watch?v=x7t2Sp174eI> |
Twitter <https://twitter.com/varnishsoftware>

Travis.Crowder at penton

Jun 22, 2012, 3:16 PM

Post #4 of 4 (389 views)
Permalink

Re: Varnish using more RAM than allocated [In reply to]

Thank you. That about "1k per object for internal structures" is what I was looking for.

Sent from my iPhone

On Jun 22, 2012, at 4:04 PM, "Per Buer" <perbu [at] varnish-software<mailto:perbu [at] varnish-software>> wrote:

On Fri, Jun 22, 2012 at 8:45 PM, Crowder, Travis <Travis.Crowder [at] penton<mailto:Travis.Crowder [at] penton>> wrote:
I recall seeing on the mailing list before a reason why Varnish would use more RAM than allocated due to storing something outside of the defined allocated memory.

For example, I have a box that is allocated 6G of RAM, but it is using about 12G virtual memory.

That doesn't make sense. Virtual memory is virtual. You're not using it, Varnish has mapped it into it's memory space. Just ignore it.

Also, note that allocating memory to a backend (6G) does not include the amount of memory needed for the various internal structures - more or less 1k per object.

--
[http://www.varnish-software.com/sites/default/files/varnishsoft_white_190x47.png]
Per Buer
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Whitepapers<http://www.varnish-software.com/whitepapers> | Video<http://www.youtube.com/watch?v=x7t2Sp174eI> | Twitter<https://twitter.com/varnishsoftware>

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads