Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Varnish: Dev

Controlling memory usage

 

 

Varnish dev RSS feed   Index | Next | Previous | View Threaded


anders at fupp

Jul 17, 2006, 4:31 AM

Post #1 of 16 (260 views)
Permalink
Controlling memory usage

Hi,

Will Varnish dynamically use RAM optimally given how much you have
available?

A problem with Squid today is that it's not possible to set an upper
limit for how much RAM it will use, and with time it can easily grow
beyond the limits of the VM causing it to get killed (that is it
restarts itself, losing all cache in RAM).

I take it this will not be a problem with Varnish? :-)

Do you use kernel/userspace memory as needed?

Cheers,

--
Anders.


anders at fupp

Jul 17, 2006, 4:31 AM

Post #2 of 16 (255 views)
Permalink
Controlling memory usage [In reply to]

Hi,

Will Varnish dynamically use RAM optimally given how much you have
available?

A problem with Squid today is that it's not possible to set an upper
limit for how much RAM it will use, and with time it can easily grow
beyond the limits of the VM causing it to get killed (that is it
restarts itself, losing all cache in RAM).

I take it this will not be a problem with Varnish? :-)

Do you use kernel/userspace memory as needed?

Cheers,

--
Anders.


des at linpro

Jul 17, 2006, 5:01 AM

Post #3 of 16 (254 views)
Permalink
Controlling memory usage [In reply to]

Anders Nordby <anders at fupp.net> writes:
> Will Varnish dynamically use RAM optimally given how much you have
> available?

Yes and no. With the file-based storage backend, Varnish stores
documents in a single file which it mmaps, and malloc() is used only
for housekeeping structures. Memory efficiency in this scenario is
really up to the kernel, specifically the buffer cache.

You can see from the screenshots from day 2 of the live test (online
at <URL:http://www.des.no/varnish/day2/>) that Varnish uses 400 MB RAM
with ~23,000 objects cached. This later grew to ~45,000 objects, but
I don't have memory usage numbers for that part of the test (I killed
top at some point to look at some other numbers and didn't restart
it). You can however look at the systat numbers from the last
screenshot:

293100 wire
214924 act
3075272 inact
201504 cache
219632 buf

Since the machine isn't swapping, "act" is more or less the sum of the
stacks and heaps of all running processes, including Varnish, while
"cache" and "buf" give an estimate of how much memory is used to cache
disk pages, including Varnish's storage file. The most impressive
number is "inact", which says that almost three quarters of the
server's 4 GB of memory are completely unused.

Next time, I'll try to remember to take snapshots of Varnish's
/proc/$$/map to get a precise breakdown.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


des at linpro

Jul 17, 2006, 5:01 AM

Post #4 of 16 (254 views)
Permalink
Controlling memory usage [In reply to]

Anders Nordby <anders at fupp.net> writes:
> Will Varnish dynamically use RAM optimally given how much you have
> available?

Yes and no. With the file-based storage backend, Varnish stores
documents in a single file which it mmaps, and malloc() is used only
for housekeeping structures. Memory efficiency in this scenario is
really up to the kernel, specifically the buffer cache.

You can see from the screenshots from day 2 of the live test (online
at <URL:http://www.des.no/varnish/day2/>) that Varnish uses 400 MB RAM
with ~23,000 objects cached. This later grew to ~45,000 objects, but
I don't have memory usage numbers for that part of the test (I killed
top at some point to look at some other numbers and didn't restart
it). You can however look at the systat numbers from the last
screenshot:

293100 wire
214924 act
3075272 inact
201504 cache
219632 buf

Since the machine isn't swapping, "act" is more or less the sum of the
stacks and heaps of all running processes, including Varnish, while
"cache" and "buf" give an estimate of how much memory is used to cache
disk pages, including Varnish's storage file. The most impressive
number is "inact", which says that almost three quarters of the
server's 4 GB of memory are completely unused.

Next time, I'll try to remember to take snapshots of Varnish's
/proc/$$/map to get a precise breakdown.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


phk at phk

Jul 17, 2006, 6:53 AM

Post #5 of 16 (251 views)
Permalink
Controlling memory usage [In reply to]

In message <20060717113114.GA76977 at totem.fix.no>, Anders Nordby writes:


Gee, you really know how to ask "simple" questions, don't you ? :-)


>Will Varnish dynamically use RAM optimally given how much you have
>available?

That is the intent, but we implement it by letting the kernel
decide on that.

>A problem with Squid today is that it's not possible to set an upper
>limit for how much RAM it will use, and with time it can easily grow
>beyond the limits of the VM causing it to get killed (that is it
>restarts itself, losing all cache in RAM).

Yes, I've seen that one already many years ago.

Squids two-tier storage architecture where some stuff is in ram
and the rest on disk is collapsed in varnish so that it all is
on disk, but mapped into memory.

It's important to understand that we are talking about at least
three different kinds of memory of relevance: Virtual address room,
Virtual allocated space and Virtual mapped space.

The virtual address room is the amount of memory that the program
can address and it is basically the width of the CPU: 32 or
64 bits. On 32 bit systems, we're limited to approx 3.5 GB. On
64bit systems it's a non-issue. (This will limit 32bit systems
total cache capacity accordingly, but 64bit systems are no more
expensive these days).

Virtual allocated space is the varnish programs variables and
malloc(3) allocated space. This is what is normally counted
as "programs memory use". All of squids memory falls in this
category. If you use the 'malloc" storage method of varnish
it also falls here.

Virtual mapped space is the backing store file which is mapped
into the varnish process address space, and this is accounted
differently than virtual allocated space.

Finally there is the concept of "resident set size" which is
the amount of actual physical RAM the process occupies
with the bits of the above two which are mapped into memory
at any one particular time.

>I take it this will not be a problem with Varnish? :-)

I hope not, but quite frankly, apart from some scientific apps I
have seen very few programs that use memory this way (yet! but after
all we have only had virtual memory for 20 years :-/ ) so the scope
for OS bugs is certainly present.

>Do you use kernel/userspace memory as needed?

Well, yes and no: I leave it to the operating system to do so.

I think another way to answer your question is this:

Squid is written for the old UNIX/swap model before the VAX/780
computer came around: it doesn't in any sense of the word comprehend
virtual memory. In may ways this gets in the way of efficiency.
Reading things from a file, into userland and then writing it to a
socket moves the data:

from disk (with DMA) to kernel RAM buffer
from kernel RAM buffer to userland RAM.
from userland RAM to network buffer in kernel
from network buffer in kernel to network interface (with DMA)

This is just plain waste of good cycles.

Varnish on the other hand, is written with the maximum comprehension
of virtual memory and tries to do everything it can to avoid getting
in the way of the operating systems efficient. Varnish maps the
backing file into the process, so the above scenario cuts out
one copy already there:

from disk (with DMA) to userland mapped memory
from userland mapped memory to network buffer in kernel
from network buffer in kernel to network interface (with DMA)

And by using the sendfile(2) system call, this even gets reduced to:

from disk (with DMA) to userland mapped memory
send userland mapped memory to network interface (with DMA)

(Various details left out, TCP/IP checksums in particular)

I hope this if not exactly answers your question, at least gives
you the impression that I've thought hard about it :-)

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


phk at phk

Jul 17, 2006, 6:53 AM

Post #6 of 16 (250 views)
Permalink
Controlling memory usage [In reply to]

In message <20060717113114.GA76977 at totem.fix.no>, Anders Nordby writes:


Gee, you really know how to ask "simple" questions, don't you ? :-)


>Will Varnish dynamically use RAM optimally given how much you have
>available?

That is the intent, but we implement it by letting the kernel
decide on that.

>A problem with Squid today is that it's not possible to set an upper
>limit for how much RAM it will use, and with time it can easily grow
>beyond the limits of the VM causing it to get killed (that is it
>restarts itself, losing all cache in RAM).

Yes, I've seen that one already many years ago.

Squids two-tier storage architecture where some stuff is in ram
and the rest on disk is collapsed in varnish so that it all is
on disk, but mapped into memory.

It's important to understand that we are talking about at least
three different kinds of memory of relevance: Virtual address room,
Virtual allocated space and Virtual mapped space.

The virtual address room is the amount of memory that the program
can address and it is basically the width of the CPU: 32 or
64 bits. On 32 bit systems, we're limited to approx 3.5 GB. On
64bit systems it's a non-issue. (This will limit 32bit systems
total cache capacity accordingly, but 64bit systems are no more
expensive these days).

Virtual allocated space is the varnish programs variables and
malloc(3) allocated space. This is what is normally counted
as "programs memory use". All of squids memory falls in this
category. If you use the 'malloc" storage method of varnish
it also falls here.

Virtual mapped space is the backing store file which is mapped
into the varnish process address space, and this is accounted
differently than virtual allocated space.

Finally there is the concept of "resident set size" which is
the amount of actual physical RAM the process occupies
with the bits of the above two which are mapped into memory
at any one particular time.

>I take it this will not be a problem with Varnish? :-)

I hope not, but quite frankly, apart from some scientific apps I
have seen very few programs that use memory this way (yet! but after
all we have only had virtual memory for 20 years :-/ ) so the scope
for OS bugs is certainly present.

>Do you use kernel/userspace memory as needed?

Well, yes and no: I leave it to the operating system to do so.

I think another way to answer your question is this:

Squid is written for the old UNIX/swap model before the VAX/780
computer came around: it doesn't in any sense of the word comprehend
virtual memory. In may ways this gets in the way of efficiency.
Reading things from a file, into userland and then writing it to a
socket moves the data:

from disk (with DMA) to kernel RAM buffer
from kernel RAM buffer to userland RAM.
from userland RAM to network buffer in kernel
from network buffer in kernel to network interface (with DMA)

This is just plain waste of good cycles.

Varnish on the other hand, is written with the maximum comprehension
of virtual memory and tries to do everything it can to avoid getting
in the way of the operating systems efficient. Varnish maps the
backing file into the process, so the above scenario cuts out
one copy already there:

from disk (with DMA) to userland mapped memory
from userland mapped memory to network buffer in kernel
from network buffer in kernel to network interface (with DMA)

And by using the sendfile(2) system call, this even gets reduced to:

from disk (with DMA) to userland mapped memory
send userland mapped memory to network interface (with DMA)

(Various details left out, TCP/IP checksums in particular)

I hope this if not exactly answers your question, at least gives
you the impression that I've thought hard about it :-)

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


andersb at vgnett

Jul 17, 2006, 10:49 AM

Post #7 of 16 (255 views)
Permalink
Controlling memory usage [In reply to]

Thx for detailed answers Poul-Henning and Dag-Erling,

The reason Anders N. asks about this is how Squid works today. The
squid.conf file leaves you with a option to specify how much RAM you
wanna use for Squid. The problem is that Squid (probably because of
old design) does not really "follow" that option. If you set 256 MB
RAM, you can still end up using 500 MB RAM, and if you set option
close to max memory you will for certain overflow your physical RAM
and start swapping/die. Your answer was detailed Poul-Henning, but
what will prevent this from happpening in Varnish? Lets say you have
2 applications running on a Varnish box, and both use the memory
model Varnish uses, what will happen in the long run with a lot of
traffic? Will they both adjust themselvs to match pysical RAM, how
would they "compete" for RAM etc? It's possible you have already
answered that below, but how do we "limit" RAM usage?

Also I would like to mention that while at VG neither Squid or
Varnish (it seems) have any problem dealing with I/O, the traffic
pattern, pysical size'es of dataset's etc. make it 100% CPU bound. My
impression is that finn.no's pattern is quite different and interesting.

A quick look on:

http://www.tns-gallup.no/index.asp?
type=tabelno_url&did=185235&sort=uv&sort_ret=desc&UgeSelect=&path_by_id=
/12000/12003/12077/12266&aid=12266

will show what I mean. They have "few" users and sessions, but loads
of pageviews, also at any given time many thousand "wares" are "hot"
for the user, not a few popular articles. This does something to mem
usage and I/O. I am looking forward to test-run Varnish on finn.no :)
At the same time when a ware is "sold" purging becomes important :)
Maybe not so much on finn.no today, but I can easily see auctions
happen in the close future :)

Anders Berg

On Jul 17, 2006, at 15:53, Poul-Henning Kamp wrote:

> In message <20060717113114.GA76977 at totem.fix.no>, Anders Nordby
> writes:
>
>
> Gee, you really know how to ask "simple" questions, don't you ? :-)
>
>
>> Will Varnish dynamically use RAM optimally given how much you have
>> available?
>
> That is the intent, but we implement it by letting the kernel
> decide on that.
>
>> A problem with Squid today is that it's not possible to set an upper
>> limit for how much RAM it will use, and with time it can easily grow
>> beyond the limits of the VM causing it to get killed (that is it
>> restarts itself, losing all cache in RAM).
>
> Yes, I've seen that one already many years ago.
>
> Squids two-tier storage architecture where some stuff is in ram
> and the rest on disk is collapsed in varnish so that it all is
> on disk, but mapped into memory.
>
> It's important to understand that we are talking about at least
> three different kinds of memory of relevance: Virtual address room,
> Virtual allocated space and Virtual mapped space.
>
> The virtual address room is the amount of memory that the program
> can address and it is basically the width of the CPU: 32 or
> 64 bits. On 32 bit systems, we're limited to approx 3.5 GB. On
> 64bit systems it's a non-issue. (This will limit 32bit systems
> total cache capacity accordingly, but 64bit systems are no more
> expensive these days).
>
> Virtual allocated space is the varnish programs variables and
> malloc(3) allocated space. This is what is normally counted
> as "programs memory use". All of squids memory falls in this
> category. If you use the 'malloc" storage method of varnish
> it also falls here.
>
> Virtual mapped space is the backing store file which is mapped
> into the varnish process address space, and this is accounted
> differently than virtual allocated space.
>
> Finally there is the concept of "resident set size" which is
> the amount of actual physical RAM the process occupies
> with the bits of the above two which are mapped into memory
> at any one particular time.
>
>> I take it this will not be a problem with Varnish? :-)
>
> I hope not, but quite frankly, apart from some scientific apps I
> have seen very few programs that use memory this way (yet! but after
> all we have only had virtual memory for 20 years :-/ ) so the scope
> for OS bugs is certainly present.
>
>> Do you use kernel/userspace memory as needed?
>
> Well, yes and no: I leave it to the operating system to do so.
>
> I think another way to answer your question is this:
>
> Squid is written for the old UNIX/swap model before the VAX/780
> computer came around: it doesn't in any sense of the word comprehend
> virtual memory. In may ways this gets in the way of efficiency.
> Reading things from a file, into userland and then writing it to a
> socket moves the data:
>
> from disk (with DMA) to kernel RAM buffer
> from kernel RAM buffer to userland RAM.
> from userland RAM to network buffer in kernel
> from network buffer in kernel to network interface (with DMA)
>
> This is just plain waste of good cycles.
>
> Varnish on the other hand, is written with the maximum comprehension
> of virtual memory and tries to do everything it can to avoid getting
> in the way of the operating systems efficient. Varnish maps the
> backing file into the process, so the above scenario cuts out
> one copy already there:
>
> from disk (with DMA) to userland mapped memory
> from userland mapped memory to network buffer in kernel
> from network buffer in kernel to network interface (with DMA)
>
> And by using the sendfile(2) system call, this even gets reduced to:
>
> from disk (with DMA) to userland mapped memory
> send userland mapped memory to network interface (with DMA)
>
> (Various details left out, TCP/IP checksums in particular)
>
> I hope this if not exactly answers your question, at least gives
> you the impression that I've thought hard about it :-)
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-dev
>


andersb at vgnett

Jul 17, 2006, 10:49 AM

Post #8 of 16 (252 views)
Permalink
Controlling memory usage [In reply to]

Thx for detailed answers Poul-Henning and Dag-Erling,

The reason Anders N. asks about this is how Squid works today. The
squid.conf file leaves you with a option to specify how much RAM you
wanna use for Squid. The problem is that Squid (probably because of
old design) does not really "follow" that option. If you set 256 MB
RAM, you can still end up using 500 MB RAM, and if you set option
close to max memory you will for certain overflow your physical RAM
and start swapping/die. Your answer was detailed Poul-Henning, but
what will prevent this from happpening in Varnish? Lets say you have
2 applications running on a Varnish box, and both use the memory
model Varnish uses, what will happen in the long run with a lot of
traffic? Will they both adjust themselvs to match pysical RAM, how
would they "compete" for RAM etc? It's possible you have already
answered that below, but how do we "limit" RAM usage?

Also I would like to mention that while at VG neither Squid or
Varnish (it seems) have any problem dealing with I/O, the traffic
pattern, pysical size'es of dataset's etc. make it 100% CPU bound. My
impression is that finn.no's pattern is quite different and interesting.

A quick look on:

http://www.tns-gallup.no/index.asp?
type=tabelno_url&did=185235&sort=uv&sort_ret=desc&UgeSelect=&path_by_id=
/12000/12003/12077/12266&aid=12266

will show what I mean. They have "few" users and sessions, but loads
of pageviews, also at any given time many thousand "wares" are "hot"
for the user, not a few popular articles. This does something to mem
usage and I/O. I am looking forward to test-run Varnish on finn.no :)
At the same time when a ware is "sold" purging becomes important :)
Maybe not so much on finn.no today, but I can easily see auctions
happen in the close future :)

Anders Berg

On Jul 17, 2006, at 15:53, Poul-Henning Kamp wrote:

> In message <20060717113114.GA76977 at totem.fix.no>, Anders Nordby
> writes:
>
>
> Gee, you really know how to ask "simple" questions, don't you ? :-)
>
>
>> Will Varnish dynamically use RAM optimally given how much you have
>> available?
>
> That is the intent, but we implement it by letting the kernel
> decide on that.
>
>> A problem with Squid today is that it's not possible to set an upper
>> limit for how much RAM it will use, and with time it can easily grow
>> beyond the limits of the VM causing it to get killed (that is it
>> restarts itself, losing all cache in RAM).
>
> Yes, I've seen that one already many years ago.
>
> Squids two-tier storage architecture where some stuff is in ram
> and the rest on disk is collapsed in varnish so that it all is
> on disk, but mapped into memory.
>
> It's important to understand that we are talking about at least
> three different kinds of memory of relevance: Virtual address room,
> Virtual allocated space and Virtual mapped space.
>
> The virtual address room is the amount of memory that the program
> can address and it is basically the width of the CPU: 32 or
> 64 bits. On 32 bit systems, we're limited to approx 3.5 GB. On
> 64bit systems it's a non-issue. (This will limit 32bit systems
> total cache capacity accordingly, but 64bit systems are no more
> expensive these days).
>
> Virtual allocated space is the varnish programs variables and
> malloc(3) allocated space. This is what is normally counted
> as "programs memory use". All of squids memory falls in this
> category. If you use the 'malloc" storage method of varnish
> it also falls here.
>
> Virtual mapped space is the backing store file which is mapped
> into the varnish process address space, and this is accounted
> differently than virtual allocated space.
>
> Finally there is the concept of "resident set size" which is
> the amount of actual physical RAM the process occupies
> with the bits of the above two which are mapped into memory
> at any one particular time.
>
>> I take it this will not be a problem with Varnish? :-)
>
> I hope not, but quite frankly, apart from some scientific apps I
> have seen very few programs that use memory this way (yet! but after
> all we have only had virtual memory for 20 years :-/ ) so the scope
> for OS bugs is certainly present.
>
>> Do you use kernel/userspace memory as needed?
>
> Well, yes and no: I leave it to the operating system to do so.
>
> I think another way to answer your question is this:
>
> Squid is written for the old UNIX/swap model before the VAX/780
> computer came around: it doesn't in any sense of the word comprehend
> virtual memory. In may ways this gets in the way of efficiency.
> Reading things from a file, into userland and then writing it to a
> socket moves the data:
>
> from disk (with DMA) to kernel RAM buffer
> from kernel RAM buffer to userland RAM.
> from userland RAM to network buffer in kernel
> from network buffer in kernel to network interface (with DMA)
>
> This is just plain waste of good cycles.
>
> Varnish on the other hand, is written with the maximum comprehension
> of virtual memory and tries to do everything it can to avoid getting
> in the way of the operating systems efficient. Varnish maps the
> backing file into the process, so the above scenario cuts out
> one copy already there:
>
> from disk (with DMA) to userland mapped memory
> from userland mapped memory to network buffer in kernel
> from network buffer in kernel to network interface (with DMA)
>
> And by using the sendfile(2) system call, this even gets reduced to:
>
> from disk (with DMA) to userland mapped memory
> send userland mapped memory to network interface (with DMA)
>
> (Various details left out, TCP/IP checksums in particular)
>
> I hope this if not exactly answers your question, at least gives
> you the impression that I've thought hard about it :-)
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-dev
>


phk at phk

Jul 17, 2006, 12:02 PM

Post #9 of 16 (252 views)
Permalink
Controlling memory usage [In reply to]

In message <B06E690C-CA93-4547-AD2A-61246521142D at vgnett.no>, Anders Berg writes
:

>The reason Anders N. asks about this is how Squid works today. The
>squid.conf file leaves you with a option to specify how much RAM you
>wanna use for Squid.

And this is right where the trouble starts.

Squid is written for a machine model that has not existed since
1980 when 3BSD was released.

In that model, a process has some amount of "memory" and either all
of that "memory" is present in RAM or none of it is. When RAM grew
short, an entire process was swapped out (hence the name: "swap out
one process for another")

In that environment, it gives great meaning to tell Squid how much
RAM it can use, because there is some magic size where the best
performance compromise for the entire machine is reached.

We spent a lot of time tuning stuff like that in 1980ies, we told
sort(1) how many records to sort in memory and to switch to merging
temporary files if it found more etc etc.

Virtual memory on the other hand, means that the kernel "fakes"
things such that the process has access to the entire address-space
(ie: 2^32 bytes or 2^64 bytes) and the operating. It does this by
tracking which pages are used, which are modified and all that
stuff.

In a VM system, what you think of as "RAM" is not RAM in the hardware
sense. You may in fact have all of it accessible in hardware RAM,
but if the system is short of memory, you won't have, some of it
will be "paged out to disk" or because we sloppily adopted the old
terminology: swapped out.

The real trouble starts when Squid decides that an object in its
"RAM" should be purged to disk. Quite likely, the operating system
already found that out earlier so the "RAM" is already on disk,
somewhere in the paging- (or swap-)partition.

So what happens is that first we do a disk read to pull in the RAM,
then we write it to disk some other place.

Twice as much I/O for no gain. The same pattern happens all over
Squid, and that is responsible for the observed "once squid starts
paging, it goes straight south".

It doesn't help in this context that Squid stores headers and body
the same place. That means that if the "RAM" of some object has
been paged out, we have to page it in to see the headers, even for
a conditional request which ends up not transferring the objects
body.


>Your answer was detailed Poul-Henning, but
>what will prevent this from happpening in Varnish? Lets say you have
>2 applications running on a Varnish box, and both use the memory
>model Varnish uses, what will happen in the long run with a lot of
>traffic?

All programs running in a VM system has a function which describes
how fast they reach their goal, for a given number of pages of
hardware memory they have access to.

Unfortunately the function also has other variables, the input to
the program, the timing its interaction with the world (how long
must it wait for disk-I/O etc) and the state of all sorts of kernel
caches come into play.

There is no way to predict the function realistically. You can
measure it under some set of circumstances and get an idea how it
looks.

The only trick there really is to writing an VM kernel is being
good at estimating this function on the fly.

If two processes run at the same time, and they both need more
hw-RAM pages than the system has, the kernel will be flipping some
pages between them.

When a program accesses a page which is not "resident", the kernel
will hunt around for a page that doesn't look used (ideally: doesn't
look like it _will be_ used (soon)) writes that page to disk and
reassigns the page to the faulting process, possibly after filling
it from a disk first.

In the meantime the process (or at least: thread) cannot do anthing.

If you're just one page short, there is undoubtedly some page in
the process which is seldomly used, the first bit of the program
which is only used during startup, some table of error messages
that are only accessed when there is an error etc.

As memory pressure increases, more and more such pages will be paged
out. At some point, we get to pages which are infact used every
so often, and then it starts hurting performance.

The thing to remember in writing programs for virtual memory sytems
is therefore not to be careful about how much memory you allocate,
but instead be careful about how much of it you use.

Something as simple as variable order in the source code can affect
this:

int busy_integer_variable;
char seldomly_used_error_string[5000];
double often_used_number;

With a pagesize of 4096, this will take up two pages, both of which
will be busy. Flipping the order:

int busy_integer_variable;
double often_used_number;
char seldomly_used_error_string[5000];

means that one page will seldomly be used, and the other will be
used all the time. (The example also improves CPU-cache hitrates,
but forget that for now).

What Varnish does is to rely on the kernel to do this work. Instead
of trying to control how much memory we use and partition our data
into the fast stuff which should be in RAM and the slow stuff which
we can put on the disk, we simply operate on one data set, but make
sure to arrange our data such that the kernel can easily deport
data which we don't use, without us needing to get involved.

Therefore all object storage in Varnish is allocated on a page-aligned
border. That means that entire objects can be paged out, without
affecting the neighbor objects. Yes, this may waste 4095 bytes for
padding, but you'd be surprised what you save in performance.

>http://www.tns-gallup.no/index.asp?
>type=tabelno_url&did=185235&sort=uv&sort_ret=desc&UgeSelect=&path_by_id=
>/12000/12003/12077/12266&aid=12266
>
>will show what I mean. They have "few" users and sessions, but loads
>of pageviews, also at any given time many thousand "wares" are "hot"
>for the user, not a few popular articles. This does something to mem
>usage and I/O.

You're thinking about memory in the oldfashioned terms here :-)

Try this: Imagine the disk is the real memory and the RAM is
only a cache.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


phk at phk

Jul 17, 2006, 12:02 PM

Post #10 of 16 (251 views)
Permalink
Controlling memory usage [In reply to]

In message <B06E690C-CA93-4547-AD2A-61246521142D at vgnett.no>, Anders Berg writes
:

>The reason Anders N. asks about this is how Squid works today. The
>squid.conf file leaves you with a option to specify how much RAM you
>wanna use for Squid.

And this is right where the trouble starts.

Squid is written for a machine model that has not existed since
1980 when 3BSD was released.

In that model, a process has some amount of "memory" and either all
of that "memory" is present in RAM or none of it is. When RAM grew
short, an entire process was swapped out (hence the name: "swap out
one process for another")

In that environment, it gives great meaning to tell Squid how much
RAM it can use, because there is some magic size where the best
performance compromise for the entire machine is reached.

We spent a lot of time tuning stuff like that in 1980ies, we told
sort(1) how many records to sort in memory and to switch to merging
temporary files if it found more etc etc.

Virtual memory on the other hand, means that the kernel "fakes"
things such that the process has access to the entire address-space
(ie: 2^32 bytes or 2^64 bytes) and the operating. It does this by
tracking which pages are used, which are modified and all that
stuff.

In a VM system, what you think of as "RAM" is not RAM in the hardware
sense. You may in fact have all of it accessible in hardware RAM,
but if the system is short of memory, you won't have, some of it
will be "paged out to disk" or because we sloppily adopted the old
terminology: swapped out.

The real trouble starts when Squid decides that an object in its
"RAM" should be purged to disk. Quite likely, the operating system
already found that out earlier so the "RAM" is already on disk,
somewhere in the paging- (or swap-)partition.

So what happens is that first we do a disk read to pull in the RAM,
then we write it to disk some other place.

Twice as much I/O for no gain. The same pattern happens all over
Squid, and that is responsible for the observed "once squid starts
paging, it goes straight south".

It doesn't help in this context that Squid stores headers and body
the same place. That means that if the "RAM" of some object has
been paged out, we have to page it in to see the headers, even for
a conditional request which ends up not transferring the objects
body.


>Your answer was detailed Poul-Henning, but
>what will prevent this from happpening in Varnish? Lets say you have
>2 applications running on a Varnish box, and both use the memory
>model Varnish uses, what will happen in the long run with a lot of
>traffic?

All programs running in a VM system has a function which describes
how fast they reach their goal, for a given number of pages of
hardware memory they have access to.

Unfortunately the function also has other variables, the input to
the program, the timing its interaction with the world (how long
must it wait for disk-I/O etc) and the state of all sorts of kernel
caches come into play.

There is no way to predict the function realistically. You can
measure it under some set of circumstances and get an idea how it
looks.

The only trick there really is to writing an VM kernel is being
good at estimating this function on the fly.

If two processes run at the same time, and they both need more
hw-RAM pages than the system has, the kernel will be flipping some
pages between them.

When a program accesses a page which is not "resident", the kernel
will hunt around for a page that doesn't look used (ideally: doesn't
look like it _will be_ used (soon)) writes that page to disk and
reassigns the page to the faulting process, possibly after filling
it from a disk first.

In the meantime the process (or at least: thread) cannot do anthing.

If you're just one page short, there is undoubtedly some page in
the process which is seldomly used, the first bit of the program
which is only used during startup, some table of error messages
that are only accessed when there is an error etc.

As memory pressure increases, more and more such pages will be paged
out. At some point, we get to pages which are infact used every
so often, and then it starts hurting performance.

The thing to remember in writing programs for virtual memory sytems
is therefore not to be careful about how much memory you allocate,
but instead be careful about how much of it you use.

Something as simple as variable order in the source code can affect
this:

int busy_integer_variable;
char seldomly_used_error_string[5000];
double often_used_number;

With a pagesize of 4096, this will take up two pages, both of which
will be busy. Flipping the order:

int busy_integer_variable;
double often_used_number;
char seldomly_used_error_string[5000];

means that one page will seldomly be used, and the other will be
used all the time. (The example also improves CPU-cache hitrates,
but forget that for now).

What Varnish does is to rely on the kernel to do this work. Instead
of trying to control how much memory we use and partition our data
into the fast stuff which should be in RAM and the slow stuff which
we can put on the disk, we simply operate on one data set, but make
sure to arrange our data such that the kernel can easily deport
data which we don't use, without us needing to get involved.

Therefore all object storage in Varnish is allocated on a page-aligned
border. That means that entire objects can be paged out, without
affecting the neighbor objects. Yes, this may waste 4095 bytes for
padding, but you'd be surprised what you save in performance.

>http://www.tns-gallup.no/index.asp?
>type=tabelno_url&did=185235&sort=uv&sort_ret=desc&UgeSelect=&path_by_id=
>/12000/12003/12077/12266&aid=12266
>
>will show what I mean. They have "few" users and sessions, but loads
>of pageviews, also at any given time many thousand "wares" are "hot"
>for the user, not a few popular articles. This does something to mem
>usage and I/O.

You're thinking about memory in the oldfashioned terms here :-)

Try this: Imagine the disk is the real memory and the RAM is
only a cache.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


des at linpro

Jul 17, 2006, 12:34 PM

Post #11 of 16 (252 views)
Permalink
Controlling memory usage [In reply to]

Anders Berg <andersb at vgnett.no> writes:
> The reason Anders N. asks about this is how Squid works today. The
> squid.conf file leaves you with a option to specify how much RAM you
> wanna use for Squid.

I've worked with Squid quite a lot myself, and IIRC it normally uses
about four times as much as you specify in the config file...

> The problem is that Squid (probably because of
> old design) does not really "follow" that option. If you set 256 MB
> RAM, you can still end up using 500 MB RAM, and if you set option
> close to max memory you will for certain overflow your physical RAM
> and start swapping/die. Your answer was detailed Poul-Henning, but
> what will prevent this from happpening in Varnish? Lets say you have
> 2 applications running on a Varnish box, and both use the memory
> model Varnish uses, what will happen in the long run with a lot of
> traffic? Will they both adjust themselvs to match pysical RAM, how
> would they "compete" for RAM etc? It's possible you have already
> answered that below, but how do we "limit" RAM usage?

We don't. The kernel takes care of it. If memory is scarce, the
kernel will allocate less memory to the buffer cache and Varnish
performance will decrease.

(assuming you use file storage, not malloc storage)

As for heap usage, which the kernel has less control over, it is
mostly a function of the number of objects in the cache; we can take
care of that by setting an upper limit on the number of cached
objects, or even on the amount of heap used to track objects.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


des at linpro

Jul 17, 2006, 12:34 PM

Post #12 of 16 (253 views)
Permalink
Controlling memory usage [In reply to]

Anders Berg <andersb at vgnett.no> writes:
> The reason Anders N. asks about this is how Squid works today. The
> squid.conf file leaves you with a option to specify how much RAM you
> wanna use for Squid.

I've worked with Squid quite a lot myself, and IIRC it normally uses
about four times as much as you specify in the config file...

> The problem is that Squid (probably because of
> old design) does not really "follow" that option. If you set 256 MB
> RAM, you can still end up using 500 MB RAM, and if you set option
> close to max memory you will for certain overflow your physical RAM
> and start swapping/die. Your answer was detailed Poul-Henning, but
> what will prevent this from happpening in Varnish? Lets say you have
> 2 applications running on a Varnish box, and both use the memory
> model Varnish uses, what will happen in the long run with a lot of
> traffic? Will they both adjust themselvs to match pysical RAM, how
> would they "compete" for RAM etc? It's possible you have already
> answered that below, but how do we "limit" RAM usage?

We don't. The kernel takes care of it. If memory is scarce, the
kernel will allocate less memory to the buffer cache and Varnish
performance will decrease.

(assuming you use file storage, not malloc storage)

As for heap usage, which the kernel has less control over, it is
mostly a function of the number of objects in the cache; we can take
care of that by setting an upper limit on the number of cached
objects, or even on the amount of heap used to track objects.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


andersb at vgnett

Jul 17, 2006, 1:44 PM

Post #13 of 16 (253 views)
Permalink
Controlling memory usage [In reply to]

Okay, that made more sense to me.

Thank you for another detailed explanation. Rest assured that
question number 3, after "how do I start?", "how do I stop?", is "how
can I allocate memory?". This is good material in the process of
trying to tell old Squid users how we "use" memory. FAQ material.

Anders Berg




On Jul 17, 2006, at 21:02, Poul-Henning Kamp wrote:

> In message <B06E690C-CA93-4547-AD2A-61246521142D at vgnett.no>, Anders
> Berg writes
> :
>
>> The reason Anders N. asks about this is how Squid works today. The
>> squid.conf file leaves you with a option to specify how much RAM you
>> wanna use for Squid.
>
> And this is right where the trouble starts.
>
> Squid is written for a machine model that has not existed since
> 1980 when 3BSD was released.
>
> In that model, a process has some amount of "memory" and either all
> of that "memory" is present in RAM or none of it is. When RAM grew
> short, an entire process was swapped out (hence the name: "swap out
> one process for another")


andersb at vgnett

Jul 17, 2006, 1:44 PM

Post #14 of 16 (253 views)
Permalink
Controlling memory usage [In reply to]

Okay, that made more sense to me.

Thank you for another detailed explanation. Rest assured that
question number 3, after "how do I start?", "how do I stop?", is "how
can I allocate memory?". This is good material in the process of
trying to tell old Squid users how we "use" memory. FAQ material.

Anders Berg




On Jul 17, 2006, at 21:02, Poul-Henning Kamp wrote:

> In message <B06E690C-CA93-4547-AD2A-61246521142D at vgnett.no>, Anders
> Berg writes
> :
>
>> The reason Anders N. asks about this is how Squid works today. The
>> squid.conf file leaves you with a option to specify how much RAM you
>> wanna use for Squid.
>
> And this is right where the trouble starts.
>
> Squid is written for a machine model that has not existed since
> 1980 when 3BSD was released.
>
> In that model, a process has some amount of "memory" and either all
> of that "memory" is present in RAM or none of it is. When RAM grew
> short, an entire process was swapped out (hence the name: "swap out
> one process for another")


andersb at vgnett

Jul 17, 2006, 1:47 PM

Post #15 of 16 (253 views)
Permalink
Controlling memory usage [In reply to]

On Jul 17, 2006, at 21:34, Dag-Erling Sm?rgrav wrote:

> Anders Berg <andersb at vgnett.no> writes:
>> The reason Anders N. asks about this is how Squid works today. The
>> squid.conf file leaves you with a option to specify how much RAM you
>> wanna use for Squid.
>
> I've worked with Squid quite a lot myself, and IIRC it normally uses
> about four times as much as you specify in the config file...
>
>> The problem is that Squid (probably because of
>> old design) does not really "follow" that option. If you set 256 MB
>> RAM, you can still end up using 500 MB RAM, and if you set option
>> close to max memory you will for certain overflow your physical RAM
>> and start swapping/die. Your answer was detailed Poul-Henning, but
>> what will prevent this from happpening in Varnish? Lets say you have
>> 2 applications running on a Varnish box, and both use the memory
>> model Varnish uses, what will happen in the long run with a lot of
>> traffic? Will they both adjust themselvs to match pysical RAM, how
>> would they "compete" for RAM etc? It's possible you have already
>> answered that below, but how do we "limit" RAM usage?
>
> We don't. The kernel takes care of it. If memory is scarce, the
> kernel will allocate less memory to the buffer cache and Varnish
> performance will decrease.
>
> (assuming you use file storage, not malloc storage)
>
> As for heap usage, which the kernel has less control over, it is
> mostly a function of the number of objects in the cache; we can take
> care of that by setting an upper limit on the number of cached
> objects, or even on the amount of heap used to track objects.

Thx for input DES. Setting an upper limit either in objects or heap
is a good way to control memory usage (And the right one I guess).
Are we gonna implement this?

Anders Berg



> DES
> --
> Dag-Erling Sm?rgrav
> Senior Software Developer
> Linpro AS - www.linpro.no
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-dev
>


andersb at vgnett

Jul 17, 2006, 1:47 PM

Post #16 of 16 (251 views)
Permalink
Controlling memory usage [In reply to]

On Jul 17, 2006, at 21:34, Dag-Erling Sm?rgrav wrote:

> Anders Berg <andersb at vgnett.no> writes:
>> The reason Anders N. asks about this is how Squid works today. The
>> squid.conf file leaves you with a option to specify how much RAM you
>> wanna use for Squid.
>
> I've worked with Squid quite a lot myself, and IIRC it normally uses
> about four times as much as you specify in the config file...
>
>> The problem is that Squid (probably because of
>> old design) does not really "follow" that option. If you set 256 MB
>> RAM, you can still end up using 500 MB RAM, and if you set option
>> close to max memory you will for certain overflow your physical RAM
>> and start swapping/die. Your answer was detailed Poul-Henning, but
>> what will prevent this from happpening in Varnish? Lets say you have
>> 2 applications running on a Varnish box, and both use the memory
>> model Varnish uses, what will happen in the long run with a lot of
>> traffic? Will they both adjust themselvs to match pysical RAM, how
>> would they "compete" for RAM etc? It's possible you have already
>> answered that below, but how do we "limit" RAM usage?
>
> We don't. The kernel takes care of it. If memory is scarce, the
> kernel will allocate less memory to the buffer cache and Varnish
> performance will decrease.
>
> (assuming you use file storage, not malloc storage)
>
> As for heap usage, which the kernel has less control over, it is
> mostly a function of the number of objects in the cache; we can take
> care of that by setting an upper limit on the number of cached
> objects, or even on the amount of heap used to track objects.

Thx for input DES. Setting an upper limit either in objects or heap
is a good way to control memory usage (And the right one I guess).
Are we gonna implement this?

Anders Berg



> DES
> --
> Dag-Erling Sm?rgrav
> Senior Software Developer
> Linpro AS - www.linpro.no
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-dev
>

Varnish dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.