Mailing List Archive: Zope: Users

Re: one zope instance per cpu

Index | Next | Previous | View Threaded

tseaver at palladion

May 13, 2008, 10:29 PM

Post #1 of 3 (377 views)
Permalink

Re: one zope instance per cpu

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Pat McNamee wrote:
>> Generally, the rule-of-thumb for scaling Zope on a multi-CPU box is to
>> run an appserver instance per CPU, each talking to a shared storage...
>
> I came across the preceeding post while googling for info about the
> one zope per cpu question.
>
> First, when you say one zope instance per cpu, do you count a quad-core
> cpu as four cpus?

Four: the idea is to split up the processes so that their threads make
best use of the available CPUs. Python's global interpreter lock makes
using more than one CPU within a single process problematic.

> Also, I've been (unsuccessfully) searching all over for an answer to
> the question: if you create one zope instance per cpu each talking
> to a shared zeo instance, how do the instances know about the cpus?
> Does each instance automagically use a separate cpu or what?

Modern linux kernels will tend to migrate all the processes to separate
CPUs (in steady state).

> I thank you in advance for your reply, and I hope I'm not being
> rude by emailing you instead of the Zope list. If this is rude
> please tell me and I'll post to the list.

CC'ing the list does leave the conversation open, and makes future
searches likelier to show the "whole picture".

Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver[at]palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKni9+gerLs4ltQ4RAg9rAJ4xaklSySaGqR2bdZhxE6RVL2qOPACdEqWO
lq2zMLDM4+G4q2WuqmAO2Gk=
=D5M6
-----END PGP SIGNATURE-----
_______________________________________________
Zope maillist - Zope[at]zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

KSours at carsdirect

May 14, 2008, 7:49 AM

Post #2 of 3 (357 views)
Permalink

RE: Re: one zope instance per cpu [In reply to]

> Four: the idea is to split up the processes so that their threads
make best
> use of the available CPUs. Python's global interpreter lock makes
using more
> than one CPU within a single process problematic.

In the multiprocess setup, is it better to run a single thread per
process or multiple threads? Seems like multiple threads would still
prevent long running requests from tying up a processor, but the
overhead may not be worth it. I haven't had a change to do any testing,
so I was wondering if there was a concensus.
Kevin

_______________________________________________
Zope maillist - Zope[at]zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

tseaver at palladion

May 14, 2008, 10:14 AM

Post #3 of 3 (354 views)
Permalink

Re: one zope instance per cpu [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sours, Kevin wrote:
>
>
>> Four: the idea is to split up the processes so that their threads
> make best
>> use of the available CPUs. Python's global interpreter lock makes
> using more
>> than one CPU within a single process problematic.
>
> In the multiprocess setup, is it better to run a single thread per
> process or multiple threads? Seems like multiple threads would still
> prevent long running requests from tying up a processor, but the
> overhead may not be worth it. I haven't had a change to do any testing,
> so I was wondering if there was a concensus.

I would still go with multiple threads, for the cases where the
application *does* release the GIL (e.g., making a call to a relational
database, etc.) The tradeoff is that you hit the ceiling on RAM faster
with more threads, so if your system is RAM-constrained, you should
perhaps consider reducing the number of threads.

Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver[at]palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKx3h+gerLs4ltQ4RAmiNAJ98AhMS8+pyi3p0xOjOVMVczxQGeACguaIE
NdL21l8nX3Y3NsKyNZevJxU=
=Z6LP
-----END PGP SIGNATURE-----

_______________________________________________
Zope maillist - Zope[at]zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com