Mailing List Archive: Zope: Users

Fwd: Building a fast, scalable yet small Zope application

Index | Next | Previous | View Threaded

do3ccqrv at googlemail

Apr 25, 2009, 6:45 AM

Post #1 of 7 (1524 views)
Permalink

Fwd: Building a fast, scalable yet small Zope application

Argl, why does this mailing list not use the reply-to: field?

---------- Forwarded message ----------
From: Patrick Gerken <do3ccqrv [at] googlemail>
Date: Sat, Apr 25, 2009 at 15:43
Subject: Re: [Zope] Building a fast, scalable yet small Zope application
To: "Morten W. Petersen" <morten [at] nidelven-it>

On Sat, Apr 25, 2009 at 13:24, Morten W. Petersen <morten [at] nidelven-it> wrote:
> Hi,
>
> I'm considering building a large scale, but small in features site. It
> will contain
> lots of small objects (millions, tens of millions, hundreds of millions)
> of objects,
> where each object has a couple of strings and maybe some other light
> attributes.
>
> So far, I've been contemplating disabling undo (if that's possible), and
> using
> BTree structures, maybe segmenting objects into different groups
> (folders) to
> further speed up lookups. Scalability is also an issue, should I
> consider using
> RelStorage? Should I consider using the ZCatalog for faster lookups?
>
> Has anyone else developed something similar? Are there Zope product
> examples out there that fit the bill?

Hi, I once was involved in developing a system that supposedly should
be able to handle a very big number of objects. They chose to put all
the base objects into mysql too, and use mysql for object lookup. I do
not know what type of benchmarks they did to conclude that this was
the best choice. But they regulary did queries for which relational
databases were made.
If your data contains lots of small items, that are all similar of
content, and you want to do queries that involves many of these
attributes, you might want to take a look at orm tools.
relstorage will not help you with your queries, because it stores
everything as a pickle, afaik.

I suggest you experiment a bit. Create 100 million objects, and do
some of the actions you are planning to do on them.

Best regards,

Patrick
_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

tseaver at palladion

Apr 26, 2009, 10:45 AM

Post #2 of 7 (1428 views)
Permalink

Re: Fwd: Building a fast, scalable yet small Zope application [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Patrick Gerken wrote:

> Argl, why does this mailing list not use the reply-to: field?

That would be a "religious" question, and one which is not helpful to
debate here.. E.g., see:

http://www.unicom.com/pw/reply-to-harmful.html

The Zope mailing lists are *all* organized that way, because the admins
/ long-time users (including myself) are adherents of the "munging
considered harmful" denomination.

Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver [at] palladion
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ9J3E+gerLs4ltQ4RAnIyAKDAsEHOn49A4rY7S5GZ84NCbypytACfdNGr
yXP1QjIwUofepjFrwccbwjU=
=eapg
-----END PGP SIGNATURE-----

_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

garito at sistes

Apr 26, 2009, 11:31 AM

Post #3 of 7 (1428 views)
Permalink

Re: Fwd: Building a fast, scalable yet small Zope application [In reply to]

jajajajajajajajajajajajaja

Amazing!

I don't know how the other list doesn't collapse using reply-to field
jajajajajajajajajajajajaja

What wonders me more is how belligerent some in this list are with this
question

You cause the problem, at least try to don't threaten us with ban us if we
send you private messages by mistake

This kind of things are to don't take you seriously, sorry, but this cause a
very bad impression of the people of this list

2009/4/26 Tres Seaver <tseaver [at] palladion>

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Patrick Gerken wrote:
>
> > Argl, why does this mailing list not use the reply-to: field?
>
> That would be a "religious" question, and one which is not helpful to
> debate here.. E.g., see:
>
> http://www.unicom.com/pw/reply-to-harmful.html
>
> The Zope mailing lists are *all* organized that way, because the admins
> / long-time users (including myself) are adherents of the "munging
> considered harmful" denomination.
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver +1 540-429-0999 tseaver [at] palladion
> Palladion Software "Excellence by Design" http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFJ9J3E+gerLs4ltQ4RAnIyAKDAsEHOn49A4rY7S5GZ84NCbypytACfdNGr
> yXP1QjIwUofepjFrwccbwjU=
> =eapg
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Zope maillist - Zope [at] zope
> http://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope-dev )
>

--
Mis Cosas
http://blogs.sistes.net/Garito
Zope Smart Manager
http://blogs.sistes.net/Garito/670

do3ccqrv at googlemail

Apr 26, 2009, 1:02 PM

Post #4 of 7 (1437 views)
Permalink

Re: Fwd: Building a fast, scalable yet small Zope application [In reply to]

On Sun, Apr 26, 2009 at 19:45, Tres Seaver <tseaver [at] palladion> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Patrick Gerken wrote:
>
>> Argl, why does this mailing list not use the reply-to: field?
>
> That would be a "religious" question, and one which is not helpful to
> debate here.. E.g., see:
>
> http://www.unicom.com/pw/reply-to-harmful.html

Thank you for that explanation.
That makes sense.

Best regards,

Patrick
_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

tseaver at palladion

Apr 27, 2009, 7:47 AM

Post #5 of 7 (1425 views)
Permalink

Re: Fwd: Building a fast, scalable yet small Zope application [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Garito wrote:
> jajajajajajajajajajajajaja
>
> Amazing!
>
> I don't know how the other list doesn't collapse using reply-to field
> jajajajajajajajajajajajaja

I said "religious" because I meant that people have strong opinions
about which style they prefer, and are not going to change them because
others argue for the other style. I didn't say (or imply) that those
who disagree with me are wrong / evil / stupid. I *did* say that this
list is run according to the "no-munging" rules, and that it was a
deliberate choice, not subject to debate.

> What wonders me more is how belligerent some in this list are with this
> question

I don't see how my reply can be considered belligerent. Patrick asked
"why doesn't the list munge 'Reply-To'?", and I replied with a link to
an explanation, along with a statement that arguing the question would
not be fruitful.

> You cause the problem,

If you had actually read the link I sent, you would have learned about
the problems caused by munging 'Reply-To'. I have participated in
*lots* of mailing lists since the early nineties: those which do munge
the header are more painful, and routinely have "oops, that was supposed
to be a private reply" messages show up. The damage from such messages
is *much* higher than the occasional "forget to CC the list" stuff on
non-munging lists. Again, this is *opinion,* shared by lots of mailing
list maangers / users, but certainly not by all.

> at least try to don't threaten us with ban us if we
> send you private messages by mistake

I don't know what you are talking about here: I didn't threaten to ban
anybody. I generally ask (politely) that people keep their replies
on-list, and assume that the occasional off-list reply is an honest
mistake. I *have* set up filters (in my own mail processing) dropping
messages from a couple of users who persist in sending me private mail
after being asked to keep the traffic on the list.

> This kind of things are to don't take you seriously, sorry, but this cause a
> very bad impression of the people of this list

I'm sorry you feel that way.

Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver [at] palladion
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ9cVx+gerLs4ltQ4RAoCdAJ9xOVCuNjNoxf5uMgTeaSQhiaXMNwCgpnvR
GHTZGcy1s0hP1/HRJPxX5xE=
=shhQ
-----END PGP SIGNATURE-----

_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

garito at sistes

Apr 27, 2009, 4:58 PM

Post #6 of 7 (1420 views)
Permalink

Re: Fwd: Building a fast, scalable yet small Zope application [In reply to]

2009/4/27 Tres Seaver <tseaver [at] palladion>

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Garito wrote:
> > jajajajajajajajajajajajaja
> >
> > Amazing!
> >
> > I don't know how the other list doesn't collapse using reply-to field
> > jajajajajajajajajajajajaja
>
> I said "religious" because I meant that people have strong opinions
> about which style they prefer, and are not going to change them because
> others argue for the other style. I didn't say (or imply) that those
> who disagree with me are wrong / evil / stupid. I *did* say that this
> list is run according to the "no-munging" rules, and that it was a
> deliberate choice, not subject to debate.

Making joke, man, that's the point about religious, isn't it?

>
>
> > What wonders me more is how belligerent some in this list are with this
> > question
>
> I don't see how my reply can be considered belligerent. Patrick asked
> "why doesn't the list munge 'Reply-To'?", and I replied with a link to
> an explanation, along with a statement that arguing the question would
> not be fruitful.

I mix other conversations with that

>
>
> > You cause the problem,
>
> If you had actually read the link I sent, you would have learned about
> the problems caused by munging 'Reply-To'. I have participated in
> *lots* of mailing lists since the early nineties: those which do munge
> the header are more painful, and routinely have "oops, that was supposed
> to be a private reply" messages show up. The damage from such messages
> is *much* higher than the occasional "forget to CC the list" stuff on
> non-munging lists. Again, this is *opinion,* shared by lots of mailing
> list maangers / users, but certainly not by all.
>
> > at least try to don't threaten us with ban us if we
> > send you private messages by mistake
>
> I don't know what you are talking about here: I didn't threaten to ban
> anybody. I generally ask (politely) that people keep their replies
> on-list, and assume that the occasional off-list reply is an honest
> mistake. I *have* set up filters (in my own mail processing) dropping
> messages from a couple of users who persist in sending me private mail
> after being asked to keep the traffic on the list.

There are people that response very rude to a mistake private message

>
>
> > This kind of things are to don't take you seriously, sorry, but this
> cause a
> > very bad impression of the people of this list
>
> I'm sorry you feel that way.

I respect the opinions, yes, nothing to say about that. I respect you, all
of you, sometimes the email is not a good way to express. I'm a little bit
critic with some of the decisions you make because I think that Zope is the
best operating system (yes, I said operating system) someone could need

I think I need to be as critic because when you decide to make a step far
the vision of Zope I see more work I need to do for myself
I see Zope as a very very smart idea, a univers for my data in a very
similar way as a human brain seems to work and when I see how you invest
your time in another direction I sufer a lot

But, yes, I know that perhaps my vision of Zope is not like yours but it
was, at least when I know Zope the people publish some products with the
same vision

In that point I want to apologize. If I never offend you: sorry, nothing
personal

>
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver +1 540-429-0999 tseaver [at] palladion
> Palladion Software "Excellence by Design" http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFJ9cVx+gerLs4ltQ4RAoCdAJ9xOVCuNjNoxf5uMgTeaSQhiaXMNwCgpnvR
> GHTZGcy1s0hP1/HRJPxX5xE=
> =shhQ
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Zope maillist - Zope [at] zope
> http://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope-dev )
>

--
Mis Cosas
http://blogs.sistes.net/Garito
Zope Smart Manager
http://blogs.sistes.net/Garito/670

tseaver at palladion

Apr 28, 2009, 6:31 AM

Post #7 of 7 (1403 views)
Permalink

Re: Fwd: Building a fast, scalable yet small Zope application [In reply to]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Garito wrote:
> 2009/4/27 Tres Seaver <tseaver [at] palladion>

<snip>

>>>> at least try to don't threaten us with ban us if we
>>>> send you private messages by mistake
> I don't know what you are talking about here: I didn't threaten to ban
> anybody. I generally ask (politely) that people keep their replies
> on-list, and assume that the occasional off-list reply is an honest
> mistake. I *have* set up filters (in my own mail processing) dropping
> messages from a couple of users who persist in sending me private mail
> after being asked to keep the traffic on the list.
>
>
>> There are people that response very rude to a mistake private message

Aside from a couple of recent flare-ups, the traffic on this list has
been remarkable for its civility over time (I've been reading the list
for ten years now). I'm sorry you have experienced rudeness, which I
don't find acceptable. I quite like the Ubuntu Code of Conduct[1], and
think we could all do to emulate it here.

>>>> This kind of things are to don't take you seriously, sorry, but this
> cause a
>>>> very bad impression of the people of this list
> I'm sorry you feel that way.
>
>
>> I respect the opinions, yes, nothing to say about that. I respect you, all
>> of you, sometimes the email is not a good way to express. I'm a little bit
>> critic with some of the decisions you make because I think that Zope is the
>> best operating system (yes, I said operating system) someone could need
>
>> I think I need to be as critic because when you decide to make a step far
>> the vision of Zope I see more work I need to do for myself
>> I see Zope as a very very smart idea, a univers for my data in a very
>> similar way as a human brain seems to work and when I see how you invest
>> your time in another direction I sufer a lot
>
>> But, yes, I know that perhaps my vision of Zope is not like yours but it
>> was, at least when I know Zope the people publish some products with the
>> same vision
>
>> In that point I want to apologize. If I never offend you: sorry, nothing
>> personal

OK, no worries.

[1] http://www.ubuntu.com/community/conduct

Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver [at] palladion
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ9wVJ+gerLs4ltQ4RAq/qAJ9O2B1nxthBYGoo+NXYGaRrqb98CACfSfsj
/w1UTQ97zaKnSlyxx8J+ct8=
=mL7R
-----END PGP SIGNATURE-----

_______________________________________________
Zope maillist - Zope [at] zope
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads