Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Zope: Dev

zc.recipe.cmmi shared builds

 

 

Zope dev RSS feed   Index | Next | Previous | View Threaded


ws at gocept

Mar 24, 2009, 2:26 AM

Post #1 of 8 (1076 views)
Permalink
zc.recipe.cmmi shared builds

Hello,

I'd like to extend zc.recipe.cmmi to support shared build directories.

Use case example: we use lxml in a lot of our projects, which
currently means having to build libxml and libxslt over and over
again, since the buildout needs to be standalone and thus can't depend
on them being installed on the system. But while that's a necessity
for deployment, it's really annoying for development.

Plan of attack:
Introduce an option "shared" (defaults to False). If that is set,
a) calculate HASH as a hash of the recipe's current options (e. g.
configure parameters, environment variables)
b) perform the cmmi in ${buildout:download-cache}/cmmi/build/<HASH>,
if that directory is not present yet. (Probably needs a little thought
about how to differentiate between "present and has a complete build"
and "present but we errored out")

Any thoughts?

Thanks,
Wolfgang

--
Wolfgang Schnerring · ws [at] gocept
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1
Zope and Plone consulting and development
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


jens at bluedynamics

Mar 24, 2009, 5:51 AM

Post #2 of 8 (1023 views)
Permalink
Re: zc.recipe.cmmi shared builds [In reply to]

Am Tue, 24 Mar 2009 10:26:08 +0100 schrieb Wolfgang Schnerring:
> I'd like to extend zc.recipe.cmmi to support shared build directories.
>
> Use case example: we use lxml in a lot of our projects, which currently
> means having to build libxml and libxslt over and over again, since the
> buildout needs to be standalone and thus can't depend on them being
> installed on the system. But while that's a necessity for deployment,
> it's really annoying for development.

So build once and use often, we do this i.e. for ldap by compiling in a
separate buildout located in /opt/openldap and then have a section in the
specific buildout like:

[python-ldap]
recipe = zc.recipe.egg:custom
egg = python-ldap
include-dirs =
/opt/is/openldap/parts/open-ldap/include
library-dirs =
/opt/is/openldap/parts/open-ldap/lib
rpath =
/opt/is/openldap/parts/open-ldap/lib


> Plan of attack:
> Introduce an option "shared" (defaults to False). If that is set, a)
> calculate HASH as a hash of the recipe's current options (e. g.
> configure parameters, environment variables) b) perform the cmmi in
> ${buildout:download-cache}/cmmi/build/<HASH>, if that directory is not
> present yet. (Probably needs a little thought about how to differentiate
> between "present and has a complete build" and "present but we errored
> out")
>
> Any thoughts?

sounds good to me, it would simplify our infrastructure a lot. important
is the hash: it needs to include all parameters, environment et al passed
to the cmmi.

regards
Jens

--
Jens W. Klein - Klein & Partner KEG - BlueDynamics Alliance

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


me at rpatterson

Mar 24, 2009, 7:43 AM

Post #3 of 8 (1026 views)
Permalink
Re: zc.recipe.cmmi shared builds [In reply to]

Wolfgang Schnerring <ws [at] gocept> writes:

> I'd like to extend zc.recipe.cmmi to support shared build directories.
>
> Use case example: we use lxml in a lot of our projects, which
> currently means having to build libxml and libxslt over and over
> again, since the buildout needs to be standalone and thus can't depend
> on them being installed on the system. But while that's a necessity
> for deployment, it's really annoying for development.
>
> Plan of attack:
> Introduce an option "shared" (defaults to False). If that is set,
> a) calculate HASH as a hash of the recipe's current options (e. g.
> configure parameters, environment variables)
> b) perform the cmmi in ${buildout:download-cache}/cmmi/build/<HASH>,
> if that directory is not present yet. (Probably needs a little thought
> about how to differentiate between "present and has a complete build"
> and "present but we errored out")

This would be great. I'm always happy when a build recipe provides this
option so I think generalizing it is a good idea.

Ross

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


ws at gocept

Mar 24, 2009, 8:01 AM

Post #4 of 8 (1022 views)
Permalink
Re: zc.recipe.cmmi shared builds [In reply to]

* Wolfgang Schnerring <ws [at] gocept> [2009-03-24 10:26]:
> I'd like to extend zc.recipe.cmmi to support shared build directories.

Implemented in r98334. I'll use this locally over the next few days, and will
then try to push out a release if it proves stable.

Wolfgang

--
Wolfgang Schnerring · ws [at] gocept
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1
Zope and Plone consulting and development


_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


reinout at vanrees

Mar 24, 2009, 12:51 PM

Post #5 of 8 (1029 views)
Permalink
Re: zc.recipe.cmmi shared builds [In reply to]

On 2009-03-24, Wolfgang Schnerring <ws [at] gocept> wrote:
>
> I'd like to extend zc.recipe.cmmi to support shared build directories.
>
> Use case example: we use lxml in a lot of our projects, which
> currently means having to build libxml and libxslt over and over
> again, since the buildout needs to be standalone and thus can't depend
> on them being installed on the system. But while that's a necessity
> for deployment, it's really annoying for development.

I added a feature to http://pypi.python.org/pypi/z3c.recipe.staticlxml last
week that skips the expensive compilation step if there's already an lxml egg.

I could do that by using a couple of assumptions that were build into the
recipe. So if you need the shared build directory just for lxml, this could
save you some work.

The recipe uses zc.recipe.cmmi, though, if I remember correctly. So your
approach could fix it at the source instead of my one-recipe-only solution :-)


Reinout


--
Reinout van Rees - reinout [at] vanrees - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
"Military engineers build missiles. Civil engineers build targets

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


ws at gocept

May 12, 2009, 7:56 AM

Post #6 of 8 (780 views)
Permalink
Re: zc.recipe.cmmi shared builds [In reply to]

* Wolfgang Schnerring <ws [at] gocept> [2009-03-24 16:01]:
> * Wolfgang Schnerring <ws [at] gocept> [2009-03-24 10:26]:
>> I'd like to extend zc.recipe.cmmi to support shared build directories.
>
> Implemented in r98334. I'll use this locally over the next few days, and will
> then try to push out a release if it proves stable.

Looks good so far. Could someone do a release of zc.recipe.cmmi
(or grant 'wosc' pypi access so I can do it myself)?

Thanks,
Wolfgang

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


fdrake at gmail

May 16, 2009, 5:09 AM

Post #7 of 8 (780 views)
Permalink
Re: zc.recipe.cmmi shared builds [In reply to]

On Tue, May 12, 2009 at 10:56 AM, Wolfgang Schnerring <ws [at] gocept> wrote:
> Looks good so far. Could someone do a release of zc.recipe.cmmi
> (or grant 'wosc' pypi access so I can do it myself)?

Done.


-Fred

--
Fred L. Drake, Jr. <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller
_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )


ws at gocept

May 17, 2009, 11:36 PM

Post #8 of 8 (764 views)
Permalink
Re: zc.recipe.cmmi shared builds [In reply to]

* Fred Drake <fdrake [at] gmail> [2009-05-16 14:09]:
> On Tue, May 12, 2009 at 10:56 AM, Wolfgang Schnerring <ws [at] gocept> wrote:
>> Looks good so far. Could someone do a release of zc.recipe.cmmi
>> (or grant 'wosc' pypi access so I can do it myself)?
>
> Done.

Thanks! I've just cut the 1.2.0 release that includes the shared build
directory functionality.

Wolfgang

_______________________________________________
Zope-Dev maillist - Zope-Dev [at] zope
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Zope dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.