Mailing List Archive: Maemo: Developers

maemo-optify, autobuilder & /opt

1 2 3 4

View All

Index | Next | Previous | View Threaded

bartosh at gmail

Nov 8, 2009, 3:43 AM

Post #76 of 88 (1219 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

2009/11/6 Marius Vollmer <marius.vollmer [at] nokia>:
> ext Ed Bartosh <bartosh [at] gmail> writes:
>
>> I've discussed this with sbdmock author and we decided to make small
>> change to sbdmock: New configurable action will be introduced. This
>> action will be executed by sbdmock between unpacking rootstrap and
>> installing build-deps.
>>
>> For Fremantle we can simply put 'apt-get install maemo-optify' in there.
>>
>> I'll try to make this change on weekends and deploy new sbdmock and
>> devkit in production.
>
> Excellent, thanks!
>
Done.

Here you can see my test builds:
Fremantle builds:
without debian/optify: https://garage.maemo.org/builder/fremantle/sbdmock_0.4.4/
'auto' in debian/optify:
https://garage.maemo.org/builder/fremantle/sbdmock_0.4.4.optify/

Diablo build:
with 'auto' in debian/optify:
https://garage.maemo.org/builder/diablo/sbdmock_0.4.4/

Looking forward to your feedback and further instructions

--
BR,
Ed
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

marius.vollmer at nokia

Nov 8, 2009, 11:01 PM

Post #77 of 88 (1207 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

ext Ed Bartosh <bartosh [at] gmail> writes:

>> Maemo-optify can be invoked in a number of ways. I'll explain what
>> happens when the modified dpkg-buildpackage calls
>>
>> maemo-optify-deb --auto
>
> Which is how it's used in modified dpkg-buildpackage, right?

Correct.

>> after running ./debian/rules build.
>>
>>> - source package doesn't contain anything /opt specific, binary
>>> package doesn't contain /opt
>>
>> Nothing will be done because debian/optify does not exist, and the
>> default mode is "none".
>
> So, after deployment nothing should change unless developers add
> debian/optify to their packages, right?

Correct.

> When autobuilder expected to start to optify packages without
> debian/optify in them?

I don't know. We certainly need to tune the heuristic of maemo-optify
first to handle Python.
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

bartosh at gmail

Nov 9, 2009, 12:33 AM

Post #78 of 88 (1208 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

2009/11/9 Marius Vollmer <marius.vollmer [at] nokia>:
>
>> When autobuilder expected to start to optify packages without
>> debian/optify in them?
>
> I don't know. We certainly need to tune the heuristic of maemo-optify
> first to handle Python.
>
Just in case you need my help. I'm here for this week. My 2 weeks
vacation starts on Satuday.

--
BR,
Ed
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

bartosh at gmail

Dec 2, 2009, 5:02 AM

Post #79 of 88 (1080 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

2009/11/9 Marius Vollmer <marius.vollmer [at] nokia>:

>> When autobuilder expected to start to optify packages without
>> debian/optify in them?
>
> I don't know. We certainly need to tune the heuristic of maemo-optify
> first to handle Python.
>
As far as I see Python is optified now. When we should do next step?

--
BR,
Ed
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

anderson.lizardo at openbossa

Dec 2, 2009, 5:12 AM

Post #80 of 88 (1089 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

On Wed, Dec 2, 2009 at 9:02 AM, Ed Bartosh <bartosh [at] gmail> wrote:
> 2009/11/9 Marius Vollmer <marius.vollmer [at] nokia>:
>
>>> When autobuilder expected to start to optify packages without
>>> debian/optify in them?
>>
>> I don't know. We certainly need to tune the heuristic of maemo-optify
>> first to handle Python.
>>
> As far as I see Python is optified now. When we should do next step?

Correct. But, in Python's case, we didn't add such debian/optify file,
and we relied on the fact that the (current) default optify policy was
"none".

If you have plans to begin enabling auto-optification by default,
please inform us here on the list so we can begin adding the
debian/optify file to avoid optifying packages that were manually
optified by other means (e.g. python packages).

Regards,
--
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

bartosh at gmail

Dec 2, 2009, 5:43 AM

Post #81 of 88 (1082 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

2009/12/2 Anderson Lizardo <anderson.lizardo [at] openbossa>:
> On Wed, Dec 2, 2009 at 9:02 AM, Ed Bartosh <bartosh [at] gmail> wrote:
>> 2009/11/9 Marius Vollmer <marius.vollmer [at] nokia>:
>>
>>>> When autobuilder expected to start to optify packages without
>>>> debian/optify in them?
>>>
>>> I don't know. We certainly need to tune the heuristic of maemo-optify
>>> first to handle Python.
>>>
>> As far as I see Python is optified now. When we should do next step?
>
> Correct. But, in Python's case, we didn't add such debian/optify file,
> and we relied on the fact that the (current) default optify policy was
> "none".
>
> If you have plans to begin enabling auto-optification by default,
> please inform us here on the list so we can begin adding the
> debian/optify file to avoid optifying packages that were manually
> optified by other means (e.g. python packages).
>

I hope you'll see everything in this thread.

--
BR,
Ed
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

anderson.lizardo at openbossa

Dec 2, 2009, 5:50 AM

Post #82 of 88 (1094 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

On Wed, Dec 2, 2009 at 9:43 AM, Ed Bartosh <bartosh [at] gmail> wrote:
> 2009/12/2 Anderson Lizardo <anderson.lizardo [at] openbossa>:
>> If you have plans to begin enabling auto-optification by default,
>> please inform us here on the list so we can begin adding the
>> debian/optify file to avoid optifying packages that were manually
>> optified by other means (e.g. python packages).
>>
>
> I hope you'll see everything in this thread.

Yes, I read the thread and I know it *will* be enabled by default, but
I ask *when* because last time I checked it was still not enabled by
default. (sorry because my e-mail did not make that clear).

I think a simple "heads up" e-mail would be enough, because many
people might have forgotten this rather old and long thread by now.

Thanks,
--
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

marius.vollmer at nokia

Dec 2, 2009, 8:56 AM

Post #83 of 88 (1082 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

ext Anderson Lizardo <anderson.lizardo [at] openbossa> writes:

> If you have plans to begin enabling auto-optification by default,
> please inform us here on the list so we can begin adding the
> debian/optify file to avoid optifying packages that were manually
> optified by other means (e.g. python packages).

If a package already contains a /opt directory, no further optification
is done by the maemo-optify tools in any case. Is that enough to
protect you?

We can put additional checks into maemo-optify to disable optification.
I don't know enough about Python packages to suggest a good one, but
maybe just looking for /usr/lib/py* in a package and stopping to do
anything would work.

Please suggest such an additional check if you want it and I'll add it,
of course.
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

anderson.lizardo at openbossa

Dec 2, 2009, 9:55 AM

Post #84 of 88 (1091 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

On Wed, Dec 2, 2009 at 12:56 PM, Marius Vollmer
<marius.vollmer [at] nokia> wrote:
> ext Anderson Lizardo <anderson.lizardo [at] openbossa> writes:
>
>> If you have plans to begin enabling auto-optification by default,
>> please inform us here on the list so we can begin adding the
>> debian/optify file to avoid optifying packages that were manually
>> optified by other means (e.g. python packages).
>
> If a package already contains a /opt directory, no further optification
> is done by the maemo-optify tools in any case. Is that enough to
> protect you?

No, because the optification was done by creating a "pymaemo-optify"
package which bind mounts the relevant Python directories from
/usr/... to /opt/pymaemo. These mounts are done during device boot by
/etc/init.d/pymaemo-optify (they are not mounted inside scratchbox, of
course).

This means that there is no change required for python packages to
become optified, they just need to depend on python.

> We can put additional checks into maemo-optify to disable optification.
> I don't know enough about Python packages to suggest a good one, but
> maybe just looking for /usr/lib/py* in a package and stopping to do
> anything would work.

Actually the list of directories handled by pymaemo-optify is variable
(but not expected to change). They are defined in
/etc/default/pymaemo-optify and currently is:

/usr/lib/python2.5
/usr/share/pyshared
/usr/lib/pyshared
/usr/share/python-support
/usr/lib/python-support

One idea might be to check whether this file exists and ignore any
packages which install files to the directories listed on the
"BIND_MOUNTS" directory (note that this file is a snippet of shell
script read by /etc/init.d/pymaemo-optify).

Or you can hard code this list somewhere, and you will get at least
most Python packages ignored. As with every heuristics, we can't get
100% of cases, and others will need to have to be manually opted-out.

My two cents,
--
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

nathan at andersonsplace

Dec 2, 2009, 1:20 PM

Post #85 of 88 (1080 views)
Permalink

RE: maemo-optify, autobuilder & /opt [In reply to]

Anderson Lizardo,

Unless I misunderstood; if the package itself has a /opt path in it
the maemo-optify won't run on it. So if you are "installing" anything
(even one file) under the /opt path (which based on what I see you saying
below it appears you are); it should cause the maemo-optify to not run.

Nathan Anderson

-----Original Message-----
From: maemo-developers-bounces [at] maemo
[mailto:maemo-developers-bounces [at] maemo] On Behalf Of Anderson Lizardo
Sent: Wednesday, December 02, 2009 11:56 AM
To: Marius Vollmer
Cc: maemo-developers [at] maemo
Subject: Re: maemo-optify, autobuilder & /opt

On Wed, Dec 2, 2009 at 12:56 PM, Marius Vollmer <marius.vollmer [at] nokia>
wrote:
> ext Anderson Lizardo <anderson.lizardo [at] openbossa> writes:
>
>> If you have plans to begin enabling auto-optification by default,
>> please inform us here on the list so we can begin adding the
>> debian/optify file to avoid optifying packages that were manually
>> optified by other means (e.g. python packages).
>
> If a package already contains a /opt directory, no further
> optification is done by the maemo-optify tools in any case. Is that
> enough to protect you?

No, because the optification was done by creating a "pymaemo-optify"
package which bind mounts the relevant Python directories from /usr/... to
/opt/pymaemo. These mounts are done during device boot by
/etc/init.d/pymaemo-optify (they are not mounted inside scratchbox, of
course).

This means that there is no change required for python packages to become
optified, they just need to depend on python.

> We can put additional checks into maemo-optify to disable optification.
> I don't know enough about Python packages to suggest a good one, but
> maybe just looking for /usr/lib/py* in a package and stopping to do
> anything would work.

Actually the list of directories handled by pymaemo-optify is variable (but
not expected to change). They are defined in /etc/default/pymaemo-optify and
currently is:

/usr/lib/python2.5
/usr/share/pyshared
/usr/lib/pyshared
/usr/share/python-support
/usr/lib/python-support

One idea might be to check whether this file exists and ignore any packages
which install files to the directories listed on the "BIND_MOUNTS" directory
(note that this file is a snippet of shell script read by
/etc/init.d/pymaemo-optify).

Or you can hard code this list somewhere, and you will get at least most
Python packages ignored. As with every heuristics, we can't get 100% of
cases, and others will need to have to be manually opted-out.

My two cents,
--
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

anderson.lizardo at openbossa

Dec 2, 2009, 2:40 PM

Post #86 of 88 (1082 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

On Wed, Dec 2, 2009 at 5:20 PM, Nathan Anderson
<nathan [at] andersonsplace> wrote:
> Anderson Lizardo,
>
> Unless I misunderstood; if the package itself has a /opt path in it
> the maemo-optify won't run on it. So if you are "installing" anything
> (even one file) under the /opt path (which based on what I see you saying
> below it appears you are); it should cause the maemo-optify to not run.

Not really. All Python packages (unless they were manually optified)
continue to install files under /usr (you can check that with e.g.
"dpkg -L python2.5"), but given that pymaemo-optify bind mount
directories with a command like:

mount --bind /opt/pymaemo/usr/lib/python2.5 /usr/lib/python2.5

All files installed under e.g. /usr/lib/python2.5 go "automatically"
to /opt. But note that the package itself is unchanged (because
pymaemo-optify takes care of handling these mount binds), so there is
no way for maemo-optify to know whether to optify some Python package
simply by looking at where it installs files.

Regards,
--
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

andrew at bleb

Dec 3, 2009, 2:46 AM

Post #87 of 88 (1068 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

2009/12/2 Anderson Lizardo <anderson.lizardo [at] openbossa>:
>
> All files installed under e.g. /usr/lib/python2.5 go "automatically"
> to /opt. But note that the package itself is unchanged (because
> pymaemo-optify takes care of handling these mount binds), so there is
> no way for maemo-optify to know whether to optify some Python package
> simply by looking at where it installs files.

Short version of the required heuristics for NOT invoking maemo-optify:

* any package including /opt
* any package with debian/optify containing 'none'
* any package with a direct, or indirect, dependency on pymaemo-optify.

Any others?

Cheers,

Andrew

--
Andrew Flegg -- mailto:andrew [at] bleb | http://www.bleb.org/
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

anderson.lizardo at openbossa

Dec 3, 2009, 5:28 AM

Post #88 of 88 (1065 views)
Permalink

Re: maemo-optify, autobuilder & /opt [In reply to]

On Thu, Dec 3, 2009 at 6:46 AM, Andrew Flegg <andrew [at] bleb> wrote:
> 2009/12/2 Anderson Lizardo <anderson.lizardo [at] openbossa>:
>>
>> All files installed under e.g. /usr/lib/python2.5 go "automatically"
>> to /opt. But note that the package itself is unchanged (because
>> pymaemo-optify takes care of handling these mount binds), so there is
>> no way for maemo-optify to know whether to optify some Python package
>> simply by looking at where it installs files.
>
> Short version of the required heuristics for NOT invoking maemo-optify:
>
> * any package including /opt
> * any package with debian/optify containing 'none'
> * any package with a direct, or indirect, dependency on pymaemo-optify.

That "indirect dependency" part may be tricky to implement, maybe just
check for dependency on python or python2.5 ?

Regards,
--
Anderson Lizardo
OpenBossa Labs - INdT
Manaus - Brazil
_______________________________________________
maemo-developers mailing list
maemo-developers [at] maemo
https://lists.maemo.org/mailman/listinfo/maemo-developers

1 2 3 4

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads